Skip to content

docs(extraction): pass NVIDIA_API_KEY explicitly in Live RAG examples (NVBug 6622594) - #2575

Open
kheiss-uwzoo wants to merge 2 commits into
NVIDIA:mainfrom
kheiss-uwzoo:docs/6622594-live-rag-nvidia-api-key
Open

docs(extraction): pass NVIDIA_API_KEY explicitly in Live RAG examples (NVBug 6622594)#2575
kheiss-uwzoo wants to merge 2 commits into
NVIDIA:mainfrom
kheiss-uwzoo:docs/6622594-live-rag-nvidia-api-key

Conversation

@kheiss-uwzoo

Copy link
Copy Markdown
Collaborator

Summary

  • Live RAG LiteLLMClient / LLMJudge examples now pass api_key="os.environ/NVIDIA_API_KEY" so the default nvidia_nim model receives the hosted-inference key.
  • Published API-key and Python API pages state that LiteLLM does not read NVIDIA_API_KEY for nvidia_nim; omit-api_key looks up NVIDIA_NIM_API_KEY.
  • Docs-only. The LiteLLMClient class docstring still lists NVIDIA_API_KEY as auto-read (litellm.py:57-58); that is source drift for a follow-up eng change.

Fixes NVBug 6622594.

Test plan

  • Copy the Live RAG README example with only NVIDIA_API_KEY set and confirm the request is sent (not error='transport_error' from a missing LiteLLM key name)
  • Confirm api_key="os.environ/NVIDIA_API_KEY" still matches the working OpenAI example earlier on the same README
  • Confirm the PR diff is only the three documentation files

pre-draft: leakage, mkdocs --strict, ::a, ::p, ::r on the diff vs main

Base: upstream/main
Files: docs/docs/extraction/api-keys.md, docs/docs/extraction/nemo-retriever-api-reference.md, nemo_retriever/README.md

Check Result
Leakage (page roles + see [ CTAs) PASS — no see [ CTAs on these three files; no nimOperator / nvcr.io/nim / installFfmpeg on faq.md, overview.md, or multimodal-extraction.md. Leakage script vs origin/main reported leftover untracked custom-metadata.md (not in this diff; fork origin/main is behind upstream/main).
Allowed paths PASS — 3 documentation files
mkdocs --strict PASS for this change — exit 1 from untracked leftover pages custom-metadata.md and user-defined-stages.md, which are not in this diff
::a audit PASS — 4 claims at ~95%: os.environ/NVIDIA_API_KEY resolves at call time (resolve_environment_reference, LiteLLMClient.complete, test_llm_params.py); default models are nvidia_nim/... (litellm.py:70, judge.py:237); omit-api_key is provider-native (LLMRemoteClientParams); nvidia_nim lookup name is NVIDIA_NIM_API_KEY (evaluation README). Dropped NVIDIA_NIM_ADMIN_KEY (no in-repo evidence).
::p polish Applied — aligned README wording with api-keys; used "worker" instead of "library"; removed unverified admin-key name
::r style 95% — no blocking issues on the new prose

Code drift (not in this docs PR): nemo_retriever/src/nemo_retriever/models/llm/clients/litellm.py:57-58 still lists NVIDIA_API_KEY as auto-read. Evaluation README LLMJudge nvidia_nim example still omits api_key. transport_error masking is a separate product defect.

PR: this draft

… (NVBug 6622594)

LiteLLM nvidia_nim does not read NVIDIA_API_KEY from the environment, so the README and API-key docs now pass os.environ/NVIDIA_API_KEY.
@kheiss-uwzoo kheiss-uwzoo self-assigned this Aug 24, 2026
@kheiss-uwzoo kheiss-uwzoo added 26.08 doc Improvements or additions to documentation labels Aug 24, 2026
@kheiss-uwzoo
kheiss-uwzoo marked this pull request as ready for review August 24, 2026 17:02
@kheiss-uwzoo
kheiss-uwzoo requested review from a team as code owners August 24, 2026 17:02
@kheiss-uwzoo
kheiss-uwzoo requested review from jioffe502 and jperez999 and removed request for jioffe502 August 24, 2026 17:02
@greptile-apps

greptile-apps Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR updates Live RAG documentation to pass NVIDIA_API_KEY explicitly to LiteLLM-backed NVIDIA NIM clients.

  • Adds explicit environment-key references to the LiteLLMClient and LLMJudge examples.
  • Documents LiteLLM’s provider-native key lookup behavior when api_key is omitted.
  • Cross-links the credential guidance from the Python API reference.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
docs/docs/extraction/api-keys.md Clarifies the explicit credential reference required by LiteLLM-backed NVIDIA NIM models.
docs/docs/extraction/nemo-retriever-api-reference.md Adds credential guidance for default LiteLLMClient and LLMJudge models in persisted configurations.
nemo_retriever/README.md Updates both Live RAG examples to forward the configured NVIDIA API key explicitly.

Reviews (2): Last reviewed commit: "Merge branch 'main' into docs/6622594-li..." | Re-trigger Greptile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

26.08.1 doc Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant