docs: describe how to add a core config option and CLI switch - #1999
Open
ehtishammubarik wants to merge 1 commit into
Open
docs: describe how to add a core config option and CLI switch#1999ehtishammubarik wants to merge 1 commit into
ehtishammubarik wants to merge 1 commit into
Conversation
Adds a reference page covering the steps to add a value to garak's core configuration and expose it on the command line: the core config default, the section's *_params entry, the argparse definition, and the test hooks. Also covers adding a new config section, and notes that cliref.rst is generated. Registers the page in the index toctree and links it from extending. Closes NVIDIA#385 Signed-off-by: ehtishammubarik <ehtisham.m7@gmail.com>
Collaborator
|
Can you fill in a blank for me - why would there be such a code path? |
Author
That path is intentional. The issue documented here is that a genuine config option accidentally omitted from its |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #385
There's currently no documented path for adding a value to garak's core configuration and
exposing it on the command line. The reference docs cover configuration from the user side
(
configurable), and plugin authoring viaDEFAULT_PARAMS(extending.probe,extending.generator), but nothing describes the core-side procedure.cli.rstand_config.rstare autodoc stubs, andcliref.rstis generated--helpoutput.That gap matters more than it looks, because the procedure fails silently when you get it
wrong. Registering a switch with argparse but not adding its name to the matching
*_paramslist in_config.pymeanscli.pydrops the value intoignored_params(cli.py:396-407) — the switch parses, the value shows up in the argparse
Namespace, and itnever reaches
_config. Nothing is printed above debug log level.Change
Adds
docs/source/extending.cli.rst, "Adding a configuration option", covering:garak/resources/garak.core.yaml, and the*_paramslists ingarak/_config.py*_paramsentry,argparse definition taking its default from the config section, and the test hooks in
tests/test_config.py(OPTIONS_PARAM/OPTIONS_SOLO)--list_config, and what it means if the value is missing*_paramsstring,_store_config(),the
cli.pydispatch branch, and thegarak.core.yamlkeycliref.rstis generated by theclirefmake target and shouldn't behand-edited
Registered in the
index.rst"Extending and Contributing" toctree and linked fromextending.rst.The content follows the step list posted on #385, with one correction: that list says "add a
new dataclass in
_config.py" for a new config group, but current code has all four sectionsas instances of the single
GarakSubConfigdataclass, so the page describes adding aninstance.
Docs only — three
.rstfiles, no code, no data, no new dependencies.Verification
Each documented step was checked against current
mainbehaviour rather than taken from theissue comment. Two scratch experiments were run in a throwaway working tree and reverted
before writing the page:
--demo_cap, norun_paramsentry): the value appears in the parsedNamespacebut_config.runhas no such attribute — confirming the silent-drop behaviourthe page warns about.
--demo_cap 3 --list_configgivesrun.demo_cap = 3;--list_configalone gives7, thegarak.core.yamldefault. Both match the page.Commands run:
cd docs/source && python -m sphinx -T -E -b html -d _build/doctrees -D language=en -W ./ html— exit 0,
build succeeded, 152 files (151 before). Same command as.github/workflows/docs.yml, including-W. Re-run from a cleanhtml/_build.python -m pytest tests/test_docs.py -q— exit 0, 684 passed (682 before; the new pageis picked up by the parametrised docs checks)
python -m pytest tests/test_config.py -q— exit 0, 85 passedpython -m pytest tests/test_configurable.py -q— exit 0, 10 passedextending.cli.htmlbuilds, appears in the index toctree, andthe cross-links to/from
extendingresolveEnvironment: Python 3.12.13, editable install with the
testsandlintextras plusdocs/requirements-docs.txt, onmainat 5486199.blackandpylintare not applicable — no.pyfile is changed.A full
python -m pytest tests/run is slow on this machine and hadn't finished when I wrotethis up — it was still progressing with no failures. Since the diff is three
.rstfiles withno code changes, the Python test surface is unaffected, and CI covers the full matrix.
Why this isn't a duplicate
No open or closed PR addresses #385 (
gh pr list --state all --search "385 in:body"returnsnothing, and no PR in the current open list touches CLI or config documentation). #385 is
unassigned; it was marked stale in September 2025, then un-staled and given
dontautocloseby@leondz. The related open PR #1964 covers #1511 (calibration/bag reference generation) and
doesn't overlap with this.
Limitations
*_paramsregistration step. document how to add a cli switch using the new config setup #385itself notes this "ideally shouldn't need docs" — the underlying duplication between
argparse definitions and the
*_paramslists is acknowledged in a comment incli.py. ThisPR doesn't attempt that refactor.
run.max_retries) is illustrative; no such option is added.extending.cli.rstfollows the existingextending.<topic>.rstnaming, but it sits next to two plugin-authoring guides while beingcore-level. Happy to fold it into
configurable.rstorextending.rstinstead if you'dprefer fewer pages.
Note: this contribution was prepared with AI coding-assistant help.