Skip to content

fix(cli): make lint-translations exit non-zero when translations differ - #2996

Open
Zuhef wants to merge 1 commit into
Chainlit:mainfrom
Zuhef:fix/lint-translations-exit-code
Open

fix(cli): make lint-translations exit non-zero when translations differ#2996
Zuhef wants to merge 1 commit into
Chainlit:mainfrom
Zuhef:fix/lint-translations-exit-code

Conversation

@Zuhef

@Zuhef Zuhef commented Jul 29, 2026

Copy link
Copy Markdown

Problem

chainlit lint-translations reports differences and then exits 0, so it cannot gate anything. Issue #2993 identified this as the reason locale keys drift unnoticed, and it is true even for the app translations the command is actually pointed at.

The chain is silent end to end:

  • lint_translation_json prints the differences and returns None
  • lint_translations returns None
  • chainlit_lint_translations calls it and ignores the result, so Click exits 0

Change

  • lint_translation_json returns the differences it printed. Additive - existing callers that ignore the return value are unaffected, and the printed output is unchanged.
  • lint_translations returns the total count across all linted files.
  • The CLI command prints a summary and raises SystemExit(1) when the count is non-zero.

Two incidental cleanups in the same function: the per-file loop was nested inside the ground-truth with open(...) block, holding that handle open for the entire run, so it is dedented; and os.listdir is now sorted so the report order is deterministic. Happy to drop either if you would rather keep the diff to the exit code alone.

Verification

Ran on Windows against a directory containing all 23 packaged locales, comparing a clean main worktree with the patched tree:

files linted differences reported exit code
main 23 6 0
patched 23 6 1

Same six differences either way - only the exit code changes. A directory whose translations match the ground truth still exits 0.

The six it finds are the real gaps from #2993, which is a useful independent cross-check: this is the CLI path rather than the test suite, and it reports exactly chat.favorites.remove (ar-SA, da-DK), components.DatePickerInput (de-DE, it, ko) and chat.fileUpload.browse (ja).

pytest backend/tests/test_translations.py - 39 passed. New tests cover the returned error list, the aggregate count for both clean and broken input, and that non-JSON files in the directory are ignored.

ruff format --check clean on all four files. ruff check reports two RUF036 findings at config.py:391-392, which are pre-existing and outside this change.

Note

While verifying this I hit a separate pre-existing problem: on a stock Windows console the command dies with a UnicodeEncodeError printing the / markers, since the console code page cannot encode them. It only runs with PYTHONIOENCODING=utf-8 set. That is independent of this change and I have not touched it here - happy to open a separate issue or PR for it if useful.


Summary by cubic

Fix the CLI so chainlit lint-translations exits non-zero when any translation differs. This enables CI to fail on missing or extra keys and prevents locale drift.

  • Bug Fixes

    • lint_translation_json now returns the list of differences (printing unchanged).
    • lint_translations returns the total difference count across files.
    • The CLI prints a summary and exits 1 when the count is non-zero.
    • Tests cover return values, aggregate counts, and ignoring non-JSON files.
  • Refactors

    • Moved the per-file loop out of the ground-truth with open(...) block to avoid holding the file open.
    • Sorted os.listdir for deterministic reporting order.

Written for commit f89b44b. Summary will update on new commits.

Review in cubic

`chainlit lint-translations` reported differences and then exited 0, so it
could not be used as a CI gate - which is how missing locale keys accumulate
unnoticed.

The chain was silent end to end: `lint_translation_json` printed the
differences and returned None, `lint_translations` returned None, and the
command ignored the result. On a checkout of main the command reports six real
missing keys across the packaged locales and still exits 0.

`lint_translation_json` now returns the differences it printed, and
`lint_translations` returns the total count, so the command can fail. Both are
additive: existing callers that ignore the return value are unaffected.

Also dedent the per-file loop out of the ground-truth `with open(...)` block,
which held that handle open for the whole run, and sort the directory listing
so the report order is deterministic.

Verified on Windows against a directory holding all 23 packaged locales:
before, 6 differences reported and exit 0; after, the same 6 differences and
exit 1. A clean directory still exits 0.
@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. bug Something isn't working translation Translation or localisation of strings. unit-tests Has unit tests. labels Jul 29, 2026

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 4 files

Re-trigger cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working size:M This PR changes 30-99 lines, ignoring generated files. translation Translation or localisation of strings. unit-tests Has unit tests.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant