Read the OpenAPI spec as UTF-8 regardless of process locale - #701
Conversation
The LC_ALL=C hardening pinned encoding: "UTF-8" across scripts/, but the
three Ruby generators under ruby/scripts/ were missed. Under a C locale —
ssh non-interactive shells, LC_ALL=C CI steps — Ruby's default external
encoding is US-ASCII, and JSON.parse dies on the spec's first multibyte
character:
lib/json/common.rb:364:in 'String#encode': "\xE2" on US-ASCII
(Encoding::InvalidByteSequenceError)
Found by rb-check-drift failing in a cold make check over ssh — the check
runs in no CI job, so no green CI ever disproved it. Verified: the old
generate-metadata.rb exits 1 with exactly that error under LC_ALL=C, the
pinned scripts exit 0.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
There was a problem hiding this comment.
Pull request overview
Ensures Ruby generators parse OpenAPI as UTF-8 under locale-independent builds, unblocking v0.14.0 verification.
Changes:
- Explicitly sets UTF-8 when reading OpenAPI.
- Applies the fix consistently across all three Ruby generators.
Tip
If you aren't ready for review, convert to a draft PR.
Click "Convert to draft" or run gh pr ready --undo.
Click "Ready for review" or run gh pr ready to reengage.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
ruby/scripts/generate-metadata.rb |
Reads OpenAPI metadata as UTF-8. |
ruby/scripts/generate-services.rb |
Reads service definitions as UTF-8. |
ruby/scripts/generate-types.rb |
Reads schemas as UTF-8. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
Merging on green CI + Copilot's clean review (no comments). Codex remains over its code-review usage limit, as on #698 — absent for quota reasons, not verdict. The fix carries its own red proof: pre-fix generate-metadata.rb exits 1 under LC_ALL=C with the exact Encoding::InvalidByteSequenceError from the cold-check log; the pinned scripts exit 0. |
Unblocks the v0.14.0 cold-verification leg (#700).
The LC_ALL=C hardening pinned
encoding: "UTF-8"on reads acrossscripts/(see
check-projected-examples.rb,validate-api-gaps.rb, …), but the threeRuby generators under
ruby/scripts/were missed. Under a C locale Ruby'sdefault external encoding is US-ASCII, and
JSON.parse(File.read(openapi_path))dies on the spec's first multibyte character:
Surfaced by
rb-check-driftfailing during a coldmake checkon a buildserver over non-interactive ssh (no locale exported). Notably,
rb-check-driftruns in no CI job — only in local
make check— so green CI neverdisproved this; the per-step
LC_ALL: Cpins intest.ymldon't reach iteither. Whether the Ruby drift check should join CI (and run under
LC_ALL: Clike its hardened siblings) is a real question, but a separate one from this
fix — flagged here rather than smuggled in.
Verified, both directions under
LC_ALL=C: the pre-fixgenerate-metadata.rbexits 1 with exactly the error above; the pinned
generate-metadata.rbandgenerate-types.rbexit 0 (generate-services.rbgets exercised by the fullrb-check-driftrerun on the build server once this lands).Summary by cubic
Force UTF-8 when reading the OpenAPI spec in the Ruby generators to prevent JSON parse errors under
LC_ALL=C. This fixesrb-check-driftfailures in coldmake checkand unblocks the v0.14.0 cold-verification leg (#700).ruby/scripts/generate-metadata.rb,ruby/scripts/generate-services.rb, andruby/scripts/generate-types.rb.Encoding::InvalidByteSequenceErroron multibyte characters.Written for commit aec43fa. Summary will update on new commits.