Skip to content

Read the OpenAPI spec as UTF-8 regardless of process locale - #701

Merged
jeremy merged 1 commit into
mainfrom
rb-generator-encoding
Aug 11, 2026
Merged

Read the OpenAPI spec as UTF-8 regardless of process locale#701
jeremy merged 1 commit into
mainfrom
rb-generator-encoding

Conversation

@jeremy

@jeremy jeremy commented Aug 11, 2026

Copy link
Copy Markdown
Member

Unblocks the v0.14.0 cold-verification leg (#700).

The LC_ALL=C hardening pinned encoding: "UTF-8" on reads across scripts/
(see check-projected-examples.rb, validate-api-gaps.rb, …), but the three
Ruby generators under ruby/scripts/ were missed. Under a C locale Ruby's
default external encoding is US-ASCII, and JSON.parse(File.read(openapi_path))
dies on the spec's first multibyte character:

lib/json/common.rb:364:in 'String#encode': "\xE2" on US-ASCII (Encoding::InvalidByteSequenceError)

Surfaced by rb-check-drift failing during a cold make check on a build
server over non-interactive ssh (no locale exported). Notably, rb-check-drift
runs in no CI job — only in local make check — so green CI never
disproved this; the per-step LC_ALL: C pins in test.yml don't reach it
either. Whether the Ruby drift check should join CI (and run under LC_ALL: C
like its hardened siblings) is a real question, but a separate one from this
fix — flagged here rather than smuggled in.

Verified, both directions under LC_ALL=C: the pre-fix generate-metadata.rb
exits 1 with exactly the error above; the pinned generate-metadata.rb and
generate-types.rb exit 0 (generate-services.rb gets exercised by the full
rb-check-drift rerun on the build server once this lands).


Summary by cubic

Force UTF-8 when reading the OpenAPI spec in the Ruby generators to prevent JSON parse errors under LC_ALL=C. This fixes rb-check-drift failures in cold make check and unblocks the v0.14.0 cold-verification leg (#700).

  • Bug Fixes
    • Read the spec as UTF-8 in ruby/scripts/generate-metadata.rb, ruby/scripts/generate-services.rb, and ruby/scripts/generate-types.rb.
    • Avoids US-ASCII default under C locale that raised Encoding::InvalidByteSequenceError on multibyte characters.

Written for commit aec43fa. Summary will update on new commits.

Review in cubic

The LC_ALL=C hardening pinned encoding: "UTF-8" across scripts/, but the
three Ruby generators under ruby/scripts/ were missed. Under a C locale —
ssh non-interactive shells, LC_ALL=C CI steps — Ruby's default external
encoding is US-ASCII, and JSON.parse dies on the spec's first multibyte
character:

    lib/json/common.rb:364:in 'String#encode': "\xE2" on US-ASCII
    (Encoding::InvalidByteSequenceError)

Found by rb-check-drift failing in a cold make check over ssh — the check
runs in no CI job, so no green CI ever disproved it. Verified: the old
generate-metadata.rb exits 1 with exactly that error under LC_ALL=C, the
pinned scripts exit 0.
Copilot AI balanced review requested due to automatic review settings August 11, 2026 21:26
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@github-actions github-actions Bot added the ruby Pull requests that update the Ruby SDK label Aug 11, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Ensures Ruby generators parse OpenAPI as UTF-8 under locale-independent builds, unblocking v0.14.0 verification.

Changes:

  • Explicitly sets UTF-8 when reading OpenAPI.
  • Applies the fix consistently across all three Ruby generators.

Tip

If you aren't ready for review, convert to a draft PR.
Click "Convert to draft" or run gh pr ready --undo.
Click "Ready for review" or run gh pr ready to reengage.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
ruby/scripts/generate-metadata.rb Reads OpenAPI metadata as UTF-8.
ruby/scripts/generate-services.rb Reads service definitions as UTF-8.
ruby/scripts/generate-types.rb Reads schemas as UTF-8.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@jeremy

jeremy commented Aug 11, 2026

Copy link
Copy Markdown
Member Author

Merging on green CI + Copilot's clean review (no comments). Codex remains over its code-review usage limit, as on #698 — absent for quota reasons, not verdict. The fix carries its own red proof: pre-fix generate-metadata.rb exits 1 under LC_ALL=C with the exact Encoding::InvalidByteSequenceError from the cold-check log; the pinned scripts exit 0.

@jeremy
jeremy merged commit 2287c75 into main Aug 11, 2026
46 checks passed
@jeremy
jeremy deleted the rb-generator-encoding branch August 11, 2026 21:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ruby Pull requests that update the Ruby SDK

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants