Map bytea to a binary String instead of hex text#2
Merged
Conversation
A bytea value now reaches Ruby as a binary (ASCII-8BIT) String holding
its exact bytes -- NUL-safe -- rather than as its \x... hex text output,
and returning a String into a bytea takes the string's raw bytes
verbatim (any encoding), with no hex/escape parsing. This matches
PL/Python's bytea <-> bytes mapping.
The conversion is intercepted at the datum level (before the type's text
I/O) wherever a bytea crosses between SQL and Ruby:
- scalar arguments and bytea[] arguments (plruby_func_build_args)
- scalar and bytea[] return values (plruby_func_handler return path,
which otherwise NUL-truncates via the text-input path)
- composite fields, SPI result rows, and trigger $_TD (the shared
plruby_hash_from_tuple)
- fields, OUT params, and return_next rows via plruby_datum_from_value
FROM-SQL uses PG_DETOAST_DATUM_PACKED + VARDATA_ANY/VARSIZE_ANY_EXHDR
into an ASCII-8BIT string; TO-SQL builds the varlena directly from the
string's raw bytes. bytea[] shares the array datum walk, generalized to
convert elements binary-wise when no FromSQL transform applies.
BREAKING: bodies that produced or consumed the hex text form must be
updated (e.g. build bytes with [..].pack('C*') rather than a \x string).
Rewrote sql/bytea for the new semantics (round-trips with NULs, high
bytes, composite/SPI/array coverage). Full suite green on PG 12 and 18;
jsonb_plruby and hstore_plruby suites still pass (shared array walk).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
e69aa63 to
f2a1404
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes the PL/Python parity gap on
bytea. Abyteavalue now reaches Ruby as a binary (ASCII-8BIT)Stringholding its exact bytes — NUL-safe — rather than as its\x...hex text, and returning aStringinto abyteatakes the string's raw bytes verbatim (any encoding), with no hex/escape parsing. This matches PL/Python'sbytea↔bytesmapping.Function bodies that previously produced or consumed the hex text form must be updated — e.g. build bytes with
[0, 255, 16].pack('C*')instead of assembling a"\\x..."string, and read raw bytes (.bytes,.bytesize) instead of parsing hex. Slated for a 2.5.0 minor bump alongside$_SD(#1).Coverage
The conversion is intercepted at the datum level (before the type's text I/O) everywhere a
byteacrosses between SQL and Ruby:bytea[]argumentsbytea[]return values (the return path otherwise NUL-truncates through the text-input path)$_TD(sharedplruby_hash_from_tuple)return_nextrows (viaplruby_datum_from_value)FROM-SQL uses
PG_DETOAST_DATUM_PACKED+VARDATA_ANY/VARSIZE_ANY_EXHDRinto an ASCII-8BIT string; TO-SQL builds the varlena directly.bytea[]reuses the array datum walk, generalized to convert elements binary-wise when no FromSQL transform applies (public signature unchanged).Testing
sql/bytearewritten for the new semantics: round-trips preserving embedded NULs and high bytes, byte inspection, building bytea from raw Ruby bytes, empty bytea, bytea in a composite, bytea via SPI, andbytea[]both directions. Full suite green on PG 12 and PG 18;jsonb_plrubyandhstore_plrubysuites still pass (they share the generalized array walk).Note for merge
Both this PR and #1 add an
## [Unreleased]section toCHANGELOG.md; whichever merges second should combine the two entries under one[Unreleased]heading.🤖 Generated with Claude Code