Skip to content

Map bytea to a binary String instead of hex text#2

Merged
jdatcmd merged 1 commit into
masterfrom
feature/bytea-binary
Jul 7, 2026
Merged

Map bytea to a binary String instead of hex text#2
jdatcmd merged 1 commit into
masterfrom
feature/bytea-binary

Conversation

@jdatcmd

@jdatcmd jdatcmd commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Summary

Closes the PL/Python parity gap on bytea. A bytea value now reaches Ruby as a binary (ASCII-8BIT) String holding its exact bytes — NUL-safe — rather than as its \x... hex text, and returning a String into a bytea takes the string's raw bytes verbatim (any encoding), with no hex/escape parsing. This matches PL/Python's byteabytes mapping.

⚠️ Breaking change

Function bodies that previously produced or consumed the hex text form must be updated — e.g. build bytes with [0, 255, 16].pack('C*') instead of assembling a "\\x..." string, and read raw bytes (.bytes, .bytesize) instead of parsing hex. Slated for a 2.5.0 minor bump alongside $_SD (#1).

Coverage

The conversion is intercepted at the datum level (before the type's text I/O) everywhere a bytea crosses between SQL and Ruby:

  • scalar and bytea[] arguments
  • scalar and bytea[] return values (the return path otherwise NUL-truncates through the text-input path)
  • composite fields, SPI result rows, and trigger $_TD (shared plruby_hash_from_tuple)
  • OUT params and return_next rows (via plruby_datum_from_value)

FROM-SQL uses PG_DETOAST_DATUM_PACKED + VARDATA_ANY/VARSIZE_ANY_EXHDR into an ASCII-8BIT string; TO-SQL builds the varlena directly. bytea[] reuses the array datum walk, generalized to convert elements binary-wise when no FromSQL transform applies (public signature unchanged).

Testing

sql/bytea rewritten for the new semantics: round-trips preserving embedded NULs and high bytes, byte inspection, building bytea from raw Ruby bytes, empty bytea, bytea in a composite, bytea via SPI, and bytea[] both directions. Full suite green on PG 12 and PG 18; jsonb_plruby and hstore_plruby suites still pass (they share the generalized array walk).

Note for merge

Both this PR and #1 add an ## [Unreleased] section to CHANGELOG.md; whichever merges second should combine the two entries under one [Unreleased] heading.

🤖 Generated with Claude Code

A bytea value now reaches Ruby as a binary (ASCII-8BIT) String holding
its exact bytes -- NUL-safe -- rather than as its \x... hex text output,
and returning a String into a bytea takes the string's raw bytes
verbatim (any encoding), with no hex/escape parsing. This matches
PL/Python's bytea <-> bytes mapping.

The conversion is intercepted at the datum level (before the type's text
I/O) wherever a bytea crosses between SQL and Ruby:

  - scalar arguments and bytea[] arguments (plruby_func_build_args)
  - scalar and bytea[] return values (plruby_func_handler return path,
    which otherwise NUL-truncates via the text-input path)
  - composite fields, SPI result rows, and trigger $_TD (the shared
    plruby_hash_from_tuple)
  - fields, OUT params, and return_next rows via plruby_datum_from_value

FROM-SQL uses PG_DETOAST_DATUM_PACKED + VARDATA_ANY/VARSIZE_ANY_EXHDR
into an ASCII-8BIT string; TO-SQL builds the varlena directly from the
string's raw bytes. bytea[] shares the array datum walk, generalized to
convert elements binary-wise when no FromSQL transform applies.

BREAKING: bodies that produced or consumed the hex text form must be
updated (e.g. build bytes with [..].pack('C*') rather than a \x string).

Rewrote sql/bytea for the new semantics (round-trips with NULs, high
bytes, composite/SPI/array coverage). Full suite green on PG 12 and 18;
jsonb_plruby and hstore_plruby suites still pass (shared array walk).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@jdatcmd jdatcmd force-pushed the feature/bytea-binary branch from e69aa63 to f2a1404 Compare July 7, 2026 02:54
@jdatcmd jdatcmd merged commit 50be762 into master Jul 7, 2026
9 checks passed
@jdatcmd jdatcmd deleted the feature/bytea-binary branch July 7, 2026 02:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant