Skip to content

Speed up hash mode by interning element names directly - #475

Open
Watson1978 wants to merge 1 commit into
ohler55:developfrom
Watson1978:codex/hash-intern
Open

Speed up hash mode by interning element names directly#475
Watson1978 wants to merge 1 commit into
ohler55:developfrom
Watson1978:codex/hash-intern

Conversation

@Watson1978

Copy link
Copy Markdown
Contributor

Ox.load(..., mode: :hash) creates a temporary Ruby String for every element name, associates its encoding, and immediately interns it. Repeated tags therefore allocate a new String even when the name is already interned.

Use rb_intern3() on the name bytes, preserving the parser's encoding and the ASCII-8BIT fallback for file input without an encoding. Keep interning after attribute processing, including attribute key modifier callbacks. This extends the element-name optimization in #474 to hash mode; attribute key handling is unchanged.

On Ruby 4.0.6 / arm64, parsing 40,000 records with five text fields per record:

Before After
Parse time 46.10 ms 41.75 ms
Allocated objects 800,006 560,005

This is a 9.4% reduction in parse time. Each process warmed up twice, then measured seven parses with GC.start before each measurement. These are the medians of four process-level medians, alternating the before/after execution order. All result digests matched. The unchanged hash_no_attrs control measured 28.27 ms before and 29.23 ms after, with identical allocation counts.

Input:

xml = '<rows>' + '<row id="1"><name>Alice</name><city>Tokyo</city><age>30</age><status>active</status><note>hello</note></row>' * 40_000 + '</rows>'
Ox.load(xml, mode: :hash)

Validation:

  • Added tests for repeated elements with and without attributes, UTF-8/Shift_JIS/binary names, Symbol and String keys, XML-declared encoding, and the file-input binary fallback.
  • Added coverage for encoded names passed to attribute and element key modifiers, including allocation/GC during the attribute callback.
  • test/hash_load_test.rb: 19 tests, 109 assertions passed against both the original and updated implementation. Also passed with ASCII-8BIT configured as the default encoding before running the suite.
  • Additional *_test.rb suite, using the Rake test task's exclusions: 344 tests, 7,044 assertions passed.
  • test/tests.rb and test/sax/sax_test.rb passed.
  • clang-format --dry-run --Werror ext/ox/hash_load.c and git diff --check passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant