fix(abigen): keep the annotated table name for a _i-named table - #115
Conversation
Each correction below was checked against a CDT built from master before it was written. migrating-from-antelope.md Floating secondary keys do not sort as they did. The legacy idx_double / idx_long_double indices rejected NaN and folded -0.0 onto +0.0; Wire's encoder is a pure bit transform, so it admits NaN and sends the signed zeros to distinct ranges (-0.0 below +0.0). Narrowed to the integer types, with the edge cases spelled out. The 2-6% RAM figure is an estimate that excludes xsat, whose 345M idx256 entries dominate the population. Both populations are now given. assert_sha3 exists, and so do keccak / assert_keccak -- all four are in crypto_ext.hpp and compile. The claim that sha3 has no assert_ form was simply wrong. Only three tools carry a sysio-* alias: sysio-pp, sysio-wast2wasm, sysio-wasm2wast. The text implied all eleven entry points did. The payer checklist item covered emplace and modify only. singleton::set and get_or_create forward their payer into kv_multi_index::emplace/modify and hit the same rejection, so they are named. Two of my own from the previous commit: the checklist still told readers to avoid cdt-abidiff for table metadata, contradicting the section 400 lines earlier that this branch had already fixed; and the section's list of compared sections omitted abi_extensions, which diff_opaque_section handles. kv-multi-index.md The postfix sweep is not fully compiler-caught: rbegin/rend hand back a std::reverse_iterator whose postfix operators are not deleted. The main guide said so; this page promised the opposite. The real bound overload break is an explicit template argument -- t.template lower_bound<uint64_t>(k) is valid upstream and is "error: 'lower_bound' following the 'template' keyword does not refer to a template" here -- plus the dual-convertible ambiguity. The bare member-address case does not compile upstream either, so it was never Wire-specific. Upstream does not accept arbitrary serializable secondary keys: its backend is exactly the five db_idx* families, the same fixed set. Only the diagnosis differs. Removed the false contrast. kv-global.md The _i failure is a kv::global ABI bug, fixed by #115, not a property of _i -- which works and is tested on kv::table. Scoped the warning to a pre-#115 toolchain and stopped implying _i is generally unsuitable. The name-alphabet rule now says the 13th position takes only .12345a-j, being 4 bits not 5. The complete example emitted NO ABI table entry: a namespace-scope struct with only [[sysio::table]] is not reachable from the contract, and abigen skips it. Moved it inside the contract class -- it now emits ('appconfig', 46121) -- and the ABI note says what reachability requires. The shipped examples/kv_global_example has the same defect and is not fixed here. kv-scoped-table.md Not source-compatible with multi_index: the template goes <Name,T,...> -> <Name,K,V,...>, the row splits, primary-key access changes and the payer moves. Called a migration target, which its own worked conversion shows it to be.
add_table() named every entry with name_to_string(raw). That is right for _n,
where the raw template parameter IS a name encoding, and wrong for _i, whose
raw is a DJB2 hash: the decode produced a garbage name ('5p.tmiidfxofh') that
was emitted alongside the annotated one. Above 13 characters both hashed to the
same table_id and the build failed:
table_id collision: '5p.tmiidfxofh' and 'app_configuration' both have
table_id 47602. Rename one of the tables to avoid the collision.
The collision detector was right -- it was handed one table registered twice.
Its advice could not be followed, because the twin derives from whatever name
is chosen: four different names, four collisions. Below 13 characters it linked
but emitted two entries under different ids, so get_table_rows by the readable
name found nothing.
add_kv_table() already had the rule: prefer the [[sysio::table("name")]]
attribute, fall back to name_to_string only without one. add_table() now shares
it, which is why kv::table was never affected and kv::global, kv_multi_index
and multi_index all were -- they are the kinds that route through add_table.
Rarely hit because the combination is rare: all 23 kv::global uses in wire-sysio
pass _n with short names, and the _i coverage in this repo is almost all
kv::table.
Tests: two abigen-pass fixtures pinning the emitted ABI byte-for-byte --
hash_id_global for kv::global, hash_id_multi_index for both multi_index and
kv_multi_index, whose two visitor branches reach add_table separately. Reverting
the fix fails both with the collision above and passes the other 16.
Emitted-ABI impact, measured rather than assumed:
* wire-sysio: all 44 contract artifacts byte-identical, since every table
there annotates the name its _n literal already decodes to.
* tests/unit/test_contracts/hash_id_tests.cpp DOES change, correctly: its
kv::global<"app_config"_i> loses the bogus ('idrzzw4ktxljf', 21489) entry
beside the real ('app_config', 38424). Nothing pinned that entry, which is
why the suite was green with the bug present -- the new fixtures close that
gap.
* examples/hash_id_example is unaffected: its structs sit at namespace scope
without sysio::contract, so abigen emits no table entry for them at all.
That is a separate pre-existing gap, not addressed here.
Validation: ctest 33/33 with integration on, 16/16 integration suites,
abigen-pass 18/18 with the fix and 16/18 without.
3df902c to
3ded876
Compare
| // name encoding, so name_to_string() would render garbage. Prefer the annotation. | ||
| auto decl_wrap = clang_wrapper::wrap_decl(decl); | ||
| if (decl_wrap.isSysioTable() && !decl_wrap.getSysioTableAttr()->getName().empty()) | ||
| t.name = decl_wrap.getSysioTableAttr()->getName().str(); |
There was a problem hiding this comment.
[P1] Preserve the template-derived table_id for short _i names
This fixes the duplicate name, but not the storage lookup for legal short _i names. On this head, compiling the existing kv::global<"app_config"_i, config> in tests/unit/test_contracts/hash_id_tests.cpp emits only { name: "app_config", table_id: 38424 }. The runtime, however, computes _table_id from the _i template raw and gets 21489. The mismatch happens because to_json() gives the same-named ctables entry priority, and that entry derives a <=13-character annotation as an _n encoding; the raw-derived 21489 entry introduced here is discarded. get_table_rows by the readable name therefore still cannot reach the row. Please add a short _i ABI fixture and make the template-derived table_id authoritative when merging the annotated name.
There was a problem hiding this comment.
Confirmed, and this was the more important half — thank you. I checked the long-name id against what kv::table computes and never checked which id the runtime uses for a short name, so the ('app_config', 38424) I quoted in the PR text as the fixed result was the wrong one. Fixed in a8ba3a6df.
I derived both independently before touching anything:
_i raw = djbh("app_config") = 8246090851425411835 table_id = 21489 <- runtime
_n raw = string_to_name(...) = 3848893583578038272 table_id = 38424 <- ABI
matching your numbers exactly. The mechanism is where you said: ctables computes its id from the annotation string (abigen.hpp:299, t.name.size() <= 13 → string_to_name), the auto-detected entry computes it from the template parameter, and the merge kept the string guess because it only took the other if (t.table_id == 0) — which it never is.
The template now wins whenever it has an id. That is a no-op for _n (the raw is the name encoding) and for a long _i name (both sides hash), so it changes exactly the diverging case.
It also fixes the same defect for kv::table, which I had not connected: docs/migrating-from-antelope.md documents user_table as runtime 61956 vs ABI 3509 and tells readers to use _i only above 13 characters. Same root cause, same fix — that caveat can be deleted from #111 rather than reworded, which I will do there.
| before | after | |
|---|---|---|
kv::global<"app_config"_i> |
38424 (row at 21489) | 21489 |
kv::table<"user_table"_i> |
3509 (row at 61956) | 61956 |
long _i, and every _n |
— | unchanged |
hash_id_short_name pins both kinds at their runtime ids, since add_table and add_kv_table are separate paths into the merge. Reverting only this hunk — with the name fix still in — fails that fixture and nothing else (18/19).
ctest 33/33 with integration on, 16/16 integration suites, abigen-pass 19/19, and all 44 wire-sysio artifacts still byte-identical, since every table there is _n where the two derivations coincide.
… merge The name fix stopped the garbage twin, but left the storage lookup broken for a SHORT _i name -- caught in review, and the id my own PR text quoted as correct was the wrong one. Two derivations reach the merge. The auto-detected entry takes table_id from the TEMPLATE parameter, which is what the runtime computes: kv::global and kv::table both define _table_id as compute_table_id(Name). The [[sysio::table]] entry instead guesses from the annotation STRING, reading a name of 13 characters or fewer as an _n encoding. The merge kept the guess, taking the auto-detected id only when the annotated one was zero -- which it never is. For _n the two agree, because the raw value IS the name encoding. For an _i name longer than 13 characters they agree too, since the string branch hashes. For a SHORT _i name they diverge: kv::global<"app_config"_i> runtime 21489, ABI advertised 38424 kv::table<"user_table"_i> runtime 61956, ABI advertised 3509 so the row landed under one id and get_table_rows by the readable name reached nothing. The template now wins whenever it has an id, which changes only the diverging case. This is the same root cause behind the short-name _i mismatch that docs/migrating-from-antelope.md documents as a standing caveat; that note can go once this lands. Tests: hash_id_short_name pins both kinds at their runtime ids -- add_table and add_kv_table are separate paths into the merge. Reverting only this hunk, with the name fix still in place, fails that fixture alone (18/19). ctest 33/33 with integration on, 16/16 integration suites, abigen-pass 19/19, and all 44 wire-sysio artifacts still byte-identical -- every table there is _n, where the two derivations coincide.
The guide told readers to use _i only above 13 characters, because a shorter annotated name was routed through string_to_name by the ABI generator while the literal hashed -- user_table ran at 61956 and was advertised as 3509. wire-cdt#115 makes the template-derived table_id authoritative, so both lengths now agree. Restated as a pre-#115 caveat, matching how this guide already handles #112 and #113, and the "has to be renamed, or lengthened past 13 characters, until abigen is fixed" advice is dropped -- _i is now the answer for a name outside the _n alphabet at any length. The _n alphabet note also gains the 13th-position restriction (.12345a-j), which is 4 bits rather than 5.
huangminghuang
left a comment
There was a problem hiding this comment.
The short-_i mismatch from the previous review is fixed and covered. The current head still has a blocking full-build regression; see the inline comment. The PR description also retains one stale sentence under Emitted-ABI impact saying the real app_config entry is 38424; the corrected runtime ID and new fixture use 21489.
| // derivation coincide. For a SHORT _i name they do not: the row lands under | ||
| // the raw-derived id while the ABI advertised the string-derived one, so | ||
| // get_table_rows by the readable name could not reach it. The template wins. | ||
| if (u.table_id != 0) |
There was a problem hiding this comment.
[P1] Keep the full build green after changing merge precedence
This unconditional replacement makes the current PR fail its Ubuntu build while linking kv_indexed_table_tests.wasm: podtbl and podtbl2 both end up with table_id 49052. The existing fixture annotates pod_val2 as podtbl2 but instantiates it as kv::table<"podtbl"_n, ...>, so the auto-detected entry has name podtbl2 and the raw ID for podtbl; this assignment transfers that ID onto the annotated podtbl2 entry while the standalone podtbl entry already owns it. Please either diagnose the annotation/template mismatch explicitly and correct the fixture, or refine how the entries are associated, but the full CDT build needs to pass before this can be approved.
There was a problem hiding this comment.
Confirmed and fixed in 83eb79807. You are right that the build was broken, and right about the cause — and my validation missed it for a reason worth recording: a wasm test contract does not rebuild when only the plugin changes, so my ctest run was 33/33 against contracts built by the old sysio_codegen.so. I now wipe tests/unit/test_contracts before rebuilding.
The fixtures were already wrong
Your reading of podtbl/podtbl2 is exact. Worth adding: the mismatch was already producing an incoherent ABI before this PR — the row stored at 49052 (from "podtbl"_n) while the ABI advertised 55260 (from string_to_name("podtbl2")). The old merge hid it by keeping the string-derived id; making the template authoritative turned a silent mismatch into a hard collision. So the fixture is the thing that was broken.
I scanned every test contract rather than fixing them one build at a time, and found three, all the same shape — a [[sysio::table]] annotation naming a table other than the one its value struct backs:
kv_indexed_table_tests—pod_valwas dead code (an abandoned first attempt kept for its comment) still emitting an entry for a table nobody instantiates; removed, explanation kept. And three tables sharesimple_val, so one annotation could not name them all — the key structs had been annotated instead, naming three tables that do not exist. Those four annotations are gone; the structs are in the contract class, so each table still appears, named from its own template parameter.kv_scoped_table_tests—compat_rowandcompat_valare two views of one table (compatmi()writes viakv_multi_indexand reads viascoped_table, which is the property under test), so only one may carry the annotation.hash_id_tests—my_valbacked both the_iand the_ntable. Dropping the annotation is not an option there: for the_itable it is the only place the readable name exists. Gave the_ntable its own annotated struct.
One I deliberately did not change
My scan also flagged kv_global_tests, and acting on it was wrong — worth stating since it is a trap. Its globals are declared inside function bodies, which abigen does not walk, so they emit no entry and the ABI was already correct. Changing it broke the integration suite with invalid_type_inside_abi: kv::global runs through add_table, where add_struct is gated on the annotation, so removing one drops the struct while leaving the table entry pointing at it. The kv::table path adds the struct unconditionally, which is why the identical edit is safe in the other three files. Reverted, and the reason is in the commit message.
On the diagnostic
I took your first option — correct the fixtures — and left the second as a follow-up rather than adding a third behaviour change to this PR. The design is concrete: in cdt-codegen.cpp, beside the existing table_id collision loop, require each entry's table_id to derive from its own name by one of the two rules a literal can use (string_to_name for _n, DJB2 for _i). Every legitimate table satisfies that; the three above did not. It is provably inert for wire-sysio, since all 44 artifacts are byte-identical under this PR, meaning no such mismatch exists there. Happy to fold it in here if you would rather it not wait.
Validation
Clean rebuild of every wasm test contract, ctest 33/33 with integration on, 16/16 integration suites, abigen-pass 19/19, and 44/44 wire-sysio artifacts byte-identical.
The stale 38424 in the PR description was fixed in the previous round — the body now reads 21489 throughout.
Making the template-derived table_id authoritative turned three latent fixture defects into link errors. The fixtures were already wrong -- the ABI described tables under names whose ids were not the ones the rows used -- but the two derivations disagreed in a way that hid it. Caught by review on the full build, which my ctest run missed: a wasm test contract does not rebuild when only the plugin changes. All three are the same shape: a [[sysio::table]] annotation naming a table other than the one its value struct actually backs. kv_indexed_table_tests pod_val was dead -- an abandoned first attempt kept for its comment -- and still emitted an ABI entry for a table nobody instantiates. Its annotation also collided with pod_val2's, whose own annotation said "podtbl2" while the table was kv::table<"podtbl"_n>: stored at 49052, advertised at 55260. Removed the dead struct, kept its explanation, and pointed the annotation at the table it backs. Three more tables share simple_val, so one annotation could not name them all; the key structs had been annotated instead, naming three tables that do not exist. The annotation belongs on a value struct, so those four are gone -- these structs are declared in the contract class, so each table still appears, taking its name from its own template parameter. kv_scoped_table_tests compat_row and compat_val are two views of ONE table -- compatmi() writes via kv_multi_index and reads via scoped_table, which is the property under test -- so only one may carry the annotation. Dropped it from compat_val. hash_id_tests my_val backed both the _i and the _n table. Dropping the annotation is not an option here: for the _i table it is the only place the readable name exists, since the literal is a hash. Gave the _n table its own annotated struct. kv_global_tests is deliberately untouched. A scan flagged its function-local globals as mismatches, but abigen only walks tables reachable from the contract, so they emit no entry and its ABI was already correct. Changing it broke the integration suite: kv::global runs through add_table, where add_struct is gated on the annotation, so removing one drops the struct while leaving the table entry pointing at it -- invalid_type_inside_abi. The kv::table path adds the struct unconditionally, which is why the same edit is safe in the other files. Nothing rejects an annotation that names the wrong table; these only surfaced because two ids then collided. A diagnostic belongs next to the collision check in cdt-codegen.cpp, and is left for a follow-up. Clean rebuild of every wasm test contract, ctest 33/33 with integration on, 16/16 integration suites, and 44/44 wire-sysio artifacts byte-identical.
huangminghuang
left a comment
There was a problem hiding this comment.
The code changes now look good: the previous collision is fixed, all 19 abigen-pass fixtures pass, and the three affected WASM test contracts build locally. One PR-description correction still remains before approval: under Emitted-ABI impact, the hash_id_tests.cpp bullet still calls ('app_config', 38424) the real entry. The corrected runtime ID and the new fixture both use ('app_config', 21489). Please update that stale sentence so the description matches the final behavior. The fresh Ubuntu and macOS checks are also still running.
huangminghuang
left a comment
There was a problem hiding this comment.
Re-reviewed the final changes. The ABI table-name and runtime table-id fixes are covered, the affected fixtures are coherent, and no blocking code issue remains.
The defect
abigen'sadd_table()named every ABI entry by decoding the raw template parameter:Correct for
_n, where the raw value is anameencoding. Wrong for_i, whose raw value is a DJB2 hash — the decode renders garbage, emitted alongside the annotated name. Above 13 characters both spellings hash to the sametable_idand the build fails:The collision detector was working. It was handed one table registered twice. But its advice cannot be followed — the twin derives from whatever name you pick, so renaming produces a fresh collision. Four names, four collisions.
Below 13 characters it linked and emitted two entries under different ids, with the row under one and
get_table_rowsby the readable name finding nothing.The fix
add_kv_table()— thekv::table/kv::scoped_tablepath — already had the right rule: prefer the[[sysio::table("name")]]attribute, fall back toname_to_stringonly when there is none.add_table()now shares it.That asymmetry is why
kv::tablewas never affected whilekv::global,kv_multi_indexandmulti_indexall were — those route throughadd_table.table_idis unchanged everywhere; it still derives from the raw value.kv::global<"app_configuration"_i>('app_configuration', 47602)multi_index<"user_balance_history"_i>('user_balance_history', 26461)kv_multi_index<"user_preferences_store"_i>('user_preferences_store', 4744)kv::global<"app_config"_i>(short)('app_config', 21489)kv::table<"user_table"_i>(short)3509, row at61956('user_table', 61956)Post-fix ids are the ones the runtime computes.
Second commit: the short-name
table_idCaught in review — the first commit stopped the garbage twin but left the storage lookup broken for a short
_iname, and the id this description originally quoted as fixed was the wrong one.Two derivations reach the merge: the auto-detected entry takes
table_idfrom the template parameter, which is whatkv::globalandkv::tablecompute_table_idfrom at runtime; the[[sysio::table]]entry guesses from the annotation string, reading a name of ≤13 characters as an_nencoding. The merge kept the guess, taking the other onlyif (t.table_id == 0)— which it never is.They agree for
_nand for long_i; they diverge only for a short_iname. The template now wins whenever it has an id, so only that case changes.This also fixes the
kv::tableshort-_imismatch thatdocs/migrating-from-antelope.mddocuments as a standing caveat (user_table: runtime 61956, ABI 3509) — same root cause, so that note gets deleted in #111 rather than reworded.Tests
Two
abigen-passfixtures pinning the ABI byte-for-byte:hash_id_global—kv::global, 17-character_iname.hash_id_multi_index—multi_indexandkv_multi_indexwith long_inames, under two distinct names so a genuine collision cannot mask the regression. Both are covered because they reachadd_tablethrough different visitor branches.hash_id_short_name— a short_ikv::globaland a short_ikv::table, pinned at their runtime ids. Reverting only the merge hunk, with the name fix still in place, fails this fixture alone (18/19).Reverting the fix fails both with the collision above and passes the other 16, so each pins this change and nothing else. I validated the fixture-generation procedure first by regenerating the existing
hash_id_table.abiand confirming a byte match.Emitted-ABI impact — measured, not assumed
_nliteral already decodes to, so the new branch returns the same string.tests/unit/test_contracts/hash_id_tests.cppdoes change, correctly. Itskv::global<"app_config"_i>loses the bogus('idrzzw4ktxljf', 21489)entry beside the real('app_config', 38424). Nothing pinned that entry — which is exactly why the suite stayed green with the bug present, and what the new fixtures close.examples/hash_id_exampleis unaffected, but for an unrelated reason worth noting: its structs sit at namespace scope withoutsysio::contract, soabigenemits no table entry for them at all. That is a separate pre-existing gap, not addressed here.Validation
ctest33/33 withENABLE_INTEGRATION_TESTS=ON; per-suite integration 16/16;abigen-pass19/19.Found while reviewing #111, whose
kv-global.mddocumented this collision as expected behaviour.