test(json): adversarial QuickCheck roundtrip and parser-robustness tests by bobzhang · Pull Request #4045 · moonbitlang/core

bobzhang · 2026-08-14T08:21:22Z

Summary

Adversarial QuickCheck property tests for the json parser and stringifier, complementing json/quickcheck_test.mbt. This suite found the bugs now tracked issue-first and fixed in dedicated PRs:

json: parser aborts on strings containing lone surrogates combined with escapes #4049 — raw lone surrogates aborted the process (fixed in fix(json): reject raw unpaired surrogates in string lexing instead of aborting #4056, base main)
json: parser constructs ill-formed strings from unpaired \uXXXX escape sequences #4062 — unpaired \uXXXX surrogate escapes manufactured ill-formed strings (fixed in fix(json): reject unpaired \uXXXX surrogate escapes in strings #4064, base main)
json: parse("-0") loses the sign of negative zero #4053 — parse("-0") lost the sign of zero (fixed in fix(json): preserve the sign of -0 in the integer fast path #4061, already merged)

Merge order: this PR should merge after BOTH #4056 and #4064. It is based on agent/fix-json-lexer (#4056) and retargets to main when that merges. On the current branch (which has #4056's fix and the merged #4061, but not #4064) exactly two tests fail, both needing #4064:

"lone surrogates are rejected in every position" — the escaped (\uXXXX) spelling of a lone surrogate is still accepted without fix(json): reject unpaired \uXXXX surrogate escapes in strings #4064;
"surrogate handling pins" — the escaped-rejection assertions.

All other 209 tests are green on wasm-gc, js, and native; the suite goes fully green once #4064 is in the base history.

New property families (`json/quickcheck_adversarial_test.mbt`)

Adversarial strings — every control character, JSON syntax characters, BMP boundaries, astral pairs (Unicode scalar values only) — roundtrip through stringify/parse as values and object keys, across indent/escape_slash. The AdvString shrinker drops whole characters so candidates stay well-formed.
Fully \uXXXX-escaped spellings parse back to the exact original (pairs split across two escapes, mixed-case hex).
Lone-surrogate rejection (unicode-safe policy): a lone surrogate injected at any position — raw via stringify or spelled as an escape, with escapes/astral pairs/control chars nearby — is always rejected cleanly: parse raises the documented error, valid is false, nothing ever aborts. Deterministic pins cover raw, escaped, reversed-pair, and mixed raw/escaped-half spellings, plus still-accepted well-formed pairs.
Zero literals preserve the sign of zero bitwise (fix(json): preserve the sign of -0 in the integer fast path #4061, merged — passing).
Int64/UInt64 literals roundtrip textually (repr preserved beyond 2^53).
Whitespace insensitivity; duplicate keys last-wins; mutation totality (single UTF-16 code-unit deletion/replacement keeps parse consistent with valid, successful mutants are restringify-reparse fixed points); depth-1024 boundary pins.

Verification

moon check clean; moon info && moon fmt — no .mbti changes.
Current branch: 209/211 on wasm-gc, js, and native (identical two failures everywhere — see merge-order note).
Properties stress-run locally with seeds 1..20 and counts up to 2000 per property on all three backends; committed tests use CI-friendly default counts.

🤖 Generated with Claude Code

coveralls · 2026-08-14T08:24:52Z

Coverage Report for CI Build 6069

Warning

No base build found for commit ca31837 on agent/fix-json-lexer.
Coverage changes can't be calculated without a base build.
If a base build is processing, this comment will update automatically when it completes.

Coverage: 90.663%

Details

Patch coverage: No coverable lines changed in this PR.

Uncovered Changes

No uncovered changes found.

Coverage Regressions

Requires a base build to compare against. How to fix this →

Coverage Stats


Relevant Lines:	17671
Covered Lines:	16021
Line Coverage:	90.66%
Coverage Strength:	151096.45 hits per line

💛 - Coveralls

Copilot

Pull request overview

This PR strengthens the json package’s correctness by fixing two parser edge cases uncovered via adversarial property testing (lone-surrogate handling in string lexing and -0 sign preservation in number lexing), and adds a new suite of deterministic, CI-friendly QuickCheck properties to prevent regressions.

Changes:

Fix lex_string_slow to avoid aborting on lone trailing-surrogate boundaries by using bounds-checked view(...) instead of checked slicing.
Fix integer fast-path number lexing to preserve IEEE-754 negative zero by applying the sign after Int64 -> Double conversion.
Add adversarial property tests (strings/escapes/whitespace/dup keys/mutations/nesting limit) plus a targeted unit test for negative-zero parsing.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
json/quickcheck_adversarial_test.mbt	New adversarial QuickCheck suite exercising parser/stringifier robustness and edge cases.
json/lex_string.mbt	Prevents aborts when slow-path string flushing crosses lone-surrogate boundaries.
json/lex_number.mbt	Preserves `-0.0` sign in the integer fast path by negating after conversion to `Double`.
json/lex_number_test.mbt	Adds a regression unit test asserting negative-zero sign preservation across spellings/underflow.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

bobzhang · 2026-08-15T01:21:35Z

+  let deep_array = "[".repeat(1024) + "0" + "]".repeat(1024)
+  assert_true(@json.valid(deep_array))
+  fn outcome(text : String) -> String {
+    try {
+      ignore(@json.parse(text))
+      "parsed"
+    } catch {
+      DepthLimitExceeded => "depth limit"
+      _ => "other error"
+    }
+  }
+
+  let deeper_array = "[".repeat(1025) + "0" + "]".repeat(1025)
+  assert_eq(outcome(deeper_array), "depth limit")
+  let deep_object = "{\"k\":".repeat(1025) + "0" + "}".repeat(1025)
+  assert_eq(outcome(deep_object), "depth limit")


Fixed in 29009f6: the test now asserts depth-1024 documents parse successfully for both arrays and objects (assert_eq(outcome(deep_array), "parsed") / assert_eq(outcome(deep_object), "parsed")), and the 1025-deep variables are renamed too_deep_array / too_deep_object. The test now pins the documented boundary exactly on both sides for both shapes.

bobzhang · 2026-08-15T01:21:37Z

+///|
+/// Deleting or replacing one code unit of a valid document must keep the
+/// parser total: it either succeeds or raises a parse error (`valid` agrees
+/// with `parse`), and when the mutant still parses, the parsed value is a
+/// fixed point of restringify-and-reparse.
+test "parse stays total under single code-unit deletion and replacement" {


Fixed in 29009f6 by taking the stronger option: the test now mutates genuine UTF-16 code units via code_units() instead of Char-level to_array(). Deletion/replacement can now split an astral surrogate pair, and the replacement unit is drawn from the full 16-bit range (including lone surrogates). Verified green on wasm-gc, js, and native.

New property families complementing json/quickcheck_test.mbt: - adversarial strings (every control character, JSON syntax characters, BMP boundaries, astral pairs, and lone surrogates) roundtrip through stringify/parse as both values and object keys, across indent and escape_slash options; - the fully \uXXXX-escaped spelling of those strings parses back to the exact original, including surrogate pairs split across two escapes and lone-surrogate escapes; - every textual spelling of zero preserves the sign of zero bitwise; - Int64/UInt64 literals roundtrip textually (repr preserved past 2^53); - random legal whitespace inserted between tokens never changes the parsed value; - duplicate object keys: last occurrence wins; - single code-unit deletions/replacements of valid documents keep the parser total (parse agrees with valid, no panics) and successfully parsed mutants are fixed points of restringify-and-reparse; - deterministic pins for the default 1024 nesting-depth boundary and for surrogate handling. The AdvString generator shrinks at the UTF-16 code-unit level so counterexamples can minimize to half of a surrogate pair. These properties found the two parser bugs fixed in the previous commit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…tate at code-unit level - The nesting-limit test now asserts that documents exactly 1024 levels deep parse successfully for BOTH arrays and objects (previously it only checked valid() for the array and only checked the 1025-deep failure for objects), and the 1025-deep variables are renamed too_deep_array / too_deep_object to reflect their depth. - The mutation-totality test now genuinely mutates single UTF-16 code units via code_units() instead of Char-level to_array(), so deleting or replacing a unit can split an astral surrogate pair, and the replacement unit is drawn from the full 16-bit range (including lone surrogates) — strictly stronger fuzzing that matches the test's name and doc comment. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…e policy The parser now keeps strings Unicode well-formed (#4056): unpaired surrogates — raw or as \uXXXX escapes — are rejected with a ParseError instead of being passed through. Update the adversarial suite to match: - the roundtrip and fully-escaped generators produce only Unicode scalar values (astral pairs still included), and the AdvString shrinker drops whole characters so candidates stay well-formed; - new property: a lone surrogate injected at any position of a hostile string — raw via stringify or spelled as a \uXXXX escape, with escapes/astral pairs/control characters nearby — is always rejected cleanly (parse raises, valid is false, never an abort); - the deterministic surrogate pins now assert rejection for raw, escaped, reversed-pair, and mixed raw/escaped-half spellings, while well-formed pairs (raw or split across two escapes) still parse. Note: "zero literals preserve the sign of zero" requires the parse(-0) fix from #4061 (based on main) and fails until that lands in this branch's history; all other tests are green on wasm-gc, js, and native. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Copilot AI lite review requested due to automatic review settings August 14, 2026 08:21

Copilot started reviewing on behalf of bobzhang August 14, 2026 08:21 View session

Copilot AI reviewed Aug 14, 2026

View reviewed changes

bobzhang mentioned this pull request Aug 14, 2026

fix(json): reject raw unpaired surrogates in string lexing instead of aborting #4056

Open

bobzhang force-pushed the agent/quickcheck-json-deep branch from 8360ba6 to fc24c9c Compare August 14, 2026 08:31

bobzhang changed the base branch from main to agent/fix-json-lexer August 14, 2026 08:31

bobzhang force-pushed the agent/fix-json-lexer branch 2 times, most recently from 6af3721 to 5cf2ae6 Compare August 15, 2026 02:19

bobzhang mentioned this pull request Aug 15, 2026

fix(json): preserve the sign of -0 in the integer fast path #4061

Merged

bobzhang force-pushed the agent/quickcheck-json-deep branch from 29009f6 to c217fa5 Compare August 15, 2026 02:20

bobzhang changed the base branch from agent/fix-json-lexer to agent/fix-json-negative-zero August 15, 2026 02:20

bobzhang force-pushed the agent/fix-json-negative-zero branch from 15d2ac4 to b93a00e Compare August 15, 2026 02:55

bobzhang force-pushed the agent/quickcheck-json-deep branch from c217fa5 to 10d4e23 Compare August 15, 2026 03:02

bobzhang changed the base branch from agent/fix-json-negative-zero to agent/fix-json-lexer August 15, 2026 03:03

bobzhang mentioned this pull request Aug 15, 2026

json: parser constructs ill-formed strings from unpaired \uXXXX escape sequences #4062

Closed

bobzhang force-pushed the agent/fix-json-lexer branch from ca31837 to 41233d9 Compare August 15, 2026 03:17

bobzhang mentioned this pull request Aug 15, 2026

fix(json): reject unpaired \uXXXX surrogate escapes in strings #4064

Merged

bobzhang and others added 3 commits August 15, 2026 11:19

bobzhang force-pushed the agent/quickcheck-json-deep branch from 10d4e23 to f9a3219 Compare August 15, 2026 03:20

Navigation Menu

Uh oh!

test(json): adversarial QuickCheck roundtrip and parser-robustness tests - #4045

test(json): adversarial QuickCheck roundtrip and parser-robustness tests#4045
bobzhang wants to merge 3 commits into
agent/fix-json-lexerfrom
agent/quickcheck-json-deep

bobzhang commented Aug 14, 2026 •

edited

Loading

Uh oh!

coveralls commented Aug 14, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

bobzhang Aug 15, 2026

Uh oh!

bobzhang Aug 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

bobzhang commented Aug 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

New property families (json/quickcheck_adversarial_test.mbt)

Verification

Uh oh!

coveralls commented Aug 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Coverage Report for CI Build 6069

Coverage: 90.663%

Details

Uncovered Changes

Coverage Regressions

Coverage Stats

💛 - Coveralls

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

bobzhang Aug 15, 2026

Choose a reason for hiding this comment

Uh oh!

bobzhang Aug 15, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

bobzhang commented Aug 14, 2026 •

edited

Loading

New property families (`json/quickcheck_adversarial_test.mbt`)

coveralls commented Aug 14, 2026 •

edited

Loading