Skip to content

fix(sparksql): validate json_object_keys input - #865

Open
SC123667 wants to merge 1 commit into
bytedance:mainfrom
SC123667:codex/implement-json-object-keys-742
Open

fix(sparksql): validate json_object_keys input#865
SC123667 wants to merge 1 commit into
bytedance:mainfrom
SC123667:codex/implement-json-object-keys-742

Conversation

@SC123667

Copy link
Copy Markdown

What problem does this PR solve?

Issue Number: close #742

json_object_keys used simdjson's On-Demand API without checking lazy parse errors while reading the root type, iterating fields, or unescaping keys. Malformed JSON could therefore raise a parser exception instead of returning NULL, and trailing content could be accepted.

Type of Change

  • 🐛 Bug fix (non-breaking change which fixes an issue)
  • ✨ New feature (non-breaking change which adds functionality)
  • 🚀 Performance improvement (optimization)
  • ⚠️ Breaking change (fix or feature that would cause existing functionality to change)
  • 🔨 Refactoring (no logic changes)
  • 🔧 Build/CI or Infrastructure changes
  • 📝 Documentation only

Description

  • Check the lazy root-type parse result before accessing it.
  • Return NULL for malformed object fields and invalid escaped keys.
  • Reject valid objects followed by trailing non-whitespace content.
  • Cover the issue examples, nested objects, non-object inputs, malformed structures, invalid literals, escaped keys, and trailing content.

Performance Impact

  • No Impact: The change adds lightweight error checks to parser operations already performed by this function.
  • Positive Impact: I have run benchmarks.
  • Negative Impact: Explained below (e.g., trade-off for correctness).

Release Note

Release Note:
- Fixed SparkSQL json_object_keys to return NULL for malformed JSON instead of surfacing parser errors or accepting trailing content.

Checklist (For Author)

  • I have added/updated unit tests (ctest).
  • I have verified the code with local build (Release/Debug).
  • I have run clang-format / linters.
  • (Optional) I have run Sanitizers (ASAN/TSAN) locally for complex C++ changes.
  • No need to test or manual test.

Breaking Changes

  • No
  • Yes (Description: ...)

Signed-off-by: SC123667 <205642379+SC123667@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] implement json_object_keys for sparksql

1 participant