feat(nist-ir): emit ai_reasoning rows and canonicalise AI verdict fields - #1051
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SOCPromoteAIPhaseOutputwas the only place that saw a parsed AI result, and it wrote nothing anywhere. A prompt that ran and returned nothing looked identical to a prompt that never ran, becausecontinueonerror: trueon the aiTask means neither fails the phase. War room entries are the only other evidence and every entry endpoint returns 303 on this tenant.ai_reasoning rows
One row to
xsiam_socfw_ir_execution_rawper AI execution, via the samesocfw-post-to-dataset/socfw_ir_execution_writerpathSOCCommandWrapperuses.reasoning_statusseparates the cases:promoted,parsed_not_promoted,parse_error,no_output.Structured verdict fields only — the narrative stays out; it is large, duplicates the layout, and is not what anyone queries on.
verdict_field_countmakes a thin result visible without unpacking the row.The write cannot fail the phase; a failure goes to
demisto.debug.Canonical vocabulary
The model is not constrained to an enum, so it invents spellings. Observed on deathstar in one run:
spread_levelasmulti-host,multiple_hosts,multi_hostandlateral;compromise_levelashost,host_and_identity,host_and_user,host_and_lateral,endpointandconfirmed;primary_entity_typeincludingWindowsServer2019. Anything keyed on these silently misses most rows —spread_level = "multi-host"matched about a third.Values are now snapped to canonical form before promotion, so context, layout and dataset share one vocabulary. An unrecognised value is not silently rewritten: the field becomes
unknown, the original is kept at<field>_raw, andcontract_unmappednames it.confirmedincompromise_levelis a certainty claim rather than a scope, so it has no correct target and is flagged rather than mapped.Verified on deathstar
Four replays. Before: four spellings of
spread_level, six values ofcompromise_level. After: two and two, with 2 of 37 rows correctly flagged. The telemetry also caught a realparse_error(1 in ~120) that was previously invisible.Prompts are tenant-local and versioned nowhere, so enforcement belongs here, in content that ships through the SDK and applies identically everywhere.
Labelled
version:patch→ 1.8.2.