Skip to content

Stop the SQLite timeline table dropping rows with nothing said - #349

Open
dchaudhari7177 wants to merge 1 commit into
RyanDFIR:mainfrom
dchaudhari7177:fix/343-sqlite-timeline-silent-drop
Open

Stop the SQLite timeline table dropping rows with nothing said#349
dchaudhari7177 wants to merge 1 commit into
RyanDFIR:mainfrom
dchaudhari7177:fix/343-sqlite-timeline-silent-drop

Conversation

@dchaudhari7177

Copy link
Copy Markdown
Contributor

Towards #343.

Scope — most of that issue is already fixed

Reading the code against the issue, three of its four parts have landed since it was written:

Issue item State on main
HindsightEncoder.default() dispatches on Chrome.X, so Firefox siblings fall through Fixed — the chain tests WebBrowser.X
Service Worker classes have no encoder branch Fixed — all five ServiceWorker*Item branches exist
JSONL drop is invisible Fixedwarn_unhandled() reports class and derived data_type
generate_sqlite() storage loop has no else Fixedunhandled_storage counter, and a service worker INSERT branch

What is left is the one the issue names but that did not get the fix: "both dispatchers need a final else that logs". The storage loop got its else. The timeline loop did not.

The live drop it is hiding

This is not only a defensive change. get_extension_settings appends extension (installed) and extension (updated) items to parsed_artifacts (chrome.py:2414), and the XLSX writer has a branch for them (analysis.py:1952). The SQLite timeline dispatch has neither a branch nor an else, so those events are dropped.

Measured on main, feeding two such items through generate_sqlite:

parsed_artifacts: ['extension (installed)', 'extension (updated)']
SELECT type FROM timeline  ->  []

Same shape as the Service Worker case the issue measured: parsed correctly, counted correctly, reported by the run, and then never written — so anyone working from SQLite has no indication the artifact exists.

The change

  • extension joins the preference / site setting / notification / … branch, whose column set it already fits (PreferenceItem, and the XLSX branch writes exactly these fields).
  • A final else counting into unhandled_timeline, reported one line per row_type with a count, mirroring unhandled_storage exactly.

The else is the more important half. The branch fixes the one artifact missing today; the else is what makes the next one a reported number rather than a silent absence.

Verification

Four tests driving generate_sqlite against a real on-disk SQLite file and reading the rows back. Three fail on main:

FAILED test_extension_install_events_reach_the_timeline_table
FAILED test_an_unhandled_row_type_is_reported_rather_than_dropped_silently
FAILED test_unhandled_rows_are_counted_not_reported_once_each

The fourth pins that a handled row_type is not reported as missing, so the else cannot start crying wolf.

Full suite on this branch: 243 passed, 2 skipped, 80 subtests passed. Python 3.12.4.

Not done here

I have deliberately left #343 open rather than closing it, because two of its questions are yours to settle and are not answered by this:

  • whether the SQLite storage table should hold Service Worker rows or get its own table (they are in storage today);
  • whether Service Worker JSONL rows should keep borrowing chrome:local_storage:entry-style data_type names — they currently use chrome:service_worker:*, which reads correct to me, but you flagged the collection they live in rather than the names.

Happy to take either on separately.

The storage loop in generate_sqlite grew a final else that counts what it cannot
place and logs one line per row_type. The timeline loop never got one, so an
artifact whose row_type no branch matched was discarded in silence -- the same
defect, in the dispatcher that was not fixed.

It is not hypothetical. get_extension_settings appends `extension (installed)`
and `extension (updated)` items to parsed_artifacts, and the XLSX writer has a
branch for them, but the SQLite timeline dispatch has none. Every extension
install and update event was therefore absent from the SQLite output, with no
indication the artifact existed:

    parsed_artifacts: extension (installed), extension (updated)
    SELECT type FROM timeline -> []

Both now land, and any future unhandled row_type is reported the way the storage
loop reports its own, counted per row_type rather than once per row.

The else matters more than the branch. The branch fixes the one artifact that is
missing today; the else is what makes the next one a reported number rather than
a silent absence.

Four tests, three failing on main.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant