ccl_chromium_reader is a hard runtime dependency that exists only as a git repo, and PyPI
rejects uploads whose metadata contains direct-URL dependencies. So it cannot be declared
in pyproject.toml, and a plain pip install pyhindsight yields a package that fails with
a raw ModuleNotFoundError the first time it is used, with no hint at the fix.
This is the worst first-run experience in the project. Note the frozen binaries already
ship ccl inside the exe, so the released artifact is effectively vendored already. Only the
pip path is broken.
There is a second, separate reason to care, which is control over the library's I/O.
ccl_chromium_reader reports records it cannot decode with bare print(), 43 calls across
7 modules. On Windows a redirected stdout is cp1252, and printing undecodable record text
into that stream from inside a running parser raises, at which point the parser's own
except reports its artifact as unreadable. That cost 954 Session Storage records across
three of nine test profiles before a stdout guard was added. The guard defuses all 43 calls
and stays regardless as defence against any library, but it is a workaround: those calls
should be log.warning(). Hindsight also monkeypatches the sessionstorage module's logger
at runtime because the module sets its own logger to None, which is the same problem
solved the same unsatisfying way.
Options, in preference order:
- Ask CCL to publish to PyPI. It already builds a clean wheel, so it is trivial on their
end, and dependencies then just gets ccl-chromium-reader>=0.3. This closes the
distribution half and does nothing for the control half.
- Vendor it. MIT permits it with attribution, and the repo already vendors Chromium protos.
A copy under pyhindsight/_vendor/ plus a shim that registers the vendored package under
the real name means no call-site changes are needed. Add a sync script in the manner of
regen-protos.sh.
- Minimum hardening regardless of the choice: guard the top-level import with a try/except
that prints the exact pip install git+... command instead of a raw traceback.
If vendoring, the discipline matters more than the copy. Vendor unmodified first at a
recorded commit and land each local change as its own reviewable commit, so "what have we
changed from upstream" stays answerable. File every local patch upstream, because the
failure mode of vendoring is a silent private fork where nothing ever gets reported. Keep
the reported version honest, since Hindsight logs the ccl module version and a report may
rest on it. And accept that drift is the real cost: Chrome's formats move constantly and
ccl tracks them, so a vendored copy that is not synced is worse than a pin.
For sizing, measured against the currently pinned commit: 22 Python files, 7,957 lines,
861 KB installed. MIT, with a per-file copyright header on 15 of the 22 files; the 7
without should be confirmed as covered by the repository LICENSE before relying on the
per-file headers.
ccl_chromium_readeris a hard runtime dependency that exists only as a git repo, and PyPIrejects uploads whose metadata contains direct-URL dependencies. So it cannot be declared
in
pyproject.toml, and a plainpip install pyhindsightyields a package that fails witha raw
ModuleNotFoundErrorthe first time it is used, with no hint at the fix.This is the worst first-run experience in the project. Note the frozen binaries already
ship ccl inside the exe, so the released artifact is effectively vendored already. Only the
pip path is broken.
There is a second, separate reason to care, which is control over the library's I/O.
ccl_chromium_readerreports records it cannot decode with bareprint(), 43 calls across7 modules. On Windows a redirected stdout is cp1252, and printing undecodable record text
into that stream from inside a running parser raises, at which point the parser's own
exceptreports its artifact as unreadable. That cost 954 Session Storage records acrossthree of nine test profiles before a stdout guard was added. The guard defuses all 43 calls
and stays regardless as defence against any library, but it is a workaround: those calls
should be
log.warning(). Hindsight also monkeypatches the sessionstorage module's loggerat runtime because the module sets its own logger to
None, which is the same problemsolved the same unsatisfying way.
Options, in preference order:
end, and
dependenciesthen just getsccl-chromium-reader>=0.3. This closes thedistribution half and does nothing for the control half.
A copy under
pyhindsight/_vendor/plus a shim that registers the vendored package underthe real name means no call-site changes are needed. Add a sync script in the manner of
regen-protos.sh.that prints the exact
pip install git+...command instead of a raw traceback.If vendoring, the discipline matters more than the copy. Vendor unmodified first at a
recorded commit and land each local change as its own reviewable commit, so "what have we
changed from upstream" stays answerable. File every local patch upstream, because the
failure mode of vendoring is a silent private fork where nothing ever gets reported. Keep
the reported version honest, since Hindsight logs the ccl module version and a report may
rest on it. And accept that drift is the real cost: Chrome's formats move constantly and
ccl tracks them, so a vendored copy that is not synced is worse than a pin.
For sizing, measured against the currently pinned commit: 22 Python files, 7,957 lines,
861 KB installed. MIT, with a per-file copyright header on 15 of the 22 files; the 7
without should be confirmed as covered by the repository LICENSE before relying on the
per-file headers.