Skip to content

chore: streamline toolchain, packaging and CI - #314

Merged
Maximvdw merged 9 commits into
masterfrom
chore/streamline
Aug 8, 2026
Merged

chore: streamline toolchain, packaging and CI#314
Maximvdw merged 9 commits into
masterfrom
chore/streamline

Conversation

@Maximvdw

@Maximvdw Maximvdw commented Aug 8, 2026

Copy link
Copy Markdown
Member

Brings @openhps/core onto the shared OpenHPS toolchain baseline and fixes several packaging defects that were shipping.

Toolchain

TypeScript 5.9, ESLint 9 with a native flat config, typescript-eslint 8, prettier 3, mocha 11, chai 6, c8, typedoc 0.28, @types/node 24, engines.node >=22. npm replaces yarn fleet-wide.

TypeScript 7 and ESLint 10 are deliberately not adopted: typedoc 0.28 peers typescript 5.0-6.0, typescript-eslint 8 peers <6.1, and eslint-plugin-import@2.32 peers eslint <=9.

Packaging defects fixed

  • exports.import pointed at dist/esm5, leaving the modern dist/esm output unreachable to every consumer. The esm5 target was pure duplication: its tsconfig used target es6 while the esm one used es2015 — the same TypeScript alias, with the same module: es2020.
  • dist/esm had never been loadable by Node as ESM. Three causes, all needing to be fixed together: no type marker (Node read it as CommonJS), TypeScript-emitted directory imports (export * from './graph'ERR_UNSUPPORTED_DIR_IMPORT), and typedjson deep imports rewritten to a lib/esm path that cannot be loaded from an ES module at all — typedjson declares no "type": "module" and its ESM build uses extensionless internal imports. Its CommonJS build can be loaded, so the specifiers now keep pointing there and only gain the extension Node requires.
  • tslib was undeclared despite the tsconfig setting both importHelpers and noEmitHelpers, a combination that always emits import ... from "tslib". It resolved only by hoisting.
  • The files array shipped all three internal.* root shims but none of the lite.* ones, so @openhps/core/lite resolved to nothing from a published tarball.
  • shelljs was required by scripts/build.three.js but declared nowhere — one dependency-tree change away from breaking npm run build at its very first step. That script is now written on node:fs.
  • @types/three was 12 minors behind the three it describes, and build.three.js copies the types over the vendored sources, so the drift was baked into dist/types/three. Aligning them required switching Matrix{3,4}.elements to the fixed-length tuple types and dropping a dead Matrix re-export.
  • Circular, unused devDependencies on @openhps/geospatial and @openhps/video put two copies of core — with two reflect-metadata decorator registries — into core's own test tree.
  • lodash.clonedeep was loaded with a bare require in a file re-exported through the public barrel, which tsc leaves intact in ESM output: a ReferenceError for any native-ESM consumer. Replaced by an internal deep clone. structuredClone is not a substitute, since the serialization layer clones typedjson metadata holding constructor references and function hooks.

Testing and CI

  • Coverage moves from nyc to c8. nyc 18's spawn-wrap breaks down on Node 22: the tests visibly run while the parent process reports 0 passing and 0% (0/1 statements). Real measured figures are 85.8% statements, 85.97% branches, 73.75% functions, and thresholds are set from those rather than the previous 50/50/50/50, which was never enforceable.
  • cover:ci now writes a real test-results.xml. reporterEnabled had been sitting at the top level of .mocharc.json where mocha ignores it, so the JUnit file CI consumed had never been produced.
  • CI adopts the OpenHPS/workflows reusable workflow: Node 22 and 24 instead of EOL Node 18, every job installs its own dependency tree rather than depending on an actions/cache hit for node_modules, and lint no longer waits on build (it previously ran type-aware rules with src/three absent).
  • paper.yml filtered on './docs/paper/**'. GitHub path patterns are repository-root-relative and reject a leading ./, so those triggers had never once fired.

Verified locally

345 tests passing, lint clean (0 errors), publint no errors, all four bundle budgets met, and both require() and import() resolving 172 exports.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FtN6Tx33QW5a6AtCH1zYQh

Maximvdw and others added 9 commits August 8, 2026 21:36
tsconfig.bundle.esm5.json used target "es6" while tsconfig.bundle.esm.json used
"es2015" -- the same TypeScript target alias, with the same module: es2020. The
two outputs were identical apart from which typedjson lib directory the babel
pass rewrote imports to, so esm5 never provided any downlevelling and was pure
duplication. Meanwhile exports.import pointed at dist/esm5, which left the
dist/esm output unreachable to every ESM consumer.

The typedjson lib/cjs -> lib/esm specifier rewrite now runs as a zero-dependency
script, replacing babel, @babel/cli and babel-plugin-replace-imports. @babel/core
and @babel/preset-env stay: build.three.js needs them to transpile three's ESM
sources to CommonJS.

build.three.js is also rewritten on node:fs. It required shelljs, which was never
a declared dependency and resolved only through hoisting -- one dependency-tree
change away from breaking `npm run build` at its very first step.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FtN6Tx33QW5a6AtCH1zYQh
@types/three was pinned to ^0.163 while three itself was ^0.175. build.three.js
copies @types/three/src over the vendored three sources, so twelve minors of
type/implementation drift were being baked into dist/types/three.

Aligning the two is not free, because three tightened its types in between:

- Matrix3.elements and Matrix4.elements went from number[] to the fixed-length
  Matrix3Tuple / Matrix4Tuple, so the serializable subclasses have to match the
  base declaration. Both tuple types are now re-exported from the math _internal
  barrel. This is type-only; the runtime value is still an array.
- The Matrix interface export was dropped. Nothing in this package used it -- its
  only occurrence anywhere is inside a vendored three.js docstring -- so the dead
  re-export is removed rather than replaced.

UUID.fromBuffer is unrelated to three: TypeScript 5.9 made TypedArrays generic
over their backing buffer, so `new Uint8Array()` narrows to
Uint8Array<ArrayBuffer> and no longer accepts the Uint8Array<ArrayBufferLike>
that BufferUtils.concatBuffer returns.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FtN6Tx33QW5a6AtCH1zYQh
- src/graph/PushPromise.ts imported `resolve` from 'path' and never used it. The
  three `resolve` references in the file are the Promise executor parameter, which
  shadows the import. Beyond being dead it pulled node:path into the browser
  bundle graph, which is why webpack carries a `fallback: { path: false }`.
- _eslint.config.js was a dormant flat-config draft that could never load (the
  leading underscore) and would have regressed linting if renamed: its entire
  `extends` array was commented out, eslint-plugin-deprecation was imported but
  never registered, and it referenced an uninstalled eslint-plugin-compat.
- .gitconfig contained a single malformed line (`core.safecrlf = false:`) and was
  inert regardless.

src/service/index.lite.ts is removed in the preceding commit: it was unreachable,
since src/index.lite.ts imports './service' rather than './service/index.lite'.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FtN6Tx33QW5a6AtCH1zYQh
npm is now the package manager fleet-wide. The fleet was split 13 yarn / 12 npm /
2 with both lockfiles, and every CI workflow ran a bare `yarn install` or
`npm install` rather than a frozen install, so lockfiles silently churned in CI.

Yarn 1.22 is unmaintained; npm additionally gives `overrides` for pinning
transitive CVE fixes, `--provenance` for sigstore attestations, and ships with
Node 22 and 24 so the CI matrix needs no extra install step.

Landed as its own commit so `git bisect` can distinguish "npm resolved a
dependency differently" from the deliberate version bumps that follow.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FtN6Tx33QW5a6AtCH1zYQh
Replaces a hand-forked main.yml with a thin caller. The fleet had 24 distinct
variants of this file; only three repositories shared one.

What this fixes for core specifically:

- Node 18 only -> a 22 and 24 matrix. Node 18 has been EOL since April 2025, so
  CI was validating an unsupported runtime and nothing else.
- Only the build job ran setup-node and installed dependencies. quality,
  test_coverage and documentation relied on an actions/cache hit for
  **/node_modules and used the runner's default Node; on a miss they failed
  confusingly or silently linted nothing. Every job now installs its own tree.
- The quality job ran type-aware ESLint rules without src/three present, because
  it did not restore the build cache. lint now runs the prebuild script itself.
- paper.yml filtered on './docs/paper/**'. GitHub path patterns are
  repository-root-relative and reject a leading './', so the push and
  pull_request triggers had never once fired.
- dependabot.yml was daily with no PR limit, grouping or ignores, which is why
  ~10 abandoned dependabot branches accumulated. Now weekly, limited, and grouped
  so the lint and test stacks move as units.
- auto-merge combined `permissions: write-all` with a blanket minor-version
  auto-merge; it is now least-privilege and restricted to devDependency
  patch/minor bumps.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FtN6Tx33QW5a6AtCH1zYQh
TypeScript 5.9, ESLint 9 with a native flat config, typescript-eslint 8,
prettier 3, mocha 11, chai 6, nyc 18, typedoc 0.28, @types/node 24,
engines.node >=22. TypeScript 7 and ESLint 10 are deliberately not adopted:
typedoc 0.28 peers typescript 5.0-6.0, typescript-eslint 8 peers <6.1, and
eslint-plugin-import 2.32 peers eslint <=9.

Packaging fixes, all of which were shipping:

- exports.import pointed at dist/esm5, so the modern dist/esm output was
  unreachable to every consumer. Both `module` and `exports.import` now point at
  dist/esm, and `types` comes first in each condition, where its order matters.
- The `files` array shipped all three internal.* root shims but none of the
  lite.* ones, so `@openhps/core/lite` resolved to nothing from a published
  tarball.
- tslib was undeclared despite tsconfig setting both importHelpers and
  noEmitHelpers, a combination that always emits `import ... from "tslib"`. It
  resolved only by hoisting.
- devDependencies on @openhps/geospatial and @openhps/video were unused (verified
  by grep across src, test, examples and docs) and formed a core -> geospatial ->
  core cycle that put two copies of core, with two reflect-metadata decorator
  registries, into core's own test tree.
- nyc excluded src/service/_internal/DummyDataService.ts, which does not exist --
  the real file is src/service/DummyDataService.ts and was therefore never
  excluded -- plus src/**/WorkerService.ts, which matches nothing here.
- size-limit budgeted 2 of the 5 bundles webpack emits. All four minified bundles
  are now budgeted, from measured brotli sizes rather than the previous guess.
- typedoc-umlclass had to move to 0.10: 0.8 peers typedoc 0.24-0.25, so npm
  install fails outright against typedoc 0.28. This was originally scheduled as a
  separate follow-up; it is not separable.

Tooling that was declared but never actually wired up now works: husky has hooks
and a prepare script, commitlint has a config, and lint-staged keeps the
pre-commit hook fast. cover:ci writes a real test-results.xml -- reporterEnabled
had been sitting at the top level of .mocharc.json where mocha ignores it, so the
JUnit file CI consumed had never been produced.

standard-version is replaced by commit-and-tag-version (unmaintained since 2023),
and eslint-plugin-deprecation by @typescript-eslint/no-deprecated (archived).
CONTRIBUTING.md and SECURITY.md move to the org-wide OpenHPS/.github repository.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FtN6Tx33QW5a6AtCH1zYQh
Formatting only, kept as its own commit so the toolchain diff stays reviewable.

The lint script was `eslint src/ --ignore-pattern three`, so test/ had never been
checked by prettier. Widening it to `eslint .` surfaced 868 formatting errors
across the test suite; all but one were autofixable, and the exception was a
`let` that is never reassigned in test/specs/model.spec.ts.

Three rules are relaxed for test/** in the shared config, where the fixtures
exercise the patterns deliberately as the subject under test: class/interface
declaration merging, `Object` as a type name, and `new Array()`. jsdoc/no-types
is off fleet-wide because OpenHPS documents parameter types in JSDoc on purpose --
TypeDoc renders them.

Lint is now clean: 0 errors, 186 warnings (unused vars, and 5
@typescript-eslint/no-deprecated hits where core uses its own deprecated APIs).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FtN6Tx33QW5a6AtCH1zYQh
dist/esm has never been loadable outside a bundler. Three separate problems, all
of which had to be fixed together for the "import" condition of the exports map
to resolve at all:

1. No "type" field, so Node read every dist/esm/*.js as CommonJS and died on the
   first import statement. The package root is now explicitly "commonjs" and the
   build writes dist/esm/package.json with {"type":"module"} to scope the ESM
   output, which avoids renaming every emitted file to .mjs.
2. TypeScript emits directory imports verbatim under module: es2020
   (`export * from './graph'`). Bundlers resolve those; Node rejects them with
   ERR_UNSUPPORTED_DIR_IMPORT. All relative specifiers in the ESM output are now
   rewritten to explicit file paths.
3. The removed babel step rewrote typedjson's deep imports to
   `typedjson/lib/esm/`, which cannot be loaded from an ES module at all:
   typedjson declares no "type": "module" and its ESM build itself uses
   extensionless internal imports. Its CommonJS build can be loaded, and Node's
   cjs-module-lexer picks up the named exports, so the specifiers keep pointing at
   lib/cjs and only gain the extension Node requires.

lodash.clonedeep is replaced by an internal deep clone. `const cloneDeep =
require('lodash.clonedeep')` sat in a file re-exported through the public barrel,
and tsc leaves a bare require intact in ESM output, so it was the last thing
making `import '@openhps/core'` throw "require is not defined in ES module scope".
structuredClone is not a substitute: the serialization layer clones typedjson
member metadata holding constructor references and serializer hooks, and
structuredClone throws on functions. The replacement matches lodash by carrying
functions over by reference and preserving prototypes, and drops a dependency.

Verified: require('./dist/cjs/index.js') and import('./dist/esm/index.js') both
resolve 172 exports, publint reports no errors, and the full suite passes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FtN6Tx33QW5a6AtCH1zYQh
nyc 18's spawn-wrap breaks down on Node 22: the tests visibly run and print their
results, while the parent process reports "0 passing (0ms)" and 0% (0/1
statements). c8 uses V8's native coverage, so there is no require hook and no
process wrapping involved. This also needs sourceMap: true in tsconfig.spec.json,
which was false, so that V8 coverage maps back to the TypeScript sources.

Coverage was never actually gated before. The declared 50/50/50/50 thresholds
could not fail, because cover:ci's reporter was misconfigured and the CI job had no
node_modules on a cache miss. The real, measured figures are 85.8% statements,
85.97% branches, 73.75% functions, and thresholds are set from those, floored to
the nearest 5. They only ever ratchet upward from here.

test/.mocharc.ci.json no longer uses mocha's `extends`, which resolves relative to
the working directory rather than to the config file and silently drops settings
including `timeout` -- every async test was falling back to the 2000ms default. It
now writes a real 87 KB test-results.xml with 333 cases, where the JUnit file CI has
been consuming for years had never been produced at all.

The worker spec is excluded from the coverage run only: under instrumentation the
threads library exceeds its own hard-coded 60s worker-init timeout and takes the
whole run down with it, so mocha never flushes its reporters. Those tests remain in
`npm test`, which is the correctness gate, and the timeout belongs to the tracked
work to replace threads.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FtN6Tx33QW5a6AtCH1zYQh
@Maximvdw
Maximvdw merged commit 17a37be into master Aug 8, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant