Add comprehensive CI, full installation tests, and security automation

[akan72]

Goal

Add a dedicated CI pull request that continuously verifies this dotfiles repository can bootstrap a clean machine, catches configuration errors, and blocks secrets and unsafe dependency/workflow changes before merge.

This issue records the agreed scope and should be the source of truth for the CI PR.

Decisions

• Run the full integration installation on every pull request, not only on a schedule.
• Test every PR on both supported platforms after the cross-platform installer work lands:
  • Linux (ubuntu-latest)
  • macOS (a pinned GitHub-hosted macOS runner label)
• Keep fast static/security checks in separate parallel jobs so failures are reported before the full installation finishes.
• Accept the additional runtime and exposure to upstream network/Homebrew/plugin outages in exchange for stronger confidence on every change.
• Do not use path filtering to skip integration tests for documentation-only PRs; every PR gets the same required checks.
• Use Gitleaks as the primary secret scanner because the same pinned implementation works locally through pre-commit and in CI.
• Treat Socket as optional/complementary dependency supply-chain analysis, not as a replacement for secret scanning.

Existing PRs and dependencies

• Rebase/merge Make assimilate.sh cross-platform (macOS + Linux) #12, which makes assimilate.sh cross-platform and enables Linux integration testing.
• Rebase/merge Repo hygiene: secret gitignore patterns + gitleaks pre-commit #19, which adds the pinned Gitleaks pre-commit hook and installs/activates pre-commit.
• Resolve overlap with Harden assimilate.sh: arch-aware delta, quoted $HOME, perms #18, which separately hardens assimilate.sh.
• Account for Migrate Neovim plugins from Packer to lazy.nvim #17's Neovim migration and headless plugin sync behavior.
• Rebase/merge Manage Python with uv + 7-day minimum release age #23 for uv-managed Python and global uv/uv.toml configuration.
• Account for Set 7-day minimum release age for npm and bun installs #25's package minimum-release-age settings.

No existing PR currently adds .github/workflows, and the repository has no historical GitHub Actions runs.

Primary/recommended setup

1. Shared pre-commit checks

Use .pre-commit-config.yaml as the shared local/CI source of truth. CI must run:

pre-commit run --all-files

Include pinned hooks for:

• Gitleaks
• ShellCheck
• shfmt --diff
• trailing whitespace
• end-of-file fixing/checking
• merge-conflict markers
• private-key detection
• large added files
• YAML validation
• JSON validation, with exclusions for intentional JSONC files such as zed/settings.json
• actionlint after workflows are introduced
• zizmor after workflows are introduced

The local hook should scan staged changes quickly. CI should additionally scan complete Git history using a full checkout (fetch-depth: 0).

2. Full installer integration on every PR

Run a required Linux/macOS matrix on every pull_request and on pushes to main.

For each platform:

• Use a fresh isolated HOME under the runner's temporary directory.
• Place or symlink the checkout at $HOME/dotfiles, matching the installer's documented contract.
• Run the real ./assimilate.sh without a links-only or package-skip mode.
• Verify the command exits successfully.
• Verify every expected managed file points to the correct repository source.
• Verify expected installed tools and pinned versions where applicable.
• Exercise Neovim's headless startup/plugin synchronization.
• Run the installer a second time and verify idempotency.
• Assert the second run does not create unnecessary backups or change managed state.
• Assert the repository remains clean after installation.
• Preserve useful logs while avoiding global set -x output that could expose sensitive values.

Before enabling the idempotency assertion, fix the current sym behavior:

• An already-correct symlink should be a no-op rather than being backed up and recreated.
• Backup names must be collision-safe. Using only basename plus seconds-resolution timestamps can collide, e.g. VS Code and Zed both backing up settings.json in the same second.

3. uv malware protection

• Set UV_MALWARE_CHECK=1 by default in the shared shell environment so normal uv sync operations perform malware checking.
• Set UV_MALWARE_CHECK=1 explicitly in CI.
• Require uv >= 0.11.16, the version that introduced this environment variable.
• Add an automated assertion that the environment variable is present in Bash and Zsh sessions.
• Document that the check is currently a uv preview feature and performs an OSV-backed scan of the lockfile for OpenSSF malicious-package advisories after uv sync.
• Keep the exclude-newer = "7 days" policy from Manage Python with uv + 7-day minimum release age #23; it is complementary to malware advisory checking.

4. GitHub Actions security baseline

• Pin all third-party Actions to full commit SHAs and leave a version comment beside each pin.
• Set top-level permissions: {} and grant only required job-level permissions.
• Use actions/checkout with persist-credentials: false.
• Use full history only for the job that needs it.
• Add concurrency groups with cancel-in-progress: true for superseded PR commits.
• Avoid workflows that execute untrusted PR code with write tokens or repository secrets.
• Run actionlint for workflow syntax/type validation.
• Run zizmor for GitHub Actions security analysis; upload SARIF when useful.
• Add Dependabot version updates for the github-actions ecosystem so pinned action SHAs remain maintainable.

Worthwhile CI checks

Shell and installer

• bash -n for Bash scripts/configuration.
• zsh -n for Zsh configuration.
• ShellCheck.
• shfmt check/diff mode.
• Verify assimilate.sh's shebang matches the language features it uses.
• Test backup behavior with pre-existing regular files, directories, correct symlinks, incorrect symlinks, dangling symlinks, paths containing spaces, and duplicate basenames.
• Test failure behavior for unsupported OS/architecture combinations and failed downloads/checksums.

Configuration validation

• Parse gitconfig with git config --file gitconfig --list.
• Validate strict JSON files and separately validate intentional JSONC files.
• Validate YAML files.
• Run luac -p over Hammerspoon Lua files.
• Start Neovim headlessly with the managed configuration.
• Load tmux configuration in an isolated tmux server.
• Run docker compose config --quiet.
• Run Hadolint against scripts/docker/Dockerfile.notebook.
• Consider a Docker image build/smoke test if runtime remains acceptable.

Dependency and supply-chain security

• Run pip-audit or OSV-Scanner against scripts/docker/requirements.txt.
• Keep Gitleaks as the blocking secret scanner in pre-commit and CI.
• Consider TruffleHog only if verified-live credential checks add enough value to justify overlapping scans and external verification calls.
• Evaluate Socket for supported manifests such as requirements.txt and GitHub Actions. Do not assume it covers Brewfile; current Socket manifest documentation does not list it.
• Verify downloaded release assets with pinned SHA-256 checksums.
• Keep cloned tools/plugins pinned to immutable commits.
• Pin container base images by digest or add automated base-image update tracking.

GitHub-native security and repository features

These are currently disabled or not configured and should be enabled alongside the CI PR:

• Secret scanning.
• Secret scanning push protection.
• Secret scanning validity checks.
• Secret scanning for non-provider patterns, if available for the repository.
• Dependency graph.
• Dependabot alerts.
• Dependabot security updates.
• Dependabot version updates for GitHub Actions and Python/pip manifests.
• Code scanning/SARIF ingestion for tools such as zizmor where appropriate.
• A branch ruleset requiring the pre-commit/security and Linux/macOS integration checks before merge.
• Restrictive default workflow token permissions at the repository level.

Workflow shape

Recommended required jobs:

1. pre-commit — shared fast format/lint/security hooks.
2. secret-history — complete-history Gitleaks scan.
3. dependency-audit — Python/OSV checks and optional Socket analysis.
4. actions-security — actionlint and zizmor.
5. install (ubuntu) — full clean-home assimilation, assertions, and second-run idempotency.
6. install (macos) — full Homebrew/cask/tool assimilation, assertions, and second-run idempotency.

All jobs should run in parallel where dependencies allow. The two full installer jobs remain required on every PR even if the fast jobs have already passed.

Acceptance criteria

• A new PR automatically runs all required jobs above.
• The full Linux and macOS installations run on every PR and push to main.
• The installer passes twice in a clean isolated home without destructive backup collisions or unnecessary changes.
• Local pre-commit and CI enforce the same pinned core checks.
• Full Git history is scanned for secrets and the initial baseline passes.
• UV_MALWARE_CHECK=1 is enabled locally and in CI with a compatible uv version.
• Workflow files pass actionlint and zizmor.
• GitHub-native secret/dependency protections are enabled.
• Required checks are enforced by a branch ruleset.
• README/contributor documentation explains how to run the CI checks locally.

Known tradeoffs

Running full installations on every PR will make CI slower and can fail due to Homebrew, GitHub release downloads, rustup, package registries, or plugin hosts even when repository code is unchanged. This is an explicit choice: prioritize end-to-end reproducibility on every change. Mitigate it with immutable pins, checksums, minimal safe caching, concurrency cancellation, clear logs, and deterministic assertions rather than moving the test to a schedule.

References

• Make assimilate.sh cross-platform (macOS + Linux) #12 — cross-platform installer
• Harden assimilate.sh: arch-aware delta, quoted $HOME, perms #18 — installer hardening
• Repo hygiene: secret gitignore patterns + gitleaks pre-commit #19 — Gitleaks pre-commit/repository hygiene
• Manage Python with uv + 7-day minimum release age #23 — uv-managed Python and minimum release age
• Set 7-day minimum release age for npm and bun installs #25 — npm/bun minimum release age
• Gitleaks pre-commit hook: https://github.com/gitleaks/gitleaks/blob/v8.30.1/.pre-commit-hooks.yaml
• GitHub Actions hardening: https://docs.github.com/en/actions/security-for-github-actions/security-guides/security-hardening-for-github-actions
• uv malware checks: https://docs.astral.sh/uv/concepts/projects/sync/#malware-checks
• Socket manifest support: https://docs.socket.dev/docs/manifest-file-detection-in-socket
• zizmor integrations: https://github.com/zizmorcore/zizmor/blob/main/docs/integrations.md

[akan72]

Implemented in draft PR #31, stacked on #12.

Verified runner choices:

• macos-26 is the standard arm64 GitHub-hosted image and exercises the Apple Silicon installer path.
• ubuntu-24.04 is x64.

Hosted run 31763320564 passes all six jobs: two-pass installs on both platforms, pre-commit/Gitleaks, full-history Gitleaks, static checks, actionlint, and zizmor. No local dotfiles installation was used.

One adjustment to the original isolated-HOME proposal: the jobs use the hosted VM default HOME, which is already disposable. Overriding HOME broke the macOS image’s Homebrew trust and post-install state for preinstalled tools. The checkout is still placed at $HOME/dotfiles, and the entire VM is discarded after the job.

[akan72]

Implemented the VPS validation discussed here in draft PR #31. CI now runs two-pass, non-root installs in digest-pinned Ubuntu 24.04 and Amazon Linux 2023 containers on every PR, in addition to the hosted Ubuntu and macOS 26 jobs. The clean checks exposed and drove fixes for masked pipeline failures, an annotated rustup tag being used as a commit, Neovim version drift, and Amazon Linux curl-minimal compatibility. Final run 31766063939 passes all eight jobs.