various improvements by Byron · Pull Request #3032 · GitoxideLabs/gitoxide

Byron · 2026-09-29T19:48:45Z

Tasks

refackiew

A trusted rebase document can intentionally describe reference edits beyond those shown by its generated body. Exposed that edits to the undo queue itself were rejected only after those refs and other refs had already changed, destroying the very history needed to recover. Validate the undo change set before applying the checked, non-dereferencing reference transaction, then record its actual applied changes. Keep the complete document trusted: tag and remote-reference edits remain possible and are recorded for undo. Only unrecordable changes fail before any reference is written. Regression coverage applies edited state anchors to both queue refs and verifies that the queue and other refs survive. A companion test applies explicit tag/remote deletions and restores them through undo. The prior implementation fails the reference-preservation assertion. Git reference: `Documentation/git-update-ref.adoc` at `d38352cd43ab` describes checked reference updates; the Tix undo queue is an additional local invariant. Review correction: a `MustNotExist` update can succeed as a no-op if the ref already points at the requested target. The regression now models creation while editing the document and verifies undo does not delete the existing ref. Validation: all 134 edit tests pass with SHA-1 enabled.

A cloned repository may contain a local branch named `-f`. Returning through its symbolic Tix pin passed that name to `git checkout` as an option, forcing away uncommitted work while reporting a successful branch return. Reject leading-dash branch operands in `checkout_branch()` and route symbolic pin checkouts through that same guard. Existing checkout failure handling preserves the departure and return pin. A `--` separator would instead change the operand into a pathspec and would not perform the intended branch checkout. An isolated regression starts from `refs/heads/-f`, travels to an ancestor, changes a tracked file, and returns through the HEAD pin. Before the fix it reports success; afterwards it refuses the option and preserves the file, detached HEAD and pin. Explicit branch checkout shares the validation. Git reference: `refs.c::check_branch_ref()` at `d38352cd43ab` rejects leading `-` in porcelain branch names, although plumbing can still create such refs. Validation: all 21 time-travel tests pass with SHA-1 enabled.

Display quoting was private to `gitoxide-core`, preventing other crates from reusing its established output behavior. Expose it as `gix_quote::for_display()` and migrate the existing index-output callers. The helper returns borrowed display bytes while retaining the reusable scratch buffer. Ordinary Unicode stays readable; control characters, quotes, backslashes, and invalid UTF-8 retain their lossless debug representation. Keep the existing byte-quoting regression coverage with the shared helper. Assisted-by: GPT 6.0 Astra Co-authored-by: GPT 6.0 Astra <codex@openai.com>

Repository-derived commit titles and authors reached plain history and rebase todo output with terminal control sequences intact. A cloned commit or mailmap could therefore overwrite output or manipulate the terminal. Use `gix::quote::for_display()` after rendering plain metadata, including mailmapped authors and rebase anchor titles. Keep todo commands and machine-readable reference state intact, and document the display behavior in the Tix specification. Regressions cover ESC, BEL and Unicode control sequences in metadata and anchor titles. Todo parsing continues to accept quoted informational metadata. Git baseline: `pager.c::setup_pager()` at `d38352cd43ab` routes terminal output through a pager; Tix's plain stdout paths instead need explicit quoting. The tree after this commit matches the previously validated sanitizer fix, whose complete 461-test Tix suite passed with isolated Git configuration. The two terminal-control regressions also pass independently with `cargo test -p gix-tix --lib --features gix/sha1 terminal_controls`.

A report found that an inherited-PATH miss left a bare program name for shebang probing. That opened a same-named file in the current worktree and allowed its contents to choose the interpreter, instead of reporting a missing program. A successful lookup could also be discarded before spawning. Anchor every unresolved bare command before returning, without probing it as a script, and execute the resolved path on successful lookup. Explicit script paths retain their shebang behavior. Existing argument-parsing assertions now account for resolved program names on Windows. Git reference: `compat/mingw.c::path_lookup()` and `mingw_spawnvpe()` at `d38352cd43ab` fail lookup before parsing an interpreter and execute the resolved path. Assisted-by: GPT 6.0 Astra Co-authored-by: GPT 6.0 Astra <codex@openai.com>

Bring file-opening and directory-traversal utilities into `gix-fs` so filesystem consumers can use a common API without requesting filesystem features from `gix-features`. Expose `open_options_no_follow()` and `open_read_only_no_follow()` for Unix and Windows. The read helper returns an opened non-symlink handle or an explicit `FileOrSymlink::Symlink` outcome, including for dangling links. Inspect opened handles before returning them and classify failed opens without retrying; preserve missing paths and other I/O errors. Protection applies to the final path component, while parent-directory symlinks are still followed. Own the Unicode-precomposition adapter shared by `read_dir()` and recursive walking. `read_dir::DirEntry` now uses the `gix-fs` type instead of re-exporting the `gix-features` type, changing its type identity while retaining its methods and normalization behavior. Add the optional `walkdir` feature with depth limits, hidden entries, configurable link following, and Git's ordering of files and directories. Traversal remains serial regardless of the requested `Parallelism`. Basic file access and `read_dir()` remain available without feature flags. Assisted-by: GPT 6.0 Astra Co-authored-by: GPT 6.0 Astra <codex@openai.com>

`gix-fs` already provides the filesystem helpers and recursive walkers. Keeping their duplicate implementation in `gix-features` adds unnecessary filesystem dependencies to the foundational feature crate. Remove the `fs` module and the `walkdir` and `fs-read-dir` features. Migrate workspace callers to `gix_fs`, move traversal requests to `gix-fs/walkdir`, and update the manifests, lockfile, traversal documentation, and feature check script together. Assisted-by: GPT 6.0 Astra Co-authored-by: GPT 6.0 Astra <codex@openai.com>

Careful refackiew, only the tests were mostly waved through as they are known to be super thorough. Only their style needs checking, which was fine here.  Fixtures that require symbolic links need to distinguish unavailable symlink creation on the host from broken fixture setup. Otherwise, tests either fail on unsupported filesystems or risk hiding unrelated errors when skipping. Inject `gix_testtools_require_symlinks` into Bash fixtures to probe creation of a real link before generating contents. Record the outcome at the fixture root and stop successfully when links are unavailable. Tests can then use `fixture_has_symlinks()` to skip explicitly; missing or malformed results, unexpected `ln` exits, copied files, and later setup failures remain errors. Preserve script arguments and Bash 3.2 source context without exporting the helper to child shells. Include the prelude in cache keys for fixtures using the probe, and document archive and writable-fixture constraints. Assisted-by: GPT 6.0 Astra Co-authored-by: GPT 6.0 Astra <codex@openai.com>

Replace manual `windows()` scans with `ByteSlice::contains_str()` and `ByteSlice::find()`. Use `contains_str()` where only presence matters instead of checking `find().is_some()`. Assisted-by: GPT 6.1 Sol Co-authored-by: GPT 6.1 Sol <codex@openai.com>

…rites  A tracked `.gitignore` symlink was absent from the skip-worktree ID mapping, so ordinary ignore lookup could parse arbitrary external files. An endless target could also exhaust memory. Always request no-follow reads for worktree `.gitignore` files, regardless of their index entry. Use `gix_fs::open_read_only_no_follow()` when pattern loading disallows symlinks, treating skipped links as absent files so skip-worktree blob fallback remains available. Discard partially read pattern bytes on failure. Use the same helper for the worktree `.mailmap`, reporting symlinks as errors without reading their targets and identifying worktree read errors correctly. Explicitly configured pattern files and `mailmap.file` may still follow symlinks. On Windows, inspect nonexclusive checkout handles for symlinks before truncating with `set_len(0)` or writing. Retain forced unlink-and-replace behavior and route symlinks checked out as ordinary files through the same validated open path. No-follow protection applies to the final path component on Unix and Windows; leading components can still be symlinks. Compare ignore lookup and skip-worktree fallback against isolated Git `check-ignore` calls. Cover mailmap policies and checkout collisions with file, directory, and dangling symlink targets, including links created after the Windows path check and validation before truncation. Assisted-by: GPT 6.0 Astra Co-authored-by: GPT 6.0 Astra <codex@openai.com>

`ein tool organize --execute` used a repository's origin URL to construct a move destination without resolving parent components or checking containment. It was demonstrated percent-decoded path traversal moving a repository outside the user-selected root. Resolve the completed destination with the existing `gix::path::realpath()` helper, which handles existing symlinks and missing leaf components, then reject paths outside the canonical destination before creating directories or moving anything. The subsequent self-nesting check now also receives a resolved path. Disposable isolated repositories reproduce HTTP percent-encoded traversal, a parent-directory hostname, SSH traversal, and an existing symlink escaping the destination. A control retains the ordinary host/owner/repository layout and `.git` suffix handling. Both escape regressions fail before the fix. Git reference: `urlmatch.c::url_normalize()` at `d38352cd43ab` also normalizes dot segments; organizing repositories is gitoxide-specific and additionally needs filesystem confinement. Validation: all three focused organizer tests pass with the `organize` feature. Assisted-by: GPT 6.0 Astra Co-authored-by: GPT 6.0 Astra <codex@openai.com>

A report describes delta base cycles that never terminate in header lookups and keep allocating during object decoding. Track a constant-space cycle checkpoint in both loops, without relying on the untrusted pack object count or rejecting valid forward references. Account for heap-backed delta-chain metadata in the existing per-allocation limit and reserve it fallibly, including when delta payloads are empty. The regressions failed before the fix and cover self, multi-entry, mixed REF/OFS cycles, empty deltas, and valid forward chains. Git reference: `t/t5309-pack-delta-cycles.sh` at `d38352cd43ab9745686d697872408bc3249a153f` rejects cycles while allowing forward delta bases. Assisted-by: GPT 6.0 Astra Co-authored-by: GPT 6.0 Astra <codex@openai.com>

…ction origins Creating a section with its own source, path, include level, or trust previously required changing the file's metadata first. Add `File::new_section_with_meta()` to attach owned or shared metadata directly to one new section, leaving the file's origin and the defaults for subsequent sections unchanged. Assisted-by: GPT 6.1 Sol Co-authored-by: GPT 6.1 Sol <codex@openai.com>

A report describes reduced-trust repository configuration overriding submodule settings, including command-valued updates. Apply the repository's existing configurable section filter for worktree, index, and HEAD module sources, and retain each copied section's origin metadata. Keep trusted API overrides usable and restrict module enumeration to the original module definitions. The regressions first reproduced both the trust bypass and lost provenance, then covered custom filters and every source. Git reference: `setup.c::ensure_valid_ownership`, `submodule-config.c`, and `builtin/submodule--helper.c` at `d38352cd43ab9745686d697872408bc3249a153f` distinguish shared module definitions from trusted local overrides. Assisted-by: GPT 6.0 Astra Co-authored-by: GPT 6.0 Astra <codex@openai.com>

…URLs  Multiple reports describe an issue about same URL-component confusion: reserved characters in a credential username can change the host selected when the credential context is serialized and parsed again. Percent-encode each URL component with its own delimiters, and preserve supplied host and credential fields when validating a synthesized URL. Keep explicit URL precedence, ports, IPv6 and scheme-specific host escaping, and retain the existing HTTP-path normalization behavior. The helper-level regression first reproduced a request for a different host; component tests cover reserved characters, percent escapes, Unicode and path handling. Git reference: `credential.c:223`, `strbuf.c:502` and `t/t0300-credentials.sh:662-715` at `d38352cd43ab9745686d697872408bc3249a153f` encode credentials as URL components. Assisted-by: GPT 6.0 Astra Co-authored-by: GPT 6.0 Astra <codex@openai.com>

…sets  A Report describes indexed delta components that cannot be reached from any base object yet pass delta-tree verification. Require every indexed item to have been inspected before returning success, using the existing object counter. Legitimate forward references remain supported. Return corruption errors for duplicate or overlapping index offsets and for an entry at or past the pack end, instead of panicking. The reported decoder cycle and allocation problem is handled by the fix. Three regressions failed before the fix: an unresolved cycle was accepted, while duplicate offsets and an invalid pack end panicked. Coverage includes self and multi-node components with and without a valid root, plus a valid forward delta base. Git reference: `builtin/index-pack.c::conclude_pack()` at `d38352cd43ab9745686d697872408bc3249a153f` rejects unresolved deltas. Also: progress initialization does not reset every progress implementation. Explicitly reset the object count before traversal so a reused logging progress instance cannot make a valid pack fail the completeness check. The new real-Log regression failed before this correction. Assisted-by: GPT 6.0 Astra Co-authored-by: GPT 6.0 Astra <codex@openai.com>

Looked at everything in detail and added error handling in callbacks that didn't have them. Reduced the scope of this commit to not overlap with `odb-parallelism` branch which makes many changes additionally. Many of the newly added tests are very specific about error handling, one might call them overkill, but I left them in for good measure. Further, errors.rs in `gix-odb` I just skimmed the test titles off as everything else would be too time consuming - better have them than not have them I thought, particularly to support `odb-paralellism`.  A report describes object lookup panics caused by unchecked pack IDs and large-offset ordinals in otherwise structurally valid MIDX files. Validate these references when reading an object entry, before indexing pack arrays or dereferencing the optional LOFF (Large Offsets) table. Bound LOFF ordinals by the chunk itself, not by the remaining file. Keep MIDX loading limited to structural validation: an eager scan of every offset entry adds linear startup work and faults in memory-mapped pages even when a command needs only a handful of objects. Git likewise checks these references on access. Make `pack_id_and_pack_offset_at_index()` return `gix_error::Result` and `iter()` yield fallible entries. Propagate corruption errors through object and header lookup, delta-base resolution, integrity verification, and CLI entry listing. ID-only iteration remains available without valid offset references. Preserve literal high-bit 32-bit offsets when LOFF is absent. Return `Result<Option<_>>` from `Find::location_by_oid()` and `Find::pack_offsets_and_oid()`: absence and lookup failure are different outcomes, and callers must be able to distinguish them. Propagate errors through forwarding implementations, object counting, pack-entry iterator construction, and thin-pack base lookup rather than treating corruption as a missing object or a reason to recompress. Report a pack that becomes unavailable during construction as an error instead of panicking. Retain failed index loads separately from missing files so repeated lookups cannot silently turn corruption into absence. Load pending indices before retrying failures, retry all failed slots, and reconcile disk state when refreshing. This keeps independent valid packs usable and permits recovery after repair or removal without endless refresh loops on persistent failures. Regressions cover boundary and extreme invalid references, lazy loading, valid LOFF ordinals and literal offsets, integrity verification, corruption propagation through object lookup and pack generation, and repeated, shared, concurrent, repaired, and removed index-load failures. Git reference: `d38352cd43ab9745686d697872408bc3249a153f`, `midx.c::midx_for_pack()` and `nth_midxed_offset()`. BREAKING CHANGE: Multi-pack-index offset access and entry iteration now return errors which callers must handle. `Find::location_by_oid()` and `Find::pack_offsets_and_oid()` return `Result<Option<_>>`. `output::entry::iter_from_counts()` returns a fallible construction result, and `output::Entry::from_pack_entry()` requires a base-lookup callback returning `Result<Option<ObjectId>>`. Assisted-by: GPT 6.0 Astra Co-authored-by: GPT 6.0 Astra <codex@openai.com>

Quoting every metadata key adds noise to diagnostic output even when the name is already unambiguous. Keep nonempty keys containing only ASCII letters, digits, `_`, `-`, or `.` unquoted in `Message` display and metadata debug output. Empty and unusual keys remain quoted and escaped, and value formatting is unchanged. Replace the public `BTreeMap` alias with a private-storage `Metadata` wrapper so dictionary formatting and its public API no longer expose the implementation. Preserve lexicographic ordering and provide construction, lookup, mutation, iteration, indexing, collection, and extension APIs. **Breaking change:** `Metadata` is no longer interchangeable with `BTreeMap`. Callers must use the wrapper API; iterator return types hide the underlying collection. Add regression coverage for key quoting, compact values, and dictionary access, and update dependent assertions and output snapshots.

Report 21717 describes ownership checks that trust a linked repository based on only one path. Compute the minimum ownership trust of the original candidate, its resolved git directory, common directory and existing checkout before selecting discovery options or trusting configuration. Apply the same checks to direct and environment-based opens. Preserve explicit trust overrides and access through a private git directory whose checkout has been removed. A private ownership callback makes foreign ownership regressions deterministic without requiring privileged chown. Git reference: `setup.c::ensure_valid_ownership` at `d38352cd43ab9745686d697872408bc3249a153f` checks gitfile, worktree and gitdir. The common directory is also checked here as a configuration source. Validation: the discovery regression failed before implementation. `gix-discover`: 3 unit, 59 integration, 8 isolated and 1 doc tests passed; the existing exFAT mount test is skipped because mounting is unavailable. The private opening regression and all 38 `gix` repository-opening tests pass. CI correction: the i386 container's checkout directory belongs to the host runner, while its Git metadata is created by the container. The old `from_dir_with_dot_dot` test discovered that checkout and incorrectly assumed full trust. Use disposable nested repositories and the existing current-dir option instead, preserving the path-normalization assertions for relative and absolute inputs without relying on checkout ownership. Validation after the CI correction: the focused test, the complete `gix-discover` suite except the unavailable exFAT mount test, and all-target package Clippy passed. The i386 run passed its other 4,443 tests.

Git paths can contain arbitrary bytes on Unix, while native Windows paths can contain unpaired surrogates that cannot be represented as UTF-8. The canonical conversion functions previously hid these failures behind panicking wrappers, even when their callers already returned errors. Make `into_bstr`, `from_bstr`, `from_bstring`, `from_byte_slice`, and `to_native_path_on_windows` fallible and remove their redundant `try_*` counterparts. Keep borrowed inputs borrowed and owned inputs owned, and retain the concrete encoding error as a source. Also return an error from `normalize_saturating` when normalization exhausts a relative or empty CWD. Adapt the workspace in the same breaking change, including public APIs for reference and reflog paths, worktree IDs, configuration, pathspecs, pattern sources, and command preparation. Convert `Prepare` to `Command` with `TryFrom`; inspect ASCII shell syntax through native encoded bytes and preserve native commands without a Git-path round trip. Keep reference names as bytes in missing-reference diagnostics. Propagate conversion failures before affected command execution, filter exchanges, and watcher updates. Debug logging records failed diagnostic conversions without changing pathspec behavior. Add regression coverage for arbitrary Unix bytes, Windows encoding failures, ownership, lazy configuration selection, and dependent operations. Validation: workspace all-target check, fast QA, workspace tests, doctests, and focused regressions pass. Tests requiring local sockets, signing agents, filesystem watching, or an exFAT image pass outside the sandbox with user Git configuration disabled. Selected Windows test targets compile; Windows execution was not available, and Criterion's `alloca` dependency prevents cross-compiling the glob/attributes/diff tests without Windows headers.

`gix-features` once coordinated several alternative implementations. Serial versus threaded execution is its remaining shared implementation policy; tracing, hashing, compression, and filesystem operations already have their own owners. Move progress, interruption, byte pipes, decoding, iterator chunks, and cache diagnostics into `gix-utils`. Keep utilities that add dependencies behind feature toggles, including `interrupt` for typed cancellation via `gix-error`. Use `crc32fast` directly in `gix-pack` and `gix-odb`. Rename the remaining crate to `gix-parallel` and expose its computation helpers, reducers, and shared ownership primitives at the crate root. Replace `gix::features` and `gix::threading` with the existing owner exports, including `gix::parallel` and `gix::utils`, and adapt all workspace callers. Expose both progress formatting choices individually on `gix`, retaining `comfort` as their bundle. Forward tracing directly to `gix-trace` and cache diagnostics to `gix-utils`. Update documentation, feature checks, the pack fuzz harness, and package-size checks for the new ownership. Validation: fast QA; 289 focused tests covering serial and threaded helpers, utilities, packs, and the object database; isolated feature checks; workspace and serial API documentation; pack fuzz harness check; Windows, wasm32-unknown-unknown, and WASI compilation checks.

Sebastian Thiel (Byron) force-pushed the sec-audit branch 2 times, most recently from 9ed6017 to 8847b32 Compare September 29, 2026 20:21

Sebastian Thiel (Byron) changed the title ~~Fix reported trust, credential, pack, and Tix undo issues~~ various improvements Sep 30, 2026

Sebastian Thiel (Byron) force-pushed the sec-audit branch 11 times, most recently from 0754b3c to db70db9 Compare October 5, 2026 09:55

Codex (codex) and others added 14 commits October 5, 2026 13:49

Sebastian Thiel (Byron) force-pushed the sec-audit branch 2 times, most recently from 2503a4a to 9c5e98b Compare October 5, 2026 12:17

Sebastian Thiel (Byron) force-pushed the sec-audit branch from 9c5e98b to 37d44ab Compare October 5, 2026 12:37

Sebastian Thiel (Byron) force-pushed the sec-audit branch from 37d44ab to c6fca3f Compare October 5, 2026 12:59

Sebastian Thiel (Byron) and others added 6 commits October 5, 2026 15:25

Sebastian Thiel (Byron) force-pushed the sec-audit branch from c6fca3f to 95d9e00 Compare October 6, 2026 04:01

Navigation Menu

Repository navigation

various improvements - #3032

various improvements#3032
Sebastian Thiel (Byron) wants to merge 21 commits into
mainfrom
sec-audit

Byron commented Sep 29, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Byron commented Sep 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Tasks

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Byron commented Sep 29, 2026 •

edited

Loading