Skip to content

Feat/subagent history - #110

Open
jpia wants to merge 5 commits into
mlhher:mainfrom
jpia:feat/subagent-history
Open

Feat/subagent history#110
jpia wants to merge 5 commits into
mlhher:mainfrom
jpia:feat/subagent-history

Conversation

@jpia

@jpia jpia commented Aug 16, 2026

Copy link
Copy Markdown
Contributor

Summary

Closes #109

Introduces opt-in persistence for subagent conversation histories, restructures sessions into a hierarchical on-disk layout, and fixes a thread-unsafe subagent-ID generation bug that would silently cause file collisions under concurrent spawns.

Changes

1. Hierarchical Storage Layout (internal/session/paths.go)

When persistence is enabled, subagent histories are written under the parent session's folder (<sessions>/<sessionID>/subagents/<childID>.json). Parent .json / .meta.json files stay flat and unchanged.

  • SubagentHistoryDir, SubagentHistoryPath, and RemoveSessionFolder manage the subagent tree
  • isValidPathElement rejects ., .., path separators, and NUL bytes to prevent directory traversal
  • late session delete also cleans up the subagent folder alongside the parent session

2. Thread-Safe ID Generation (internal/orchestrator/base.go)

  • Added a mutex-protected NextChildID(typePrefix string) counter on BaseOrchestrator
  • NewSubagentOrchestrator now mints the child ID before creating the session, so the path is known at construction time
  • The old lockless len(parent.Children()) fallback is removed entirely; a subagent must have a *BaseOrchestrator parent

3. Opt-In Persistence (cmd/late/main.go, internal/config/config.go)

  • CLI flag: --save-subagent-histories (default false)
  • Config file field: save_subagent_histories (JSON)
  • Resolution order: explicit CLI flag > config file > default (false)
  • There is deliberately no environment-variable override — the user must consciously opt in per run or per config

effectiveSessionID is derived from the actual history path so resumed sessions reuse their original folder.

4. Defensive Hardening

  • Unsafe session IDs: if effectiveSessionID is empty or unsafe, subagent histories fall back to in-memory instead of writing outside the session tree

Testing

7 test files updated (~1,128 lines) covering:

  • subagent orchestrator construction with/without persistence and parent-type enforcement
  • path validation (.., separators, NUL), resolution, and folder cleanup
  • NextChildID counter increment and concurrency safety
  • config precedence rules and CLI flag wiring
  • deriveEffectiveSessionID edge cases and session-delete cleanup path

Backwards Compatibility

  • Default behavior is unchanged: without the flag or config field, subagents stay in-memory and legacy flat sessions continue to work
  • No automatic migration of existing flat files is performed (out of scope)

Contributor License Agreement (CLA)

To accept your code, we legally need you to agree to our CLA so we can maintain the project's Business Source License (BSL) and future open-source transitions.

  • By checking this box, I confirm that I have read and agree to the terms of the CLA.md in this repository. (To check the box, put an x between the brackets like this: [x])

@mlhher

mlhher commented Aug 16, 2026

Copy link
Copy Markdown
Owner

Thank you for your contribution.

Note that I am currently away on a trip as thus this might take some time to fully evaluate. Will go over it and test it when I get the time to, if things fit will be merged. Thank you!

Comment on lines +148 to +154
want := []string{
"researcher-subagent-0",
"researcher-subagent-1",
"researcher-subagent-2",
// The counter is global per parent, NOT per type.
"coder-subagent-3",
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if I should change the naming scheme to:

subagent-[index]-[type]

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is fine as it is as subagent counter should be global.

Comment thread docs/quickstart.md
Comment on lines +258 to +259
By default, subagent conversations are kept in memory and discarded. With `--save-subagent-histories` (or `"save_subagent_histories": true` in the config file), each subagent's history is saved under the session folder — `~/.local/share/late/sessions/<session-id>/subagents/<type>-subagent-<N>.json` — while the parent session files stay in place. `late session delete <id>` removes the subagent folder along with the session. Note that saved transcripts may contain sensitive content (file contents, tool outputs); they are written with user-only permissions.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if this is TMI for the quickstart. I could remove it if it's unnecessary.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a short explanation of the flag is enough inside the table.

@mlhher

mlhher commented Aug 26, 2026

Copy link
Copy Markdown
Owner

@jpia Looks good. Two things that stood out (especially regarding the talk we had in the issue):

  1. Subagent IDs are only collision-free within one process. The counter resets after restart, so resuming a session can reuse an existing subagent history path and overwrite the previous transcript.

  2. The persistence mode is not retained per session. Resuming a previously opted-in session without the flag/config stops persisting new subagent histories, instead of detecting and continuing the session’s existing folder layout.

For the first UUIDs would solve this or you could save the most recent sequence number inside .meta.json. If you use UUIDs you might need to take care of the TUI truncating them (in the statusbar).
The second would presumably go into .meta.json or could be inferred from the directory structure.

I don’t have a preference about the implementation, and neither point is necessarily a blocker for me, I wanted to note these though as per the discussion in the issue we have had.

@jpia

jpia commented Aug 27, 2026

Copy link
Copy Markdown
Contributor Author
  1. Subagent IDs are only collision-free within one process. The counter resets after restart, so resuming a session can reuse an existing subagent history path and overwrite the previous transcript.
  2. The persistence mode is not retained per session. Resuming a previously opted-in session without the flag/config stops persisting new subagent histories, instead of detecting and continuing the session’s existing folder layout.

I'm leaning towards the .meta.json path for the solution here. I'll work on it!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Redesigning session history to set the stage for parallel subagents

2 participants