⚡ Unlimited Context LLM

Give your AI superpowers with unlimited context for Ollama — a billion+ token memory for any LLM. Local-first, on your own machine, free.

An open project from Aether · Apache-2.0 · Install

The shape of it · Proof · Safety · Use policy · Problem · How it works · MPO chain · Memory size · The math · RAM · Commands · Terminal · Install · Cite

---

Your context window didn't get bigger. Its reach did. Unlimited Context is virtual memory for an LLM. The model keeps its small window; the engine keeps a vast store on your disk and pulls the right slice back in while the model reasons. A small local model stays coherent across runs that would blow past any context window.

⚠️ Giving a model durable memory is powerful — read the safety measures first. Real concerns (runaway agents, grounding drift, an agent's own notes becoming its rules) and what we do about them: Aether AI — Ethical & Safety Measures. The project is open under Apache-2.0 and governed by the Acceptable Use Policy — these are not recommendations. Anyone using the project must comply with its terms, and by using the project you already agree to and are bound by them.

---

The shape of it

Four steps, start to reach:

1. Pick a model — any local model (Ollama, llama.cpp, Hugging Face) or your own API-backed one.
2. Start a session — one object wraps the model and its memory.
3. Allocate disk — choose a pool size; that disk is the memory.
4. Reach 1,000×+ further — overflow is encoded to the pool and the exact slice is recovered the moment the model needs it.

That's it. A 5 GB pool gives a small model ~1.16B tokens of reach — about 9,000× a 128K window — on your own machine, offline.

---

The problem

Long agentic runs all die the same way. The model fills its window, starts compressing its own history, silently drops the one detail that mattered three steps ago — and drifts. You've seen it: the runaway PR, the agent that rewrites a function it already wrote, the build that falls apart at hour two. Bigger windows just delay it, and a crammed 1M-token window rots in the middle anyway.

The fix isn't a bigger window — it's to stop throwing the overflow away. Instead of summarizing what spills over, Unlimited Context encodes it to a local pool on your disk and recovers the right slice exactly when it's needed. Nothing load-bearing is silently lost.

Compress & forget ✗  →  Encode & recover ✓

How it works

It's virtual memory, for attention. Map it to an OS and it clicks:

OS	Unlimited Context
RAM	the resident window the model sees now (small, fast)
Disk	the context pool — your encoded memory (~5 GB ≈ ~1B tokens)
Pager	the slice loader — prefetches the next slice from what the model is reasoning about right now
Page-replacement	the retention policy — useful slices stay, stale ones fade, anything relevant again comes back

All of it runs while the model generates, so reaching the pool adds no wall-clock. → full explainer in docs/how-it-works.md.

What you get

• 🧠 Unbounded reach — ~1B tokens of encoded context in ~5 GB on disk; the model reaches it in slices.
• 🧩 MPO context chain — recall pulls the whole connected thread, not isolated nearest-neighbors (5× connected-context recall in-bench).
• ⚡ Zero added latency — the pager runs concurrently with generation, hidden behind the model's own thinking.
• 🪟 Curated beats crammed — a small, relevant resident window outperforms a stuffed one (no lost-in-the-middle) — and costs less.
• 🔒 Local-first — your context never leaves your machine. Free storage, full privacy, works offline.
• 🤖 Any model — Llama, Qwen, Mistral, Phi — via Ollama, llama.cpp, or HF, or your own API-backed model.
• 📉 Coherence you can measure — ship the head-to-head: same model, engine on vs off, watch the drift rate fall off a cliff.

Pick your memory size

First run drops you into a slider — pick how much your model gets to remember:

$ aether-context init
──────────────────────────────────────────────────────────────────
  ⚡ choose your context pool          encoded reach · not a window
──────────────────────────────────────────────────────────────────
  ▸  5 GB   ████░░░░░░░░░░░░   ~1.16B tokens   a big project   (floor)
     10 GB  ████████░░░░░░░░   ~2.33B tokens   a large monorepo + docs
     15 GB  ████████████░░░░   ~3.49B tokens   multiple repos / long runs
     20 GB  ████████████████   ~4.65B tokens   massive corpus / power user
──────────────────────────────────────────────────────────────────
  reach ≈ pool_GB × 233M tokens     custom: --pool 12  (any size ≥ 5 GB)
  ↑/↓ slide      ↵ confirm

  pool [5]: 10
  ✓ 10 GB  →  your model can now reach ~2.33 billion tokens

What that buys you in coding time

The real win isn't the token count — it's that the wall disappears. A typical ~128K context window fills after well under an hour of active agent work, then starts compacting and forgetting. A 5 GB pool is ~9,000× bigger.

Rough ballpark — assuming a busy coding agent encodes ~300K–1M keep-worthy tokens/hour (chatty swarms burn more, careful single agents less):

Pool	Reach	Active autonomous coding before it even fills*
5 GB	~1.16B	~1,200–3,900 hrs — weeks of nonstop building
10 GB	~2.33B	~2,300–7,800 hrs
15 GB	~3.49B	~3,500–11,600 hrs
20 GB	~4.65B	~4,700–15,500 hrs

For color: 5 GB of reach ≈ ~100M lines of code, or a shelf of ~8,000 books — you won't fill it in one sitting.

_{* Rough order of magnitude. Because the retention policy fades stale slices, the pool never hard-stops anyway — it just keeps what's relevant. Run a build as long as you want; it won't lose the plot. The per-session RAM math is in RAM footprint below.}

Running many sessions

Running more than one agent? How the pool is shared is the single biggest RAM lever:

• --pool-mode shared — one pool, one index, all sessions reach the same memory. The index is paid once; each extra session adds only ~30 MB, so RAM barely moves as you add sessions. Best for related work (same project) or max concurrency on a small machine. Trade-off: sessions can see each other's context (no isolation).
• --pool-mode separate (default) — each session gets its own pool + index, fully isolated and private. Clean, but you pay one index per session, so RAM scales with N × pool. Best for unrelated tasks or when isolation matters.

How many actually fit:

Pool	8 GB · shared	8 GB · separate	16 GB · shared	16 GB · separate
5 GB	dozens¹	~13	dozens¹	~33
10 GB	dozens¹	~7	dozens¹	~18
15 GB	dozens¹	~4	dozens¹	~12
20 GB	dozens¹	~3	dozens¹	~9

_{Reserves: ~2.5 GB held back on an 8 GB machine, ~6 GB on 16 GB — the rest stays for your OS and editor. ¹ With a shared pool, RAM stops being the limit (50–70+ sessions fit); you're bounded by CPU and good sense, not memory.}

The math, per tier

Derived, not vibes:

Pool	Slices	Encoded reach	Slider
5 GB (floor)	2.27M	~1.16B tokens	████░░░░░░░░░░░░
10 GB	4.55M	~2.33B tokens	████████░░░░░░░░
15 GB	6.82M	~3.49B tokens	████████████░░░░
20 GB	9.09M	~4.65B tokens	████████████████

How those numbers come out: ~2.2 KB per slice (a 256-dim vector + compressed text + metadata) ÷ 512 tokens per slice → ~455K slices/GB → ~233M tokens of reach per GB. So reach ≈ pool_GB × 233M. 5 GB is the floor; bump anytime with aether-context --pool 20.

Honest: that's encoded reach, retrieved in slices — not a bigger attention window, and it rides on retrieval hit rate. A bigger pool buys more reachable codebase/corpus per session — not more concurrent sessions (those are RAM-bound, ~30 on 8 GB either way).

---

The proof

Not a synthetic micro-benchmark — a real, paid, end-to-end run. A reasoning model (deepseek-v4-pro, via OpenRouter) driven through a 40-turn agent session that overflows its window (2,000-token window, 60 real microsoft/vscode issues), measured engine off vs on — one live run, $0.19, 2026-06-14.

• The model stops forgetting. Recall of early facts after they fall out of the window: 0.15 → 1.00. The baseline drifts and forgets; the engine holds every early fact — zero drift.
• Failure turns into success on the real work. Tasks completed correctly: 3 / 20 → 20 / 20. The job is only done right with the engine.
• Cheaper, not just better. −24% total cost, −54% in the back half — the engine sends a compact recalled slice instead of dragging the whole transcript into every call.

Metric	Off (baseline)	On (engine)	Change
Recall coherence (early facts still correct)	0.15	1.00	6.7×
Work outcome (tasks done right)	3 / 20	20 / 20	3 → 20
Cost — full session	$0.0711	$0.0542	−24%
Cost — back half (recall phase)	$0.00117/turn	$0.00053/turn	−54%

Committed data: full write-up · raw artifacts (api_eval_results.json, api_eval_series.csv, api_eval_plot.png, RESULTS.md) · reproduce with python -m bench.api_eval --model deepseek/deepseek-v4-pro --repo microsoft/vscode --arms off,on,on_chain --plot

_{Scope, honestly: this measures the engine (retrieve-on-overflow memory), not the MPO chain — on this single-fact recall task the chain ties plain recall (both 1.00); its multi-slice edge is synthetic-only so far (bench/chain_recall.py: connected-context recall 0.15 → 0.78), with the live thread run pending, not yet claimed. The 2,000-token window is deliberately tiny to force overflow, so a realistic window shows a smaller (still real) gain. N = 20 recall turns, single run.}

---

RAM footprint

The engine stays light: vectors live on disk (mmap'd) — only the small HNSW index graph and a hot working set are ever resident. So RAM is a predictable formula, not a mystery:

RAM  ≈  ~180 MB   base (engine + shared static encoder)
      +  ~29 MB   per GB of pool   (resident index)
      +  ~30 MB   per active session

MPO: the context chain

Plain semantic search returns isolated nearest-neighbors — the single closest slices, ripped out of the thread they belonged to. Recall a fact and you often miss the three slices around it that made it make sense.

The MPO context chain fixes that. It links the session's slices into one connected structure, so when cosine pulls an entry slice, the chain pulls in the slices most coupled to it — widening the working set with the connected thread, not stray hits. Cosine is still the retrieval mechanism; the chain assists it.

The chain is Aether-tuned, deterministic, and fully local — no training, no network. It's purely additive: it only ever adds connected context, never blocks or replaces a hit, and on any hiccup it falls back cleanly to plain cosine.

In a planted-thread benchmark, this lifts connected-context recall from 0.15 (cosine alone) to 0.78 — over 5× more of the right thread in the window. On by default:

Session(model="ollama/qwen2.5", pool_gb=10)                 # chain on by default
Session(model="ollama/qwen2.5", pool_gb=10, mpo_chain=False) # plain cosine

aether-context run "..." --no-mpo-chain                      # disable for one run

Resident index cost by pool size:

Pool	Index RAM (resident)
5 GB	~146 MB
10 GB	~291 MB
15 GB	~436 MB
20 GB	~582 MB

(The encoder is always shared — stateless, ~31 MB, loaded once. Only the pool/index differs.)

TL;DR. Shared pool → RAM is not your limit — spin up as many sessions as your CPU allows. Separate pools → one index each, so plan on ~3 (20 GB) to ~13 (5 GB) sessions on 8 GB, roughly double at 16 GB. A bigger pool always buys reach, never more sessions. Need more headroom? Shrink the pool. (--index tiered is reserved for a future paged-graph index and currently runs the flat index — it does not yet reduce resident RAM.)

Common commands

Command	What it's for
aether-context init	Pick your pool size — the on-disk storage slider — on first run.
aether-context run "<task>" --no-mpo-chain	Run with the MPO context chain disabled (plain cosine).
aether-context run "<task>"	One-shot a task with full reach, then print the result.
aether-context chat	Open an interactive session; type /status anytime, /clear to reset.
aether-context status	See pool size, slices used, reach, and hit rate at a glance.
aether-context doctor	Check Ollama, your model, disk, and RAM before a long run.
aether-context --pool 20	Resize the pool anytime (non-destructive re-index).

Tip: run aether-context doctor first — it catches the three things that ever go wrong (Ollama down, model not pulled, not enough disk) and prints the exact fix.

The aether coding terminal

The same install ships a second command — aether — an open-source agentic coding terminal running on the Unlimited Context engine. It's local-first: turns run on your local Ollama by default (no account, no network); sign in and they switch to the Aether cloud API. It's the Python-native twin of aether-code (the TypeScript terminal) — same commands, same backend, same tools.

Command	What it does
aether	Open the interactive REPL (local Ollama by default).
aether "<prompt>"	One-shot turn, streamed.
aether code "<task>"	Autonomous coding run on the Unlimited Context brain (test-gated, git-checkpointed).
aether auth login	Sign in (--token <t> or --username/--password) for the hosted API (then set backend auto|cloud).
aether auth status | logout | token	Show / clear / print the stored credential.
aether models	List models available to your tier.
aether config [show|get <k>|set <k> <v>]	Local settings, incl. backend = local (default)|auto|cloud.

Slash commands (inside the REPL): /help · /models · /model <tag> · /agents · /agent <id> · /tier · /audit [n] · /web <query> · /clear · /exit.

Web tools — the agent can reach the web on any backend: web_search (DuckDuckGo, no key) and web_fetch (URL → readable text, SSRF-guarded).

Backend — local (the default) runs every turn on your own Ollama; nothing leaves your machine and no account is needed. Opt into the hosted Aether API with aether config set backend auto|cloud (or AETHER_BACKEND=auto) plus aether auth login — auto uses the cloud only when you're signed in and falls back to local otherwise. The hosted API is the maintainer's own instance, so a fresh clone never calls it.

Smoke test — with Ollama up, python -m aether_agent.smoke (or aether-smoke) runs the SSRF guard, a real local turn, a web search + fetch, and the cloud path (when signed in), printing PASS/SKIP/FAIL (exit non-zero only on a real failure — missing Ollama/network/sign-in are skips).

Quickstart

# Install straight from GitHub — works today, always the latest:
pip install git+https://github.com/DBarr3/Unlimited-Context-LLM.git

# From PyPI, once published — the package name is "aether-context"
# (note: `pip install unlimited-context` is NOT the package name):
pip install aether-context

from aether_context import Session

s = Session(model="ollama/qwen2.5", pool_gb=5)
s.run("Build me a full-stack weightlifting tracker app.")
# runs long. stays coherent. walk away.

That's the whole thing. One small model, one command, a billion tokens of reach behind it.

Honest about the word "unlimited"

"Unlimited" means reach, not attention. Your model keeps its native window — we make it reach a billion-token pool in slices, via fast retrieval. The whole thing rides on retrieval hit rate; when it's high (and the loader is built to keep it high), the pool feels like one seamless context. The measured proof of all this is up top — see The proof.

Citation

If Unlimited Context helps your work, please cite it. Built and maintained by Aether AI.

@software{unlimited_context_2026,
  title        = {Unlimited Context (aether-context): virtual memory for LLM attention},
  author       = {Barrante, Brandon},
  organization = {Aether AI},
  year         = {2026},
  url          = {https://github.com/DBarr3/Unlimited-Context-LLM},
  license      = {Apache-2.0}
}

GitHub's "Cite this repository" button reads CITATION.cff directly.

⭐ Star, share, contribute

If this gave your local model superpowers, drop a star — it's how other people find it. PRs and issues are welcome — see CONTRIBUTING.md. Runnable examples live in examples/ — start with quickstart.py, then coding_agent.py.

License

Apache-2.0. Use it, fork it, ship it in your product.

---

Built by Aether AI · Aether

Unbounded reach for the model you already run.