Content-addressed Merkle DAG · Hash-chained · Time-traveling · Bi-temporal · Causally-provable embedded database.
Replay-protected · idempotent · relational · filterable · sortable · searchable · concurrent. One Rust core → ships to PyPI and npm from a single source.
Current stable: 2.8.0 — NEDB ships as three version-aligned distributions on one tag — nedb-engine (flagship), crypto-database (verifiable v2/v3 DAG), and aof-db (fast append-only) — across npm / PyPI / crates.io with full mac + linux + windows native addons (see Releasing below). All native wheels (Linux + Windows on GitHub Actions; macOS arm64 + x86_64 on Codemagic M2 Mac Minis) plus the universal pure-Python wheel ship from a single v* tag, with the nedbd-v2 binary bundled inside pip install nedb-engine.
POST /v1/databases/<name>/cast turns a short English prompt into NQL, using a 3.33M-parameter model that runs locally on CPU. No API key, no network call, no per-token bill.
nedbd --dag --cast ./data # requires: cargo install nedb-engine --features cast
curl -X POST localhost:7070/v1/databases/shop/cast \
-d '{"prompt":"orders over 100"}'
# → {"nql":"FROM orders WHERE total > 100","valid":true,"collection_known":true,"executed":false}The model (nedb-cast-slm) was trained on NQL using NEDB's own parser as generator, grader, and gate — then shipped to PyPI, crates.io, and npm, all three loading the identical weights.
Why it lives in the engine and not in a client: the hard part of natural-language querying is knowing the schema, and the engine already holds the live collection list. A plan naming a collection that doesn't exist returns 422 with the reason, never a silently empty result set. See Cast below.
Off by default — feature-gated at compile time, flag-gated at runtime, and execute defaults to false so you review the plan before it runs.
Also in 2.8.0 — an engine bug the feature exposed. IdIndex::collections() did a bare read_dir while every other read path overlaid the WAL write buffer, so a brand-new collection was invisible until the 1s flush ticker fired. Unreachable by hand (the ticker fires between keystrokes) but reliable from a script, and latent in Db::compact() too, where a missed collection's live objects would be reclaimed as garbage. Fixed, with regression tests that seed without flushing.
New in 2.5.x:
- Durable-mode auto-flush-on-exit — a durable store flushes buffered writes on
Ctrl+C/SIGTERM, not just on a cleanDrop. Automatic in the Node and Python bindings;Db::install_exit_flush(Arc<Db>)for standalone Rust binaries. See docs/DURABILITY.md. nedb-cli— operate on a store directory offline (head·status·verify·get·scan·flush·repair·export), andnedb-inspector— a deterministic (no-regex, no-LLM) checker that warns when a durable open lacks flush-on-exit wiring. See docs/CLI.md.- Replication contract —
tip()(the latest write), a boundedsince()changefeed, and ascan_status()readiness gate. See docs/REPLICATION.md.
The v3 storage line — consolidated, spec'd, and (as of 2.4.3) cleanly published across every platform. It makes the NEDB v3 segment/pack object store a first-class, fully-documented feature:
--dag-v3(opt-in) — append-only segment store: onefsyncper group-commit,.idxsidecars, compaction, non-destructive dual-read. Took a real itcd chainstate flush from minutes to ~1.3 s. Parsed as a real flag bynedbd-v2as of v2.4.3 (or setNEDB_DAG_V3=1). (See the v3 section below.)NEDB_FAST_FSYNC— macOS fast-fsync: a plainfsync(2)instead ofF_FULLFSYNC(default off; no-op on Linux/Windows).- Durable flush-on-close — and, as of 2.5.x, flush-on-exit on
Ctrl+C/SIGTERM(see docs/DURABILITY.md) — a Windows-safe id-index (percent-encodes filesystem-unsafe ids), and idempotent object re-writes. docs/SPEC.md§3 now formally specifies the v2 object store, the v3 substrate, and the durability model.
NEDB v2 replaces the append-only log (AOF) with a content-addressed Merkle DAG. Every document version is an immutable, BLAKE2b-verified object. Nothing is ever overwritten. As of v2.2.31, restarts after the first open are O(1) warm starts (driven by a MANIFEST of seq + Merkle head), the cold scan is deferred so the daemon accepts connections immediately, and a new GET /events SSE endpoint streams scan progress + per-write events live.
# Run the v2 DAG engine — ships inside pip install nedb-engine
nedbd --dag --data ./data
# or
NEDBD_DAG=1 NEDB_TMK=<32-byte-hex> nedbd --data ./data
curl http://127.0.0.1:7070/health
# {"ok":true,"version":"2.2.31","service":"nedbd","engine":"dag","startup_ready":true,"encrypted":true}
# Tail the live event stream (new in v2.2.31)
curl http://127.0.0.1:7070/events
# event: scan data: {"objects":730000,"of":1310703,"rate":21043,"eta_s":28}
# event: ready data: {"seq":1310703,"head":"b2:9c14e07a…"}
# event: write data: {"seq":1310704,"coll":"beliefs","head":"b2:7af3c11e…"}| Property | v2 DAG | v1 AOF |
|---|---|---|
| Uncorruptable (atomic writes, hash-verified reads) | ✅ | |
| O(1) warm start via MANIFEST (no scan, no replay) | ✅ | ❌ |
| Deferred cold scan (socket open immediately) | ✅ | ❌ |
| O(1) incremental Merkle head (never recomputed) | ✅ | ❌ |
| Parallel writes (no global lock) | ✅ | ❌ |
| BLAKE2b Merkle head on every response | ✅ | ❌ |
| IdIndex sharded across 256 subdirectories | ✅ | ❌ |
| TCP_NODELAY (no 40–200 ms loopback Nagle delay) | ✅ | ❌ |
GET /events SSE log stream |
✅ | ❌ |
| Tombstone deletes (history preserved) | ✅ | ✅ |
| Auto-migrates v1 AOF → v2 DAG on startup | ✅ | — |
| Same HTTP API — Vision, Studio, all clients unchanged | ✅ | ✅ |
v1 AOF engine is still shipped and unchanged — nedbd (no flag) runs v1.
Production status: vision.interchained.org is live on v2.2.31 — 1,310,703 sequences indexed in the Vision database, AES-256-GCM encrypted at rest, at block height 620,989.
Every database stores what. NEDB stores what, when, when it was true, and why — all sealed in a cryptographic hash chain that proves none of it was tampered with.
| Capability | NEDB | SQLite | Redis | MongoDB |
|---|---|---|---|---|
| Hash-chained tamper evidence | ✅ | ❌ | ❌ | ❌ |
Time-travel reads (AS OF seq) |
✅ | ❌ | ❌ | ❌ |
Bi-temporal (VALID AS OF date) |
✅ | ❌ | ❌ | ❌ |
| Causal Write Provenance | ✅ | ❌ | ❌ | ❌ |
| Replay-protected idempotent writes | ✅ | ❌ | ❌ | ❌ |
| SQL + Redis + MongoDB adapters | ✅ | — | — | — |
| Concurrent group-commit daemon | ✅ | ❌ | ✅ | ✅ |
| At-rest AES-256-GCM encryption | ✅ | ❌ | ❌ | — |
pip install nedb-engine # Python ≥ 3.8 — pure-Python + optional Rust native wheel
npm install nedb-engine # Node ≥ 16 — napi-rs prebuilt binariesfrom nedb import NEDB
db = NEDB("./mydata") # durable: every op is AOF-logged, fsync'd, and hash-chained
# db = NEDB() # or in-memory
db.create_index("users", "status", "eq")
db.create_index("users", "bio", "search")
db.put("users", "alice", {"name": "Alice", "age": 31, "status": "active", "bio": "rust hacker"})
db.put("users", "bob", {"name": "Bob", "age": 24, "status": "active", "bio": "python dev"})
# NQL: WHERE + ORDER BY + LIMIT + SEARCH + TRAVERSE + GROUP BY
db.query('FROM users WHERE status = "active" ORDER BY age ASC')
db.query('FROM users SEARCH "rust"')
db.query('FROM users GROUP BY status COUNT')
# Time-travel — AS OF any past sequence
snap = db.seq
db.put("users", "alice", {"name": "Alice", "age": 32, "status": "retired"})
db.get("users", "alice", as_of=snap) # → age 31, status active
# Bi-temporal — VALID AS OF any past date
db.put("policy", "rate_2024", {"pct": 5.0}, valid_from="2024-01-01", valid_to="2024-12-31")
db.put("policy", "rate_2025", {"pct": 6.0}, valid_from="2025-01-01")
db.query('FROM policy VALID AS OF "2024-06-15"') # → rate 5.0
# Causal Write Provenance — why did this write happen?
db.put("inputs", "msg_1", {"text": "user prefers dark mode"})
seq_msg = db.seq
db.put("beliefs", "dark_mode", {"value": True},
caused_by=[seq_msg], evidence="user_message", confidence=0.95)
db.query('FROM beliefs WHERE _id = "dark_mode" TRACE caused_by') # → msg_1
db.query('FROM inputs WHERE _id = "msg_1" TRACE caused_by REVERSE') # → dark_mode
# Relations + graph traversal
db.link("users:alice", "follows", "users:bob")
db.query('FROM users WHERE _id = "alice" TRAVERSE follows')
# Hash-chain integrity
assert db.verify() # cryptographic proof — no tampering
# SQL, Redis, MongoDB compatibility adapters
from nedb import sql_exec, RedisCompat, MongoClient
sql_exec(db, "SELECT * FROM users WHERE status = 'active' ORDER BY age DESC")
r = RedisCompat(db); r.execute("HSET", "user:1", "name", "Alice")
MongoClient(db)["users"].find({"status": "active"}).sort("age", -1).to_list()Running the daemon? nedb.client.NedbClient is the official client for its
HTTP API — extracted from the battle-tested clients that ran a production
Redis→NEDB mainnet migration, speaking the full route surface: queries,
atomic CAS transactions, TTL, indexes, relations, Merkle proofs, and the
Mongo-compat endpoint. Env-var defaults (NEDBD_URL, NEDBD_TOKEN,
NEDB_DB) mirror the daemon's own.
from nedb import NedbClient, PreconditionFailed, op_put
c = NedbClient("http://127.0.0.1:7070", db="app", token="s3cret")
c.ensure_database()
c.put("users", "u1", {"id": "u1", "email": "a@b.c"}, idem="signup-u1")
c.query('FROM users WHERE email = "a@b.c"') # full NQL rides through
c.query("FROM users AS OF 41") # time-travel included
# Atomic all-or-nothing transaction with engine-checked preconditions —
# the primitive that replaces Redis Lua scripts (if_seq: N = CAS, -1 = create-once)
doc = c.get_doc("users", "u1") # docs carry _seq
c.tx([op_put("users", "u1", {**doc, "plan": "pro"}, if_seq=doc["_seq"])])
# Contested writes: retry ONLY on PreconditionFailed, capped backoff
def bump():
d = c.get_doc("counters", "hits") or {"n": 0}
return c.tx([op_put("counters", "hits", {"n": d.get("n", 0) + 1},
if_seq=d.get("_seq", -1))])
c.cas_retry(bump)
# Integrity, verifiable WITHOUT trusting the server
proof = c.proof(c.log(limit=1)[0]["hash"])
from nedb import verify_proof; verify_proof(proof) # -> True, locallyA CAS miss raises the same PreconditionFailed (with the same
.failures shape) the embedded engine raises — code written against
NEDB.tx ports to the HTTP client without changing its except-clauses.
Typed errors throughout: NedbAuthError, NedbNotFound, NedbBadRequest,
NedbConflict, CasExhausted.
Already running on Redis? Wrap your connection in one line and gain NEDB features alongside your existing Redis app — no migration required.
import redis, json
from nedb import wrap_redis
r = wrap_redis(redis.Redis("localhost", 6379), db_name="rideshare")
# Step 1 — register: map Redis key globs to NEDB collections (chainable)
(r.nedb
.register("driver:*", collection="driver", value_parser=json.loads)
.register("trip:*", collection="trip", value_type="hash")
)
# Step 2 — backfill: import all existing Redis data into NEDB in one pass
imported = r.nedb.backfill() # → int (keys imported)
# Step 3 — shadow: all future r.set/hset/... auto-chain into NEDB
r.nedb.shadow_writes = True
# ─── Alice's app keeps running — zero changes ───────────────────────────
r.set("driver:d1", json.dumps({"name": "Bob", "status": "active"})) # ← shadowed
r.hset("trip:t1", mapping={"status": "en_route", "driver_id": "d1"}) # ← shadowed
# ─── New features available on the same connection ──────────────────────
r.nedb.query('FROM driver WHERE status = "active" ORDER BY lat ASC')
r.nedb.verify() # → True (every write chain-verified)
r.nedb.head() # → 64-char BLAKE2b commitment hashIsolation guarantee: NEDB never writes to Alice's namespace. It owns only:
| Key | Type | Purpose |
|---|---|---|
nedb:{db_name}:oplog |
Redis Stream | append-only op log |
nedb:{db_name}:snapshot |
Redis Hash | checkpoint |
nedb:{db_name}:meta |
Redis Hash | index config |
See examples/fakeredis_demo.py for a full local demo (no Redis server needed).
import { NedbCore } from "nedb-engine";
const db = new NedbCore(); // in-memory
// const db = NedbCore.open("./data"); // durable
db.createIndex("users", "status", "eq");
db.put("users", "alice", JSON.stringify({ name: "Alice", age: 31, status: "active" }));
// Time-travel
const snap = db.seq(); // BigInt
db.put("users", "alice", JSON.stringify({ name: "Alice", age: 32, status: "retired" }));
JSON.parse(db.getAsOf("users", "alice", snap)).age; // → 31
// Full NQL
const rows = db.query('FROM users WHERE status = "active" ORDER BY age ASC');
rows.map(r => JSON.parse(r));
// Tamper evidence
db.verify(); // → true
db.head(); // → 64-char BLAKE2b commitment hash
db.seq(); // → BigIntnedbd runs NEDB as a long-lived process with an HTTP/JSON API and an optional RESP2 wire protocol. Built on a single-writer group-commit sequencer — parallel reads, batched durable writes, one hash-chain per database, zero write-write races.
nedbd # :7070, data ./nedb-data (v1 AOF engine)
nedbd --dag --data ./data # v2 DAG engine (or NEDBD_DAG=1)
NEDBD_RESP2_PORT=6380 nedbd # also speak RESP2 (redis-cli compatible)
nedbd --log-level 2 # 0=errors 1=requests 2=deploy 3=verbose
# Live event stream (new in v2.2.31) — SSE: scan progress, ready, per-write head
curl http://127.0.0.1:7070/eventsAlongside the daemon, cargo install nedb-engine ships nedb-cli — operate on a store directory offline (head/status/verify/get/scan/flush/repair/export) — and nedb-inspector, a deterministic checker that warns when a durable open lacks flush-on-exit wiring. Full reference: docs/CLI.md.
- Warm start — every restart after the first open reads the
MANIFESTfile and restoresseq+ Merkleheadin O(1). No scan, no replay, independent of dataset size. Boots in milliseconds. - Cold start — first open of an existing dataset spawns the integrity scan in a background thread and accepts connections immediately. Reads serve instantly from the content-addressed DAG; writes return
HTTP 503 startup in progressuntil thestartup_readygate flips. Progress (objects, rate, ETA) streams overGET /events.
| Variable | Default | Description |
|---|---|---|
NEDBD_DAG |
0 |
Set 1 to launch the v2 DAG engine (nedbd-v2). Same as --dag. |
NEDBD_HOST |
127.0.0.1 |
Bind address. v2.2.31 defaults to loopback (was 0.0.0.0) — security hardening fix. Set explicitly to 0.0.0.0 to expose. |
NEDBD_PORT |
7070 |
HTTP bind port. |
NEDBD_TOKEN |
unset | Optional bearer token; required on every /v1/* request when set. |
NEDB_TMK |
unset | 32-byte hex AES-256-GCM at-rest encryption key. |
NEDBD_DATA |
./nedb-data |
Root directory. v2 creates dag/, IdIndex sharded across 256 subdirectories, and a small MANIFEST file. |
NEDBD_CAST |
0 |
Set 1 to enable the /cast natural-language planner. Same as --cast. Requires a build with --features cast. See Cast. |
NEDBD_CAST_MODEL |
unset | Explicit path to a model.cast container. Otherwise searched in the data dir, $CAST_HOME, and ~/.cache/nedb-cast-slm/. |
# Create a database with seed data and relations
curl -X POST :7070/v1/databases -d '{
"name": "shop",
"init": {
"indexes": [["users","status","eq"]],
"seed": {"users": [{"_id":"u1","name":"Alice","status":"active"}]},
"links": [["users:u1","buys","orders:o1"]]
}}'
# Query (full NQL including time-travel and bi-temporal)
curl -X POST :7070/v1/databases/shop/query \
-d '{"nql":"FROM users WHERE status = \"active\" ORDER BY name ASC"}'
# Verify the hash chain
curl :7070/v1/databases/shop/verify
# MongoDB-compatible endpoint
curl -X POST :7070/v1/databases/shop/mongo \
-d '{"collection":"users","op":"find","filter":{"status":"active"},"limit":10}'From redis-cli — no Redis installation needed:
redis-cli -p 6380 SELECT shop
redis-cli -p 6380 SELECT shop EVAL 'FROM users SEARCH "alice"' 0
redis-cli -p 6380 SELECT shop EVAL 'FROM users AS OF 10 WHERE status = "active"' 0
redis-cli -p 6380 SELECT shop EVAL 'FROM beliefs TRACE caused_by' 0FROM <collection>
[ AS OF <seq> ] transaction time (when was it written?)
[ VALID AS OF "<date>" ] valid time (when was it true in the world?)
[ WHERE <field> <op> <value> (AND ...) ] op: = != < <= > >=
[ SEARCH "<text>" ] full-text search
[ ORDER BY <field> [ASC|DESC] ]
[ TRAVERSE <relation> ] graph traversal
[ TRACE caused_by [REVERSE] ] causal provenance (why? / what did this cause?)
[ LIMIT <n> ]
[ GROUP BY <field> [COUNT|SUM f|AVG f|MIN f|MAX f] ]
Combine both time axes:
# What did the system know at seq 200 about what was true on 2024-02-15?
db.query('FROM policy AS OF 200 VALID AS OF "2024-02-15"')New in 2.8.0. Optional, feature-gated, off by default.
Ten clauses and six operators. That's the whole grammar above — small enough that a 3.33M-parameter model can learn it completely, and small enough that shipping every query to a frontier model is an absurd amount of machinery.
So we trained one. It runs on CPU, in-process, in milliseconds.
curl -X POST localhost:7070/v1/databases/shop/cast \
-H 'Content-Type: application/json' \
-d '{"prompt":"orders over 100"}'{
"prompt": "orders over 100",
"nql": "FROM orders WHERE total > 100",
"valid": true,
"collection": "orders",
"collection_known": true,
"collections": ["orders"],
"executed": false,
"seq": 3,
"head": "262fd9…"
}The model is nedb-cast-slm, and NEDB built it as much as it consumes it.
NEDB's parser was the training pipeline. It generated the corpus (sample a random plan → render NQL → render a human paraphrase; 200,000 pairs in 16.5 seconds, perfect labels, zero annotation cost). It was the grader — scoring parsed plan equality, not string equality, so FROM orders WHERE total > 99 and from orders where total>99 both earn full credit. And it was the gate: no example entered the corpus unless it round-tripped through the real parser to a canonically identical plan.
Most text-to-DSL projects hand-write a verifier and hope it's right. We didn't write one — it already shipped, and it's the same code the database runs in production.
Training lineage lives in NEDB too, chained by caused_by:
datasets ──▶ training_runs ──▶ checkpoints ──▶ evals
db.query("FROM evals TRACE caused_by") # the exact data behind any scoreThe hard part of natural-language querying is not the model. It's the schema — and a client has to fetch the collection list and pass it in, where it's stale on arrival. The engine already holds the live list.
So the plan is checked against collections that actually exist, at the moment of the call:
{ "prompt": "show me all stylists",
"nql": "FROM stylists",
"valid": true,
"collection_known": false,
"error": "collection \"stylists\" does not exist in \"shop\"" }HTTP 422. Not zero rows — zero rows reads as "no matching data", which would be a lie. The query was perfectly well-formed; the collection was imagined. That's the model's known failure mode on an unfamiliar schema, and the engine is the one component positioned to catch it.
Every nedbd client — Python, Node, Studio, curl — inherits this without writing a line.
| The model never executes | It emits text. The text goes to the same nql::query path a hand-typed query uses. No second executor exists to audit. |
| Validation is parsing | nql::parse and nql::execute share one code path, so they cannot disagree about what is well-formed. Invalid output returns 422 with the offending text. |
execute defaults to false |
You get a plan for review. Running a guess silently is worse than admitting uncertainty. |
That last default earns its keep. A real miss, from a real run:
prompt "paid orders over 100"
nql FROM orders WHERE status = "paid" LIMIT 100 ← wrong
correct FROM orders WHERE status = "paid" AND total > 100
It read "over 100" as LIMIT 100 and dropped the predicate. The count still came back 2 — because both paid orders happened to exceed 100. A count-only assertion would have scored it a pass. A human reading LIMIT 100 catches it in a heartbeat; an auto-executing client does not.
Multi-predicate WHERE is the model's weakest clause: 85.1% exact-plan match on eval, 61.2% on adversarial holdout. The model card documents every failure mode with examples.
To run it anyway, ask:
curl -X POST localhost:7070/v1/databases/shop/cast \
-d '{"prompt":"orders over 100","execute":true}'
# → { …, "executed": true, "count": 2, "rows": [ … ] }A literal the model invented rather than copied:
"memories about pricing" -> FROM memories SEARCH "handoff"
That query parses. It names a real collection. It returns real rows. Both
valid and collection_known are true — and it answers a question nobody
asked. Measured on the released checkpoint:
| terms | in vocabulary | copied correctly |
|---|---|---|
release flow · guardrail · handoff |
yes | 3/3 |
pricing · deadlines · kubernetes |
no | 0/3 — all became "handoff" |
So the response carries a drift field when a quoted literal is absent from the
prompt:
{ "nql": "FROM memories SEARCH \"handoff\"",
"valid": true,
"collection_known": true,
"drift": "generated the literal \"handoff\", which does not appear in the prompt — likely outside the model's vocabulary and substituted. Verify before trusting these results." }It is advisory, never fatal — the plan may still be what you wanted, and discarding a valid query would be its own kind of lie. But an unattended caller should treat it as a third gate:
if plan["valid"] and plan["collection_known"] and not plan.get("drift"):
rows = await db.query(plan["nql"])Same root cause as truncated digits (height 400000 → 4000): no copy
mechanism over prompt tokens. Verified at 24/24 on real model output — 3 true
positives, 21 true negatives, zero false alarms, including the case that matters
most (correctly inferred enum values like "refunded orders" → status = "refunded" stay silent).
Two gates, because most deployments want neither the model dependency nor the weights:
# compile-time
cargo install nedb-engine --features cast
# or from a source checkout — builds the engine only, not the language bindings
cd rust && cargo build --release --features cast
# weights (~13 MB) — GitHub release asset, checksum-verified on load
curl -L -o ./data/model.cast \
https://github.com/aiassistsecure/nedb-cast-slm/releases/download/v10.30.90/model.cast
# runtime
nedbd --dag --cast ./data
# cast enabled — 3.33M params, vocab 581, ./data/model.castSearch order: $NEDBD_CAST_MODEL → <data_dir>/model.cast → $CAST_HOME/model.cast → ~/.cache/nedb-cast-slm/v10.30.90/model.cast (the Python/npm cache location, so a machine that has run either package is already ready).
Built without the feature, the route returns 501 rather than 404 — clients can detect the capability instead of guessing. Built with it but missing weights, the daemon logs loudly and serves everything else normally.
Verify the whole path:
./scripts/test-cast.sh --boot # boots a daemon, seeds, casts, executes, checks failure modes./scripts/seed-shop.sh # a shop database the model already understands
. ./scripts/nedb.sh # bash / zsh / Git Bash
nedb-dbs # which databases exist
nedb-use shop # pick one
cast "orders over 100" # plan only — nothing runs
cast -x "orders over 100" # plan AND executeSeed the names it was trained on. The model learned six synthetic domains, and shop is one of them — orders(total, status, quantity, customer, placed_at, discounted), products(price, stock, category, rating, title), customers(age, city, tier, lifetime_value, name), plus the relations purchased / reviewed / belongs_to. Those names live in its 581-token vocabulary.
Call your collection purchases with a cost field and it will still emit FROM orders WHERE total > …, because that is what it knows. It is a 3.3M-parameter model, not a schema reader. On an unfamiliar schema you get collection_known: false — caught, not silently wrong, but caught.
nql FROM orders WHERE total > 100
valid yes collection orders known: yes
executed no (add -x to run it)
The summary leads with the NQL because reading it is the job. valid: yes means it parses, not that it's what you meant — LIMIT 100 parses perfectly.
NEDB=http://host:7070 points at a remote daemon. Prompts are JSON-escaped, so apostrophes and quotes are safe.
Accuracy varies by clause, so phrasing matters more than length:
| you want | say | eval |
|---|---|---|
TRACE caused_by |
what caused these checkpoints | 96.5% |
TRAVERSE |
orders traverse placed_by | 93.3% |
one WHERE |
orders over 100 · active drivers | 91.2% |
LIMIT |
top 5 orders | 91.1% |
SEARCH |
search orders for refund | 90.5% |
ORDER BY |
orders sorted by total descending | 87.7% |
two+ WHERE |
paid orders with total over 100 | 85.1% |
GROUP BY + agg |
orders grouped by status with sum of total | 77.0% |
Two habits that avoid most misses:
- Name the field when a number could be a limit. "orders with total over 100" beats "orders over 100" — bare "over N" is what produced the
LIMIT 100miss above. - Check numbers over four digits. Digits are tokenized one at a time, so
height 400000can come back4000.
v2 DAG Rust server (v2.2.31, Intel iMac — 10k writes / 100k reads / 30k objects, AES-256-GCM on):
| Operation | Throughput | p50 | p99 |
|---|---|---|---|
| Sequential writes | 418 ops/s | 2.3 ms | 3.3 ms |
| Point-lookup reads | 478 ops/s | 2.0 ms | 3.0 ms |
| ORDER BY queries | 489 ops/s | 1.8 ms | 4.3 ms |
| Batch writes (500 ops/req) | 1,104 ops/s | 0.9 ms | 1.2 ms |
| Tamper-verify (30k objects) | ~21,000 BLAKE2b/sec | — | 1.38 s total |
p99 latencies hold because of TCP_NODELAY on the axum listener — without it macOS loopback adds the Nagle algorithm's 40–200 ms delay on small writes.
v1 Python server (baseline — single-threaded AOF):
| Operation | Throughput | p99 latency |
|---|---|---|
| Sequential PUT | ~23/s | 44 ms |
| Concurrent PUT (16 workers) | ~92/s | 48 ms |
| Batch PUT (500 ops/request) | ~520 ops/s | 1.9 ms/op |
| Point-lookup read (NQL) | ~23/s | 44 ms |
| Rust napi PUT (FFI) | ~70K/s | — |
| Rust napi GET (FFI) | ~330K/s | — |
Reproduce with the included benchmark:
NEDBD_DAG=1 nedbd --data /tmp/perf &
python3 tests/test_dag_perf.py --n 10000 --reads 100000v3 is an opt-in storage substrate that replaces the loose one-file-per-object layout with append-only segment packs — the difference between a chainstate flush that takes minutes and one that takes under two seconds. It is off by default (byte-for-byte v2), enabled with one flag, and transparent to everything above the storage layer: NQL, AS OF, VALID AS OF, TRACE, the BLAKE2b Merkle head, and causal provenance all behave identically.
v2 stores every document version as its own content-addressed file at objects/{hash[:2]}/{hash[2:]}. That makes writes trivially atomic (write .tmp → rename) and corruption-proof — but each write costs a file create + fsync + rename plus a directory B-tree update. At scale that filesystem-metadata churn dominates: on a busy disk it caps sustained writes around ~185/s, and a batch flush of a few thousand objects degrades into minutes. The bottleneck is the number of files touched, not the bytes written.
v3 batches objects into append-only segment packs — objects/segments/seg-NNNNNN.dat — where each record is [content_len: u32-LE][content]. A write appends to the active segment and updates an in-memory hash → (segment_id, offset, len) map; a batch commits with a single fsync. Thousands of per-file syscalls collapse into one sequential append plus one durability point, so flush cost scales with bytes (sequential I/O), not object-count × syscall overhead.
- Compaction / pruning —
compact()keeps the live set (the current version of every document, resolved from the id-index), rewrites those records into fresh segments, and reclaims the superseded/dead versions. .idxsidecars — each segment carries a sidecar (NIX1magic + entry count + fixed 44-byte entries + a BLAKE2b-256 checksum) so reopen rebuilds the in-memory index by reading the sidecar instead of scanning the whole pack. A missing or corrupt sidecar falls back to a full scan-and-heal — slower, never fatal.- Dual-read migration — opening an existing v2 store in v3 mode is non-destructive: old loose objects stay fully readable, and only new writes go to segments. No migration step, no downtime, no rewrite.
- Durable flush-on-close —
flush_all()(andDb'sDrop) fsync the active segment, matching the flush-on-close contract of sled / RocksDB.
# Engine / nedbd-v2 (the native daemon from npm / the native wheel)
nedbd-v2 --dag-v3 --data /var/lib/nedb # real flag as of v2.4.3 — or set NEDB_DAG_V3=1
# itcd — Bitcoin-fork node embedding NEDB via nedb-ffi
interchainedd -dagv3 # puts chainstate AND block index on segmentsThe switch is read once, when each database's object store is constructed at open time. Default off → v2 loose objects.
itcd (a Bitcoin Core 0.21 fork that replaces LevelDB chainstate with NEDB) syncing on -dagv3, measured FlushStateToDisk on real chainstate:
| Flush (coins → disk) | v3 segment store | v2 loose store |
|---|---|---|
| 2,002 coins / 275 kB | 1.93 s | minutes |
| 2,549 coins / 366 kB | 1.71 s | minutes |
Note the larger batch finishing faster — v3's cost is dominated by the single per-batch fsync, not per-coin work, so effective throughput (~1,000–1,500 coins/s here) climbs as batches grow, against the loose store's ~185 writes/s metadata ceiling. The gap only widens as the UTXO set grows: sequential-append cost tracks data volume, while per-file cost compounds with object count.
Reach for v3 on high-write, large-object-count workloads — blockchain chainstate / block index, event sourcing, high-frequency agent memory. For small or read-mostly stores the loose layout is perfectly fine, which is exactly why v3 stays opt-in.
┌──────────────────────────────────────────────────────────┐
put/del → │ OpLog (BLAKE2b hash chain · per-client nonce · │ ← single source of truth
link │ idempotency keys · causal provenance fields) │
└───────────────┬──────────────────────────────────────────┘
deterministic fold │ (state = pure function of the log)
┌──────────────┬──────────┴──────┬───────────────┬────────────────┐
▼ ▼ ▼ ▼ ▼
MVCC store Relations Indexes CauseMap BlobStore
(time-travel) (graph+AS OF) eq/ord/search (reverse index) (Cascade CDC)
┌─────────────────────────────────┐
Thread-safe → │ Sequencer (group-commit) │ ← single writer, parallel readers
│ — one committer thread/db │
│ — batch fsync │
└─────────────────────────────────┘
Compatibility adapters: SQL · Redis · MongoDB
Wire protocols: HTTP/JSON · RESP2
Encryption: AES-256-GCM at-rest (TMK/DEK double-envelope)
Connect to any running nedbd instance from Python or TypeScript without embedding the engine:
pip install nedb-engine-client # async Python
npm install nedb-engine-client # TypeScript / Node.js 18+from nedb_client import NedbClient
async with NedbClient("http://127.0.0.1:7070", db="mydb") as db:
await db.put("blocks", "618000", {"height": 618000})
rows = await db.query("FROM blocks ORDER BY height DESC LIMIT 10")
head = await db.head() # BLAKE2b Merkle root — changes on every write
ok = await db.verify() # tamper-evidence check across all objectsimport { NedbClient } from "nedb-engine-client";
const db = new NedbClient({ url: "http://127.0.0.1:7070", db: "mydb" });
await db.put("blocks", "618000", { height: 618000 });
const rows = await db.query("FROM blocks LIMIT 10");python/nedb/ reference engine (pure Python — always-works baseline)
rust/
nedb-core/ v1 production Rust engine (shared by both runtimes)
nedb-py/ maturin PyO3 binding → PyPI native wheels
nedb-node/ napi-rs binding → npm native addons
nedb-v2/ v2 DAG engine (tokio + axum + BLAKE2b DAG)
client/
python/ nedb-client — async Python HTTP client (pip install nedb-engine-client)
node/ nedb-client — TypeScript HTTP client (npm install nedb-client)
tests/ engine + concurrent + causal + bitemporal + deploy + perf benchmarks
examples/ resp2_python.py resp2_demo.sh
docs/ index.html reference.html SPEC.md
- Hash-chained append-only log — tamper evidence, replay protection, idempotency
- MVCC time-travel —
AS OF seq - Bi-temporal —
VALID AS OF "date"(transaction time + valid time) - Causal Write Provenance —
caused_by,evidence,confidence,TRACE - Durable AOF persistence + snapshot checkpoints
- Concurrent group-commit sequencer (nedbd, 15K writes/s under load)
- AES-256-GCM at-rest encryption (TMK/DEK double-envelope)
- SQL / Redis / MongoDB compatibility adapters
- RESP2 wire protocol (redis-cli / redis-benchmark compatible)
- Rust native core — napi-rs (npm) + maturin PyO3 (PyPI)
- Self-healing AOF — auto-truncates corrupt tail on startup, never hangs
- v2 DAG engine — content-addressed Merkle DAG, atomic writes, instant cold start
-
nedbd --dag— one flag switches to v2 Rust engine; v1 untouched - BLAKE2b Merkle head — tamper-evident root on every response
- Tombstone deletes — history preserved in DAG, live id removed from index
- Auto-migration — v1 AOF → v2 DAG on first
--dagstartup - nedb-client — async Python + TypeScript HTTP client (
pip/npm install nedb-client) - Intel Mac support — native wheels for
aarch64+x86_64Apple Darwin - v3 segment/pack object store — opt-in
--dag-v3: append-only packs, one fsync per batch, compaction +.idxsidecars, non-destructive dual-read (minutes → <2s chainstate flush on itcd) - In-memory DAG mode —
Db::in_memory()for zero-disk ephemeral sessions - PyO3 + napi-rs bindings updated to v2 DAG API
- NEDB Studio DAG mode toggle
- Merkle inclusion proofs — prove a document existed at a specific time to a third party
- Git-style branching — fork database state, experiment, merge or discard
- Agent Memory SDK —
Memory.remember()/Memory.recall()/Memory.trace() - Live query subscriptions (SSE) — push diffs when query results change
Prompt-to-database scaffolding GUI with schema graph, NQL console, time-travel slider, causal provenance panel, and MongoDB/SQL/Redis tabs. Deploy from a description, query live data, edit inline.
studio.interchained.org · github.com/aiassistsecure/nedb-studio (GPLv3)
| Repo | Description |
|---|---|
| aiassistsecure/nedb | Source — engine, Rust core, CI |
| aiassistsecure/nedb-studio | Studio UI (GPLv3) |
Packages: PyPI nedb-engine · npm nedb-engine
NEDB and its two distributions (crypto-database, aof-db) ship from a single version tag via one committed tool:
python3 scripts/release.py "vFROM" "vTO"
# e.g. the first run of the 2.4.468 line:
python3 scripts/release.py "v2.4.68" "v2.4.468"Both arguments require the leading v (e.g. v2.4.468). The script:
- Bumps every version-bearing manifest (npm / PyPI / crates + the engine crate, clients, and the maturin project) in the flagship and both distribution forks from
FROMtoTO, opening and merging a release PR per repo. - Repoints the
distributions/*submodules to the freshly-bumped fork masters. - Tags
vTOonmaster, firing CI/CD —release.yml(flagship) +release-distros.yml(distros) + Codemagic (macOS wheels/addons) — to publishnedb-engine+crypto-database+aof-dbaligned on one version across npm, PyPI, and crates.io.
It is idempotent: a manifest line already at TO is left untouched, a repo already fully at TO produces no empty PR, and an existing vTO tag is left in place — so a half-finished release can be re-run safely, and the remaining steps (submodule repoint, tag) always run even when the version was already correct.
Requires GITHUB_TOKEN (repo + workflow scope) in the environment. It never force-pushes master and never commits to it directly — every change lands through a branch + PR + merge.
See LICENSE file. · © INTERCHAINED, LLC — interchained.org
Built by Mark Allen Evans Jr. (INTERCHAINED, LLC) with Claude Sonnet 4.6 on Hyperagent.
"Take one idea, turn it into an LP, then an app, then a system, then a platform, then infrastructure that is irreplaceable."
