Test nedb-engine end-to-end — clone → install → verify → benchmark

Get the source on your machine

Two paths. If you only want to use the engine, the registries are enough — skip to step 3. If you want to verify it, audit it, or run the full test suites and benchmarks, clone the source first: everything below runs from the same checkout the releases ship from.
terminal
```
$ git clone https://github.com/aiassistsecure/nedb
$ cd nedb
$ git log --oneline -1
# note the commit — every release tag is also in the history:
$ git tag | tail -5
```
What you should see: git log on the current master, and a wall of version tags — the repo has shipped 170+ releases, one per line of git tag. The release history is the audit trail; this database does not just store auditable data, its own development is auditable the same way.

Why this matters: every claim on the README maps to a test file or a benchmark script in this tree. The rest of this walkthrough runs them, so nothing stays a claim.
Install from the registries — and check what actually shipped

Evaluate what users get, not the checkout: the packages on PyPI, npm and crates.io are built by the release pipeline from a single git tag, all three carrying the same version at the same moment.
terminal — python side
```
$ python3 -m venv venv && source venv/bin/activate
$ pip install nedb-engine
$ python3 -c "import nedb; print(nedb.__version__)"
9.0.0
$ which nesql nedbd
# both binaries should be on PATH — the compiled Rust tools ship inside the wheel
```
terminal — node side
```
$ mkdir nedb-node-demo && cd nedb-node-demo
$ npm init -y && npm install nedb-engine
$ node -e "const {NedbCore}=require('nedb-engine'); console.log(typeof NedbCore)"
function
```
What you just verified: the version matches the latest git tag from step 1, the nesql and nedbd binaries ride along on the Python install, and the Node addon loads as a real napi binding — not a stub. On platforms without a prebuilt binary, the Python package falls back to a pure-Python core (slower, same behavior); Node has no fallback, so check the platform table in the README.

The tri-distribution discipline: one tag publishes all three registries at once. If the versions ever disagree, that is a release bug — file it, because the whole point is lockstep.
Run the engine — write, query, time-travel, verify (Python)

The core semantics, executed by you. This is the block that proves the four claims: durable storage, time travel, causal provenance, tamper evidence.
terminal — python (paste this whole block)
```
$ python3 <<'EOF'
from nedb import NEDB

db = NEDB("./mydata")          # durable, hash-chained
db.put("users", "alice", {"name": "Ann", "age": 31})

# 1) TIME TRAVEL — every version is kept, so any past seq is reachable
snap = db.seq
db.put("users", "alice", {"name": "Ann", "age": 32})
print(db.get("users", "alice", as_of=snap)["age"])

# 2) CAUSAL PROVENANCE — a write can cite the writes that caused it
db.put("inputs", "m1", {"text": "prefers dark mode"})
seq = db.seq
db.put("beliefs", "b1", {"v": 1}, caused_by=[seq])
r = db.query('FROM beliefs WHERE _id = "b1" TRACE caused_by')
print(r[0]["_id"])

# 3) TAMPER EVIDENCE — the chain either verifies or it does not
print(db.verify())
EOF
```
output — what you should see (do not paste)
```
31   # ← the version AT that sequence, not the current one
m1   # ← the causal PARENT, walked automatically
True # ← the chain recomputed clean
```
What happened, line by line: the first put wrote version 1 of alice and assigned it a sequence number. db.seq captured that position. The second put wrote version 2 at a new sequence — nothing overwrote anything. get(as_of=snap) walked the version chain back to exactly that point. The caused_by array on the second put declared a causal edge, and TRACE walked it. verify() recomputed the entire BLAKE2b chain and compared.

Break it on purpose: open ./mydata in a hex editor, flip one byte, run verify() again → False. That is the whole product in one demo: the tamper is not hidden, it is detected. Restore the byte and verify returns to True.

Also try: db.query('FROM users AS OF ' + str(snap)) — the whole collection as it was. And db.head — the current Merkle head, a 64-char BLAKE2b commitment that changes on every write.
The same in Node — the full napi surface

The Node binding is the same Rust core behind a real typed API. This walks the entire surface: durable opens, indexes, time travel, the neSQL router, graph edges, replication contract, and the flush discipline. Every output below is a real captured run.
step 1 — durable open + indexes + writes
```
$ node -e "
const { NedbCore } = require('nedb-engine');
const db = NedbCore.open('./store');        // durable — writes land on flush
db.createIndex('orders', 'status', 'eq');   // eq | sorted | search
db.put('orders', 'o1', JSON.stringify({who:'acme', total:100, status:'paid'}));
db.put('orders', 'o2', JSON.stringify({who:'globex', total:250, status:'open'}));
console.log('seq:', db.seq().toString());
"
seq: 2   // BigInt — the next-to-assign counter
```
Key facts about the surface, learned by running it: documents go in as JSON strings and come back as JSON strings (the binding is a thin, honest layer over the Rust core — no silent re-shapes). seq() returns a BigInt because sequence numbers are 64-bit. NedbCore.open is durable; new NedbCore() is in-memory. The wrapper arms signal handlers so a SIGINT flushes before exit — set NEDB_NO_EXIT_FLUSH=1 if your app manages its own shutdown.
step 2 — time travel + the full document metadata
```
$ node -e "
const { NedbCore } = require('nedb-engine');
const db = NedbCore.open('./store');
const snap = db.seq();                       // capture before the update
db.put('orders','o1', JSON.stringify({who:'acme', total:140, status:'paid'}));
console.log('current :', db.get('orders','o1'));
console.log('as-of   :', db.getAsOf('orders','o1', snap - 1n));
"
current : {"who":"acme","total":140,"status":"paid","_id":"o1","_hash":"e4ce19…","_seq":4,"_coll":"orders"}
as-of   : {"who":"acme","total":100,…,"_seq":1,…}
```
Read the metadata that came back: every document carries _id, _hash (BLAKE2b of the content), _seq (its position in the log), and _coll. These are real columns — filterable, selectable, sortable in queries. getAsOf takes a BigInt sequence (pass snap - 1n to see the version before your last write). Deletes are the same shape: db.delete('orders','o1') writes a tombstone — the current read returns null, and getAsOf at any earlier sequence still returns the document, because a delete is history, not erasure.
step 3 — neSQL from Node: both dialects, one method
```
$ node -e "
const { NedbCore } = require('nedb-engine');
const db = NedbCore.open('./store');
console.log(db.nesql('SELECT who, total FROM orders WHERE total > 100'));
console.log(db.nesql('FROM orders WHERE status = \"paid\"'));
console.log(db.nesqlDialect('FROM orders WHERE total > 100'));
"
[ '{"who":"acme","total":140}', '{"who":"globex","total":250}' ]
[ '{"who":"acme","total":100,"status":"paid",…}' ]
nql
```
nesql() accepts both halves of the language — PostgreSQL SQL and the FROM-form — and routes on the leading keyword, structurally. nesqlDialect() tells you which half a statement was read as, so a routing surprise is debuggable instead of mysterious. SQL statements run through the full evaluator (joins, subqueries, set operations); FROM-form statements run through the native clause path. One engine underneath both.
step 4 — graph edges + the replication contract
```
$ node -e "
const { NedbCore } = require('nedb-engine');
const db = NedbCore.open('./store');
db.link('users:alice', 'bought', 'orders:o1');
console.log('neighbors:', db.neighbors('users:alice','bought'));
console.log('inbound  :', db.inbound('orders:o1','bought'));
console.log('tip      :', db.tip().slice(0,70));
console.log('since    :', db.since(0n, 10).slice(0,70));      // BigInt cursor!
console.log('scan     :', db.scanStatus().slice(0,70));
console.log('verify   :', db.verify(), '| head:', db.head().slice(0,16));
db.flush();
"
neighbors: [ 'orders:o1' ]
inbound  : [ 'users:alice' ]
tip      : {"_from":"users:alice","_rel":"bought","_to":"orders:o1",…}
since    : {"nodes":[…],"from_seq":0,"to_seq":6,"head_seq":6,"has_more":false}
scan     : {"scan_complete":true,"seq_index_ready":false,"tip_seq":6,…}
verify   : true | head: a904934249721d5a
```
Four things worth noticing. Links are real edges — neighbors walks forward, inbound walks backward, both have *AsOf variants for graph queries at a past sequence. tip() returns the literal latest write — note the link itself is a stored document in the DAG, not a foreign key. since() is the changefeed: pass a BigInt cursor (a plain 0 throws — this bites everyone once) and it pages every node after it, with has_more telling you whether to keep draining. scanStatus() reports replication readiness: gate followers on seq_index_ready, not scan_complete — on a warm boot the seq index is deliberately empty, and a follower that trusted scan_complete would believe it is caught up while holding nothing.
step 5 — durability: prove the flush discipline yourself
```
# A — put, then kill -9 immediately. THE WRITE IS LOST.
$ rm -rf dur-a && node -e "
const { NedbCore } = require('nedb-engine');
const db = NedbCore.open('./dur-a');
db.put('k','1', JSON.stringify({v:'survives'}));
process.kill(process.pid, 'SIGKILL');      // no flush, no time to tick
"
$ node -e "const { NedbCore } = require('nedb-engine');
console.log(NedbCore.open('./dur-a').get('k','1'));"
null

# B — put, flush(), then kill -9. THE WRITE SURVIVES.
$ rm -rf dur-b && node -e "
const { NedbCore } = require('nedb-engine');
const db = NedbCore.open('./dur-b');
db.put('k','1', JSON.stringify({v:'survives'}));
db.flush();                                // <-- the whole lesson
process.kill(process.pid, 'SIGKILL');
"
$ node -e "const { NedbCore } = require('nedb-engine');
console.log(NedbCore.open('./dur-b').get('k','1'));"
{"v":"survives","_id":"1","_hash":"…","_seq":1,"_coll":"k"}

# C — put, wait past one 1000ms tick, then kill -9. ALSO SURVIVES.
$ rm -rf dur-c && node -e "
const { NedbCore } = require('nedb-engine');
const db = NedbCore.open('./dur-c');
db.put('k','1', JSON.stringify({v:'survives'}));
setTimeout(() => process.kill(process.pid, 'SIGKILL'), 1500);
"
$ node -e "const { NedbCore } = require('nedb-engine');
console.log(NedbCore.open('./dur-c').get('k','1'));"
{"v":"survives","_id":"1","_hash":"…","_seq":1,"_coll":"k"}
```
Run A first, and expect to lose the write. That is not a bug and it is the most useful thing on this page: put() returns before its index entry is durable. A acknowledges, then dies inside the same millisecond — no flush() called, and the 1-second manifest ticker never got a chance to fire — so there is nothing on disk to find. B and C are the two ways to close that window: ask for durability explicitly, or let the ticker arrive. (Your _hash will not match a value printed here and is not meant to — it is elided above for that reason; what you are checking is null versus a document.)

The contract, stated plainly: an acknowledged write is on disk within one tick, or at exit — whichever comes first. So what you can lose to a hard kill or a power cut is bounded at the last 1000 ms of acknowledged writes (NEDB_FLUSH_MS tunes it; min 50, 0 disables the ticker when your host owns its own cadence).
The flush discipline — what to actually do in your app. Do not call flush() after every put() unless you mean it: it is an fsync, and the benchmark cost is real (≈7.9K/s durable vs ≈78K/s buffered). Pick per write class:
- Money, orders, auth, anything you would refund or apologise for — flush() before you acknowledge to your own caller. Treat the write as unacknowledged until it returns.
- Batches and imports — one flush() at the end of the batch, not per row. You get one fsync instead of N and the same guarantee for the batch as a whole.
- Telemetry, caches, feeds, anything you would re-derive — no explicit flush. The ticker is exactly the right tool and a bounded 1-second loss is the correct trade.
- Short-lived processes — a script that writes and exits within a tick is case A. A CLI, a lambda, a cron job or a test that does not flush before returning can lose everything it wrote. This is the sharpest edge in the whole API, because on a graceful exit the exit hook hides it and you only find out the first time something kills you.
Which mechanism saves you, and when. Three separate things, and they are not interchangeable:
- flush() — synchronous, immediate, works under any death including SIGKILL, because the bytes are already down before the call returns.
- the 1000 ms manifest ticker — armed by a durable open() in both bindings. This is what saves case C, and the only thing that bounds your loss on kill -9, an OOM-kill, or a power cut.
- the exit hook — covers the last partial tick on a graceful exit: a normal return, SIGINT, SIGTERM. It cannot help with SIGKILL, which POSIX makes impossible to catch, block or ignore. An earlier version of this page credited the exit hook for surviving SIGKILL; that was wrong in the one way worth being loud about, because it named a mechanism that is guaranteed not to run and so taught the opposite of the discipline above.
The full contract — every failure mode that motivated it, and the production incident that found it — is in docs/DURABILITY.md.

Full API surface in the binding: put, putEx, delete, deleteEx, get, getAsOf, query, nesql, nesqlDialect, createIndex, link, unlink, neighbors, neighborsAsOf, inbound, inboundAsOf, tip, tipCollection, since, scanStatus, verify, head, seq, flush — check Object.getOwnPropertyNames(Object.getPrototypeOf(db)) yourself. The *Ex variants carry client/nonce/idempotency-key fields for the CAS transaction paths.

The nesql CLI — SQL from a shell, exit codes as a contract

The same engine as a standalone binary: no daemon, no port, opens the store directly. Built for scripts as much as people — the exit code carries the verdict, so a shell pipeline can trust it.

terminal

$ nesql --db ./mydata query "SELECT name, age FROM users ORDER BY age DESC"
{"name":"Ann","age":32}
(1 rows)

$ nesql --db ./mydata query "FROM users WHERE age > 30"
{...}
(1 rows, 1 scanned)

$ nesql --db ./mydata version
nesql    9.0.0  ·  engine 9.0.0  ·  grammar_v1 (f82e7e02…)

exit	meaning
0	success — done, or the check ran and passed
1	failure — the operation ran and did not succeed
2	usage — the command line was not understood
3	could not determine — pruned history (not a corrupt one!)
4	not found
5	unsupported

Try the contract yourself: nesql --db ./mydata get users nobody; echo $? → prints the not-found message and 1. A missing document is exit 4; a missing database is exit 4 with a different message. Scripts can trust the difference.

Run the test suites — the same gates CI runs

These are not toy tests. The Python suites drive the real engine; the cross-engine parity suites assert the Python reference and the Rust core answer identically, query by query.

terminal — python suites

$ python3 tests/test_nedb.py              # engine core
11/11 passed
$ python3 tests/test_bitemporal.py        # both time axes
28/28 passed
$ python3 tests/test_nql_predicates.py    # cross-engine parity: predicates
257 passed
$ python3 tests/test_nql_shaping.py       # cross-engine parity: shaping
157 passed
$ python3 tests/test_wallclock_as_of.py   # AS OF by datetime (new in 9.0)
25/25 passed

terminal — rust core

$ cd rust
$ cargo test -p nedb-engine --lib
639 passed
$ cargo test -p nedb-engine --tests       # integration: real subprocesses, real stores
all targets green

Why the parity suites matter: two independent implementations of one language is how drift happens. These suites run the identical battery through both engines and refuse to pass if the answers differ by a byte. They have caught five real drifts; they will catch the next one.

Benchmarks — your machine, your numbers

The published numbers name their machine and their command. Run them yourself and compare ratios, not absolutes.
terminal — embedded core
```
$ python3 bench/benchmarks.py --save
# writes ~63K/s · point reads ~1.3M/s · time-travel reads ~940K/s
# the ratio that matters: AS OF costs ~30%, not 10x — history is cheap
```
terminal — the daemon over HTTP
```
$ NEDBD_DAG=1 nedbd --data /tmp/perf &
$ python3 tests/test_dag_perf.py --n 10000 --reads 100000
# writes ~420/s · reads ~480/s · batch ~1,100/s · p99 ≤ 4.3ms
```
terminal — indexed range scans
```
$ python3 scripts/bench_index_range.py
# point lookup 137ms → 0.01ms · BETWEEN 1% 186ms → 1.1ms
```
Honest numbers policy: the repo's published figures are dated snapshots with their machine named (bench/RESULTS.md, docs/BENCH-sqlselect.md). Your absolute numbers will differ. If the RATIOS diverge wildly from the published claims, that is a finding — file it.
The full gate — daemon, wire protocols, tamper

The complete CI surface on your box: daemon HTTP, PostgreSQL wire, RESP2, encryption.
terminal — the daemon end to end
```
$ NEDBD_DAG=1 nedbd --data ./nedb-data &
$ curl http://127.0.0.1:7070/health
{"ok":true,"version":"9.0.0","engine":"dag",...}
$ curl -X POST http://127.0.0.1:7070/v1/databases/shop/query \
    -H 'Content-Type: application/json' -d '{"nql":"SELECT 1"}'
# SQL over HTTP; the FROM-form works on the same route
$ NEDBD_RESP2_PORT=6380 nedbd --data ./nedb-data &   # redis-cli compat
$ redis-cli -p 6380 EVAL 'FROM orders LIMIT 5' 0
$ python3 tests/test_pgwire.py               # real libpq client, 65 checks
```
- Source cloned — release history auditable via git tags
- Installed from all three registries — one version on one tag
- Core semantics proven — time travel, provenance, verify, tamper
- Node binding walked end to end — full surface, BigInt gotchas and all
- Both SQL dialects through the CLI — exit codes as contract
- Test suites green on your machine — parity included
- Benchmarks run by you — ratios match the claims
- Daemon + wire protocols exercised — the full CI surface

Get the source on your machine

Install from the registries — and check what actually shipped

Run the engine — write, query, time-travel, verify (Python)

The same in Node — the full napi surface

The nesql CLI — SQL from a shell, exit codes as a contract

Run the test suites — the same gates CI runs

Benchmarks — your machine, your numbers

The full gate — daemon, wire protocols, tamper