NEDB Docs / Core concepts / The FROM-form in Depth

The extended FROM-form is neSQL's other statement shape — the form that carries the full clause set natively, and the one embedded cores execute. It began as NQL, the project's first query language; today it is folded into neSQL as an equal form, not a separate dialect (see the language overview).

Clause grammar

FROM <collection>
  [ AS OF <seq> ]                            transaction time
  [ VALID AS OF "<date>" ]                   valid time
  [ WHERE <predicate> ]                      full boolean predicate
  [ SEARCH "<text>" ]                        full-text search
  [ TRAVERSE <relation> ]                    graph traversal
  [ TRACE caused_by [REVERSE] ]              causal provenance
  [ GROUP BY <field> [COUNT|SUM f|AVG f|MIN f|MAX f] ]
  [ COUNT | SUM f | AVG f | MIN f | MAX f ]  whole-result aggregate
  [ HAVING <predicate> ]                     filters the AGGREGATED rows
  [ ORDER BY <field> [ASC|DESC] (, ...) ]
  [ LIMIT <n> ] [ OFFSET <n> ]

Clauses are evaluated in SQL's order — FROM → WHERE → GROUP BY → HAVING → ORDER BY → OFFSET → LIMIT — whatever order you write them. Before 3.3.0 this was wrong in ways that lied: LIMIT truncated an aggregate's input, ORDER BY ran before grouping (so sorting on count did nothing), and VALID AS OF ran after LIMIT in the Python engine.

Predicates

WHERE takes a full boolean expression. AND binds tighter than OR; parentheses nest to any depth.

ComparisonNotes
= != < <= > >=
IN (…) / NOT IN (…)set membership
BETWEEN a AND binclusive both ends, as in SQL
LIKE / NOT LIKE / ILIKE% any run, _ any one char; ILIKE case-insensitive
IS NULL / IS NOT NULLmatches absent and explicitly-null
Three-valued logic. An ordering comparison against a missing or null field is never true — WHERE fee < 5 will not return a row with no fee. LIKE is false in both polarities, so a null row appears in neither LIKE nor NOT LIKE. = and != do operate on null: WHERE fee = NULL selects absent-or-null rows. Use IS NULL to test presence explicitly.

Aggregates

GROUP BY rows carry the group key, count, and one named aggregate (sum_fee, max_price…). The aggregate only considers rows whose target field is numeric; a group of 5 where 2 carry numeric price reports count: 5 and averages over 2. No numeric input aggregates to null, never 0. Integer inputs stay in 64-bit integers — a sum over satoshi amounts above 2^53 is exact; AVG is always fractional.

Drop the GROUP BY for a whole-result aggregate (exactly one row); COUNT of an empty result is one row holding 0, SUM of an empty result is null. HAVING runs through the same evaluator as WHERE — the whole predicate surface, not a lesser copy.

Indexes and the planner

=, IN, BETWEEN and the one-sided inequalities are served from a sorted index when one covers the field. Measured on 20,000 rows (scripts/bench_index_range.py): a point lookup goes 137 ms → 0.01 ms; a 1%-selective BETWEEN 186 ms → 1.1 ms. The planner asks each candidate index how many rows its range covers and takes the narrowest; same-field bounds are merged.

An index only narrows candidates — the full predicate is re-evaluated on whatever comes back, so the answer never depends on an index existing. Three cases deliberately decline it:

  • IS NULL — absent fields are not in the index; a scan would return the exact complement of the answer.
  • Anything under OR/NOT — a disjunct does not constrain the result set.
  • AS OF — the index holds current versions only; it cannot answer historical queries.

Unknown clauses are errors

A clause the engine does not implement is rejected with the offending token (HTTP 400) — never silently skipped. Before 3.3.0 a misspelled ORDRE BY fee returned unsorted rows with HTTP 200; the engine answered a different query than the one asked and said nothing. That failure class is closed.

Cross-engine parity

NQL has two independent implementations — the Python reference and the Rust engine — and two parity suites (tests/test_nql_predicates.py, tests/test_nql_shaping.py) run the same battery through both and assert identical answers. They drifted in five places before the gate existed; the gate exists so they cannot drift again.