Expert Track · Phase J · 11 of 26
Above the storage engine sits the query engine. SQL text becomes AST becomes logical plan becomes optimized physical plan becomes results.
Module 57 · Expert 11 / 26 · 90 min

Query engine
internals.

The path from SELECT * FROM orders JOIN users ... to bytes returned is a compiler pipeline: parse SQL into an AST; analyze against the catalog; build a logical relational-algebra plan; optimize via rule-based rewrites + cost-based join enumeration; select physical algorithms (hash join or merge join, sequential scan or index seek); then execute — row-at-a-time via Volcano iterators (Postgres traditional), vectorized batch-at-a-time (DuckDB, MonetDB), or SIMD-accelerated columnar (ClickHouse, Snowflake, BigQuery). The 1000× performance gap between naive and expert query engines lives in these three layers. Understanding them is the specific competence for building, tuning, and operating analytical systems at scale.

// What you\'ll know by the end

  • SQL parsing + semantic analysis + logical plans
  • Rule-based + cost-based query optimization
  • Join algorithms (nested loop, hash, merge)
  • Volcano vs vectorized vs SIMD execution
§ 01 — SQL text becomes a plan becomes execution · the compiler pipeline of databases

The 1000×
gap lives
between text
and bytes.

Every database query traverses a compiler-like pipeline: SQL text is lexed into tokens, parsed into an abstract syntax tree (AST), semantically analyzed against the schema catalog, transformed into a logical relational-algebra plan, optimized via rule-based rewrites and cost-based algorithm selection, compiled into a physical execution plan, and finally executed against the storage engine to produce results. Most engineers treat the query engine as a black box — write SQL, get results. But the difference between naive and expert query engines is measured in orders of magnitude: a 1M-row aggregation runs in 10 seconds on Postgres 8.x traditional row-at-a-time execution, 300ms on Postgres 15 with JIT + parallel workers, and 30ms on DuckDB with vectorized execution — same query, same data, same hardware, 300× spread. Understanding where the time goes at each layer (parse, plan, execute) and what makes each layer fast (statistics, batching, SIMD, columnar layout) is the specific engineering competence for building and operating analytical systems. The query engine is where SQL text meets storage bytes; the engineering choices at each stage of the pipeline determine whether analytical queries take milliseconds or minutes. Postgres pioneered the modern architecture (Berkeley 1986, PostgreSQL 6.0 1996 — cost-based optimizer). MonetDB/X100 (2005) demonstrated vectorized execution (10-100× speedup on analytical workloads). Vectorwise commercialized it. DuckDB (2019) brought vectorized in-process analytics to the masses. ClickHouse (2016) proved SIMD-accelerated columnar execution scales to petabytes. Snowflake and BigQuery demonstrated distributed query execution at exabyte scale. Each represents specific engineering learning about how to compile SQL into fast execution.

// THE QUERY ENGINE PIPELINE · SQL TEXT → AST → LOGICAL PLAN → PHYSICAL PLAN → RESULTS
QUERY ENGINE · COMPILE SQL TEXT INTO EXECUTABLE PLAN SQL TEXT SELECT sum(x) FROM t WHERE... AST tokens + tree Select · Sum · Filter LOGICAL PLAN relational algebra π · σ · ⨝ · γ PHYSICAL PLAN specific algorithms HashJoin · IndexScan 1. PARSE 2. ANALYZE 3. RULE OPT 4. COST OPT EXECUTION · THREE MODELS · 300× PERFORMANCE SPREAD ROW-AT-A-TIME Volcano iterator model next() per row Postgres, SQLite 1× (baseline) VECTORIZED batch-at-a-time 1024 rows per call DuckDB, MonetDB 10-30× SIMD COLUMNAR SIMD + columnar AVX-512 · 8-16 per op ClickHouse, Snowflake 100-300× Same query · same data · same hardware · 300× spread 1M-row SUM aggregation: 10s → 300ms → 30ms depending on execution model The engineering choices at each stage determine analytical query performance
Every SQL query traverses a compiler-like pipeline. Parse: SQL text becomes tokens (lexer), tokens become AST (parser), AST is analyzed against the catalog (semantic analysis) to resolve table/column names and check types. Plan: analyzed AST becomes a logical relational-algebra tree (projections π, selections σ, joins ⨝, aggregations γ). Rule-based optimization applies deterministic rewrites (predicate pushdown, projection pruning, constant folding). Cost-based optimization uses statistics (row counts, histograms, correlations) to choose join orders and algorithms via dynamic programming (Selinger, 1979). Output: physical execution plan with specific algorithms chosen (HashJoin vs MergeJoin, SeqScan vs IndexScan). Execute: three fundamentally different models with 300× performance spread. Row-at-a-time (Volcano iterator, one row per operator call): Postgres traditional, SQLite. Vectorized (batches of ~1024 rows): DuckDB, MonetDB/X100, Vectorwise. SIMD-accelerated columnar (AVX-512, process 8-16 values per CPU instruction): ClickHouse, Snowflake, BigQuery. Same 1M-row aggregation runs in 10s, 300ms, and 30ms respectively — same hardware, same data. The engineering choices at each stage determine analytical query performance.

The specific engineering task M.57 addresses is understanding what happens at each stage of the pipeline and where performance is won or lost. The critical insight: modern query engines are compilers that turn SQL into machine-friendly execution, and the sophistication at each layer determines the performance ceiling. Parse errors are cheap and catchable; semantic errors (bad column names, type mismatches) are cheap and clear; but planning errors (wrong join order, missing predicates, ignored statistics) can produce 1000× slowdowns silently; execution model choice (row-at-a-time vs vectorized vs SIMD) provides the final 10-100× performance multiplier. Understanding when to use which type of engine — Postgres (mature, OLTP-optimized, some analytical support via JIT); DuckDB (in-process, vectorized, analytical); ClickHouse (columnar + SIMD + distributed, extreme analytical); Snowflake/BigQuery (elastic distributed analytical) — is the Expert-tier competence. Getting it wrong hits performance ceilings (Postgres analytical queries on 100M+ rows), wastes infrastructure (over-provisioned OLTP for analytical use case), or over-engineers (deploying Spark for a query DuckDB handles in-process). Modern architectures compose multiple query engines matched to workload characteristics: Postgres for transactional, DuckDB or ClickHouse for analytical, Spark for very large distributed. Each choice is a specific engineering decision with measurable consequences.

// FOUR APPROACHES TO QUERY EXECUTION · WHERE EACH FAILS OR FITS
Attempt 1: "just execute the SQL text"// no parsing · impossible
"Why compile SQL at all? Just execute the text directly, string-matching against the database." Nobody has ever built this because it\'s fundamentally impossible: SQL is a declarative language describing WHAT to compute, not HOW. SELECT sum(price) FROM orders WHERE date > '2024-01-01' describes a result, not an algorithm. The database has choices: scan the entire orders table (sequential scan) or use the date index (index scan); if the table is huge, use parallel workers; if the sum is over a filtered subset, apply the filter first. Executing text literally would mean no choices — no optimization, no algorithm selection, no performance. This is why every SQL engine (going back to System R in 1974) parses into an AST and plans execution. It\'s not optional. This attempt exists in the frame only to establish that a query engine is inherently a compiler.// FAIL MODE: SQL is declarative; execution requires choices
IMPOSSIBLE
Attempt 2: "parse + execute naively"// no optimization · 1000× slower than needed
"Parse the SQL, build a tree, walk the tree executing operations. Simple. No optimization needed." Works correctly — produces right answers — but performance is catastrophic on non-trivial queries. Specific failures: (a) bad join order — A JOIN B JOIN C could be (A JOIN B) JOIN C or A JOIN (B JOIN C); if B has 1M rows and C filters down to 100, choosing the wrong order produces 1M×1M intermediate result vs 100×A. 10000× difference. (b) no predicate pushdown — SELECT * FROM (large_view) WHERE id = 5 should filter at the storage layer; naive execution materializes the entire view then filters. 1M rows read vs 1. (c) no algorithm selection — nested-loop join for 1M×1M is 1TB of work; hash join is 100MB. (d) no batching or SIMD — process rows one at a time; hit branch mispredictions, cache misses, function-call overhead. Result: query that could run in 100ms takes 10 minutes. This is why every serious database has a query optimizer; the difference between naive and optimized execution is orders of magnitude.// FAIL MODE: no optimization = 1000× slower than possible
TOO SLOW
Attempt 3: "cost-based opt + row-at-a-time execute"// Postgres traditional · good OLTP, weak analytics
"Parse, build logical plan, apply rule-based optimizations, use statistics to choose join order (Selinger DP) and algorithms (hash vs merge vs nested loop), then execute using the Volcano iterator model — one row at a time through the operator tree." This is the mature classical architecture from System R (1979) → Postgres (1996). Works well for OLTP: point lookups (SELECT ... WHERE id = 5), small joins, transactional workloads. Postgres is the reference implementation and gold standard for correctness. Fails specifically for analytical workloads: (a) Volcano overhead — next() function call per row per operator; a query over 100M rows makes billions of virtual function calls. Interpretive overhead dominates. Measured 90%+ of CPU time in overhead, not actual work. (b) poor cache behavior — one row through many operators before the next row; each row traversal touches many code paths; instruction cache misses common. (c) no SIMD — one value per operation makes SIMD (which processes 8-16 values per instruction) impossible. (d) row-oriented storage — reads all columns even when query needs one. Result: excellent for OLTP, 10-100× slower than vectorized engines for analytical workloads. Postgres 12+ mitigates with JIT compilation and parallel workers but still lags dedicated analytical engines. This is the traditional architecture; excellent for OLTP; not the best choice for pure analytics.// FAIL MODE: row-at-a-time overhead dominates analytical workloads
GOOD FOR
OLTP
Attempt 4: cost-based opt + vectorized SIMD columnar execute// DuckDB / ClickHouse / Snowflake pattern · Expert analytics
"Parse, plan, optimize as before, but execute in batches (vectors of ~1024 rows), using columnar in-memory layout, with SIMD instructions processing multiple values per CPU cycle." The modern analytical query engine pattern from MonetDB/X100 (2005) → Vectorwise → DuckDB (2019) → ClickHouse (2016) → Snowflake/BigQuery. Specifically: (a) batch processing — operators process 1024 rows per call instead of 1. Amortizes function-call overhead across the batch. (b) columnar in-memory layout — process one column at a time; cache-friendly; SIMD-friendly. (c) SIMD instructions — AVX-512 processes 8 int64 or 16 int32 values per instruction. 10-100× speedup on tight loops. (d) hot-loop compilation — some engines (Postgres JIT, HyPer, Umbra) generate machine code for hot paths. (e) parallel execution — partition data across cores; execute in parallel; merge results. (f) distributed execution — Snowflake, BigQuery, Spark partition across many machines. Result: 1M-row aggregations in 30ms; 1B-row aggregations in seconds; 1T-row aggregations in minutes. 100-1000× faster than traditional row-at-a-time. The modern analytical query engine. Requires columnar storage or in-memory columnar buffers; different sweet spot than row-oriented OLTP engines. Composite architectures use Postgres for OLTP, DuckDB / ClickHouse / Snowflake for analytics.// FIT: batch + SIMD + columnar = analytical query engine ceiling
EXPERT
ANALYTICS
// THE COMPOSITE PATTERN

Each earlier attempt fails specifically. Just execute text is impossible (SQL is declarative). Parse + naive execute is 1000× slower than possible (no optimization). Cost-based + row-at-a-time is the mature OLTP architecture (Postgres) but hits ceilings on analytical workloads. The Expert pattern: cost-based optimization + vectorized SIMD columnar execution for analytical workloads (DuckDB, ClickHouse, Snowflake); Postgres and similar for transactional workloads; composite architectures use both matched to workload characteristics. §02 covers parsing and planning. §03 covers execution models. §04 lets you explore all three execution models across three query types.

The historical arc of query engines is specifically the story of increasingly sophisticated optimization and execution matching increasingly diverse workloads. 1970: Edgar Codd\'s relational model paper. "A Relational Model of Data for Large Shared Data Banks." Establishes relational algebra as the mathematical foundation. Turing Award (1981). 1974-1979: IBM System R. First relational database implementation at IBM Research. Introduces SQL (originally SEQUEL), the query optimizer, cost-based join ordering. 1979: Patricia Selinger\'s paper "Access Path Selection in a Relational Database Management System" (SIGMOD). The foundational cost-based optimization algorithm. Dynamic programming enumerates join orders; cost model estimates I/O + CPU. Every modern database\'s query optimizer descends from this paper. 1986: Ingres. Berkeley open-source RDBMS. Michael Stonebraker\'s project. Predecessor to Postgres. 1994: Volcano paper by Graefe. "Volcano — An Extensible and Parallel Query Evaluation System." Establishes the iterator model (row-at-a-time). Every SQL engine implemented this pattern for two decades. 1996: PostgreSQL 6.0. Modern architecture. Cost-based optimizer, extensible via user-defined functions, WAL for durability. Reference implementation of the mature relational database. 1995-2010: MySQL, Oracle, SQL Server mature. All use Volcano-style row-at-a-time execution with cost-based optimization. Different tradeoffs but same fundamental architecture. 2005: MonetDB/X100 paper. "MonetDB/X100: Hyper-Pipelining Query Execution." Establishes vectorized execution as an academic idea. Measured 10-100× speedup on analytical queries. Foundational vectorized-execution paper. 2008: Vectorwise. Commercial vectorized execution engine (later Actian). First mature vectorized commercial database. 2010: column-store revolution. Vertica (Stonebraker), C-Store paper. Columnar storage + vectorized execution. Analytical databases adopt widely. 2013: Apache Spark. Distributed query engine over Hadoop. Later adds cost-based optimizer, whole-stage codegen (2016). Democratizes distributed SQL. 2016: Apache Arrow. Columnar in-memory data format. Zero-copy data exchange between systems. Enables new architectures (compute in one system, storage in another). 2016: ClickHouse open-sourced by Yandex. Columnar + vectorized + SIMD + distributed. Analytical workloads at petabyte scale. Rapid adoption. 2019: DuckDB 0.1. "The SQLite for analytics" — in-process analytical database with vectorized execution. Rapid adoption for local analytical work; challenges the "you need a distributed system" assumption for many workloads. Andy Pavlo\'s CMU course covers DuckDB internals extensively. 2020s: Snowflake and BigQuery mature. Distributed elastic analytical databases. Separate compute from storage. Cost-based optimization with sophisticated statistics. Vectorized SIMD execution. Petabyte-scale queries in minutes. 2023+: DuckDB adoption exploding. Wide use in data engineering, analytics notebooks, embedded analytics. MotherDuck commercializes cloud DuckDB. Umbra (TU Munich) demonstrates compilation-based execution matching hand-tuned C++ (Neumann, 2011+). The historical arc explains why "we use Postgres" turned into "we use Postgres for OLTP, DuckDB or ClickHouse for analytics, Snowflake for distributed analytical scale, Spark for very large ETL." Different workloads need different query engines; understanding which fits which is Expert-tier competence.

The 1000× gap between naive and expert query engines lives in three layers: parse, plan, execute. Get all three right and analytical queries run in milliseconds. Get any one wrong and they take minutes.
§ 02 — Parsing + logical planning + cost-based optimization · SQL to physical plan

Text becomes
tree. Tree becomes
relational algebra.
Algebra becomes plan.

The path from SQL text to physical execution plan is a multi-stage compiler: lexer produces tokens, parser produces AST, semantic analyzer resolves identifiers against the catalog and validates types, plan builder produces a logical relational-algebra tree, rule-based optimizer applies deterministic rewrites, cost-based optimizer chooses join orders and algorithms using statistics, and code generator (in modern engines) produces the physical plan. Each stage has specific engineering considerations: parsers must handle ambiguous grammar (SQL is context-sensitive in some dialects); semantic analysis must resolve identifiers in the right scoping order; logical optimization applies dozens of algebraic rewrites; cost-based optimization uses Selinger dynamic programming over join graphs; statistics collection is itself a substantial engineering task (histograms, correlations, distinct counts). Understanding these stages precisely means understanding where planning errors originate and how they affect execution performance.

// PARSE + ANALYZE + LOGICAL PLAN + COST-BASED OPT · SPECIFIC MECHANISMS

SQL COMPILATION PIPELINE · TEXT → TOKENS → AST → LOGICAL → PHYSICAL INPUT SELECT u.name, sum(o.total) FROM users u JOIN orders o ON u.id=o.uid WHERE u.country='US' GROUP BY u.name; 1. LEXER tokenize SELECT, IDENT("u"), 2. PARSER AST tree Select(Project, From, Where) 3. SEMANTIC ANALYZE catalog resolution u.name → users.name (VARCHAR) 4. LOGICAL PLAN relational algebra γ(⨝(σ(users), orders)) LOGICAL PLAN TREE (relational algebra) γ (GroupBy u.name) π (Project name, sum) ⨝ (Join u.id = o.uid) σ (users WHERE country=US) Scan(orders) → 5. cost-based opt: choose HashJoin vs MergeJoin · index vs scan · reorder joins
The specific compilation stages. Lexer: turns SQL text into a stream of tokens (keywords, identifiers, operators, literals). Handles whitespace, comments, quoting. Well-understood problem. Parser: turns tokens into an abstract syntax tree (AST) using a grammar. SQL grammars are large (hundreds of production rules) and dialect-specific. Postgres uses a hand-written recursive descent parser + Bison generator; DuckDB uses PGQ (Postgres query parser); many use ANTLR. Errors here are typically clear ("syntax error near \'FROM\'"). Semantic analyzer: resolves identifiers against the catalog. "u.name" → users.name (VARCHAR). Validates types (can\'t sum a string). Checks scoping rules (aggregate over grouped columns only). Catches ~90% of the "why doesn\'t this work" errors before execution. Logical plan: relational-algebra tree using operators π (projection), σ (selection/filter), ⨝ (join), γ (aggregation/group-by), ρ (rename). Independent of specific execution strategy. Optimizations at this layer are algebraic rewrites (e.g., pushing selections below joins). Physical plan (not shown fully): rule-based optimization applies deterministic rewrites; cost-based optimization uses Selinger DP over join orders + statistics to select algorithms; output is a physical plan with specific operators (HashJoin, MergeJoin, IndexScan, SeqScan) chosen. This physical plan is what actually executes.
i
Lexer + parser.

Lexer: SQL text → tokens (keywords, identifiers, literals, operators). Parser: tokens → AST via grammar (LALR, LL, or recursive descent). Handles SQL\'s ambiguous grammar (e.g., SELECT a FROM b vs SELECT a FROM b, c). Errors here have clear messages; well-understood problem.

ii
Semantic analysis.

Resolves identifiers against catalog (table name → OID, column name → position + type). Validates types (can\'t sum(text_column)). Enforces SQL scoping rules (aggregate refs must be in GROUP BY). Catches most user errors before execution.

iii
Rule-based optimization.

Deterministic algebraic rewrites: predicate pushdown (push filters below joins), projection pruning (drop unused columns early), constant folding (2+3 → 5 at compile time), column pruning (only read needed columns), join reordering hints (obviously beneficial cases). Fast; deterministic; catches most gains without statistics.

iv
Cost-based optimization.

Uses statistics (row counts, histograms, correlations) to estimate cost of alternative plans. Selinger dynamic programming enumerates left-deep join orders (n! possibilities → O(n · 2^n) DP). Cost model estimates I/O + CPU. Foundation of every modern query optimizer since 1979.

v
Statistics collection.

Row counts per table, min/max per column, histograms (equi-width or equi-height), distinct counts (HyperLogLog), correlations between columns, most-common values. Updated via ANALYZE (Postgres) or auto-vacuum. Stale statistics cause bad plans.

vi
Join algorithm selection.

Nested loop: for each row in outer, scan inner. O(n·m). Good for small tables or when index available on inner. Hash join: build hash on smaller side, probe with larger. O(n+m). Good for unsorted equi-joins. Merge join: sort both sides, merge. O(n log n). Good when sorted already or index-supported. Cost model picks between them.

The cost-based optimization (iv) is the specific engineering mechanism that separates a query engine that runs your queries fast from one that doesn\'t. Consider the specific problem: given a query with 5 tables joined, how do you choose the join order? There are 5! = 120 possible left-deep orderings; considering bushy trees gives Catalan(5) × 5! ≈ 1600 possibilities. Enumerating all is infeasible for 10+ tables. Selinger\'s 1979 paper solved this with dynamic programming: build up optimal plans for pairs of tables, then triples, then quadruples, using the optimal 2-way plan as a building block for the 3-way plan. Complexity: O(n·2^n) where n is number of tables. Practical up to ~10-15 tables. Beyond that, engines use heuristics (greedy, genetic algorithms, or reinforcement learning). The cost estimate at each step uses statistics: row count of the base tables, selectivity of filters (fraction of rows passing), join selectivity (fraction of rows joining), size of result. Cost formulas typically combine I/O cost (pages read from disk) and CPU cost (operations per row). Real-world impact: for a 5-table join, choosing the right order vs random can be a 10000× performance difference. When statistics are stale (table has grown 100× since last ANALYZE), the optimizer\'s cost estimates are wrong; it picks the wrong join order; queries slow down by orders of magnitude. This is the specific mechanism behind "we changed nothing and suddenly this query is 100× slower" incidents — statistics went stale. The specific engineering: maintain fresh statistics; understand your optimizer\'s cost model; use EXPLAIN ANALYZE (Postgres) or equivalent to see chosen plans; use hints or query rewrites when the optimizer picks poorly.

The join algorithm selection (vi) is the specific mechanism where cost-based optimization meets execution strategy. Consider a join between users (10M rows) and orders (100M rows) on user_id: (a) Nested loop: for each of 100M orders, scan 10M users looking for matching user_id. Cost: 100M × 10M = 10^15 comparisons. Infeasible. UNLESS an index exists on users.id, in which case: 100M lookups × O(log 10M) each = ~2.5 billion operations. Still large but feasible if index fits in memory. (b) Hash join: build a hash table on the smaller table (10M users, ~1GB in memory). Then scan 100M orders once, probing hash table per row. Cost: 10M hash inserts + 100M hash lookups = ~110M operations. Feasible in seconds. Works even without indexes. Requires enough memory for hash table. (c) Merge join: sort both tables on user_id, then merge. Cost: 10M · log(10M) + 100M · log(100M) ≈ 2.7 billion comparisons for sorting + linear merge. Feasible. Beats hash join when both sides already sorted (e.g., from index scans). The specific choice depends on: whether indexes exist, whether data is already sorted, whether the smaller side fits in memory, whether the query needs sorted output. Real-world: Postgres, DuckDB, ClickHouse all implement all three; cost-based optimizer picks per join based on statistics. Getting this wrong: choosing nested loop for a 100M × 10M join without index = query never completes; choosing hash when smaller side is 100GB = out of memory. Understanding the tradeoffs precisely — and being able to read query plans to see which algorithm was chosen — is Expert-tier competence. EXPLAIN ANALYZE in Postgres shows: Hash Join (cost=1234..5678 rows=100000 width=48) — the operator, cost estimate, row estimate, row width. Cross-checking with actual runtime reveals estimation errors; large errors indicate stale statistics or model limitations.

Parse. Analyze. Optimize with statistics. Choose join algorithm. Each stage has specific engineering. Errors at planning stage produce 1000× slowdowns silently, hidden until the load hits.
§ 03 — Execution models · Volcano vs vectorized vs SIMD columnar

One row at a time.
Or a batch.
Or a SIMD instruction.

The physical execution plan runs against the storage engine to produce results — but how it runs makes a 100-300× performance difference for analytical queries. Three fundamentally different execution models: the Volcano iterator (row-at-a-time), vectorized (batch-at-a-time), and SIMD-accelerated columnar. Volcano is the classical model from Graefe\'s 1994 paper; every SQL engine used it for two decades. Vectorized execution (MonetDB/X100, 2005) processes batches of rows through operators; amortizes function-call overhead; enables SIMD. SIMD-accelerated columnar (ClickHouse, Snowflake) processes columns of values with SIMD instructions (AVX-512 processes 8 int64 values per instruction). Understanding when each model fits (row-at-a-time for OLTP; vectorized for analytical; SIMD for extreme analytical) is the specific competence for choosing and building query engines.

// THREE EXECUTION MODELS · SPECIFIC MECHANISMS AND TRADEOFFS

EXECUTION MODELS · ROW-AT-A-TIME · VECTORIZED · SIMD COLUMNAR ROW-AT-A-TIME · VOLCANO "one row per next() call" Aggregate.next() next() Filter.next() next() Scan.next() FOR EACH ROW: Scan.next() → row Filter.next() → keep? Aggregate.accumulate() OVERHEAD PROBLEMS ✗ Virtual function call/row ✗ Branch misprediction ✗ Instruction cache miss ✗ No SIMD possible ✗ Row layout: extra cols read 90%+ overhead on analytics VECTORIZED · BATCH-AT-A-TIME "1024 rows per operator call" Aggregate.exec(batch) 1024 rows Filter.exec(batch) 1024 rows Scan.exec(batch) EACH exec() PROCESSES: v[0] 42 v[1] 17 v[2] 99 ... v[1023] 55 1024-row batch, one column BENEFITS ✓ 1 function call per 1024 rows ✓ Tight inner loop (cache-friendly) ✓ SIMD-friendly layout ✓ Branch prediction stable 10-30× speedup vs Volcano SIMD COLUMNAR "8-16 values per CPU instruction" AVX-512: 512-bit register SCALAR (1 per cycle): 42 SIMD (8 per cycle): 42 17 99 31 55 78 21 64 8× int64 values per op SUM(col) with AVX-512: acc = _mm512_setzero_si512(); for (i=0; i<n; i+=8) { acc = _mm512_add_epi64(acc, load(col+i)); Combined with vectorized batching + columnar storage layout EXTREME ANALYTICS ✓ 8-16 values per instruction ✓ ClickHouse, Snowflake, BigQuery 100-300× vs Volcano
Three fundamentally different execution models. Left — Volcano iterator: operators form a tree; each operator implements next() returning one row; parent pulls rows from children one at a time. Simple, general, correct — but the virtual function call per row costs ~5-20ns; branch mispredictions frequent; instruction cache misses common; SIMD impossible (only one value per call). On a query over 100M rows, that\'s billions of function calls; 90%+ CPU spent on overhead, not work. Middle — Vectorized: operators process batches of ~1024 rows per exec(batch) call. One function call amortized across 1024 rows. Tight inner loop over the batch is cache-friendly, branch-predictable, SIMD-friendly. Same operator tree but different data flow. MonetDB/X100 (2005) established the pattern; DuckDB (2019) brought it to the masses. 10-30× speedup over Volcano on analytical queries. Right — SIMD columnar: same batching plus explicit SIMD instructions (AVX-512 processes 8 int64 or 16 int32 values per instruction). Columnar in-memory layout enables one-column-at-a-time processing that\'s directly SIMD-compatible. ClickHouse, Snowflake, BigQuery use this pattern. Additional 3-10× speedup over pure vectorized. Combined 100-300× vs Volcano. Requires columnar storage or in-memory columnar buffers.
i
Volcano iterator.

Graefe 1994. Each operator: open(), next() returns one row, close(). Parents pull from children. Simple, general, correct. Used by Postgres, SQLite, MySQL. Perfect for OLTP (small result sets, low overhead per query). Terrible for analytics (billions of function calls).

ii
Vectorized execution.

MonetDB/X100 2005. Operators process batches (typically 1024 or 2048 rows). One function call amortized across the batch. Tight inner loop is cache-friendly and SIMD-friendly. DuckDB, Vectorwise, MonetDB use this. 10-30× speedup on analytical workloads.

iii
SIMD acceleration.

Modern CPUs: AVX-2 (256-bit, 4 int64 per op), AVX-512 (512-bit, 8 int64 per op). Applied to hot loops in vectorized engines. ClickHouse extensive SIMD use; DuckDB adds SIMD to hot paths. Requires columnar layout. 3-10× additional speedup on tight loops.

iv
Compiled execution.

Instead of interpreting a plan, compile it to machine code (LLVM). HyPer (TU Munich), Umbra, Postgres JIT. Generates tight loops equivalent to hand-written C++. Combines with vectorization for extreme performance. Neumann\'s HyPer/Umbra papers foundational.

v
Parallel execution.

Partition data across cores; execute in parallel; merge results. Postgres parallel query (2016), DuckDB parallel by default, ClickHouse aggressive parallelism. Scales with core count (up to memory bandwidth ceiling). N× speedup for embarrassingly parallel operations.

vi
Distributed execution.

Partition data across many machines; execute distributed plan; shuffle for joins. Snowflake, BigQuery, Spark, ClickHouse (distributed mode). Adds network cost but scales beyond single-machine limits. Petabyte-scale queries in minutes.

The vectorized execution (ii) is the specific engineering mechanism that turned analytical query engines from "slow" to "fast enough for interactive use." The specific breakthrough from MonetDB/X100 (Boncz, Zukowski, Nes 2005 CIDR paper): instead of the operator tree processing one row at a time, each operator processes a batch of rows. Concretely: Filter.next() returning one row becomes Filter.exec(batch) that filters 1024 rows at once. The inner loop is a tight for (i=0; i<1024; i++) if (predicate(batch[i])) output[j++] = batch[i]. Modern compilers vectorize this loop automatically with SIMD; branch predictor learns the pattern; L1 cache handles the working set; no virtual function calls per row. Measured impact from the original paper: 10-100× speedup on TPC-H queries vs Volcano-based DBMSs. The specific tradeoffs: (a) BATCH SIZE MATTERS — too small (32 rows): overhead per batch dominates; too large (1M rows): batch doesn\'t fit in L1 cache. Sweet spot: 1024-2048 rows, ~50-100KB per column batch, fits L1. (b) COLUMNAR LAYOUT NEEDED — vectorized execution processes one column at a time (SIMD-friendly); requires columnar storage OR in-memory columnar buffers. Row-major storage forces gather operations (slow). (c) OPERATORS MUST BE VECTORIZABLE — some operators fit vectorized model well (filter, projection, arithmetic, aggregate); some awkwardly (sort, hash join build, complex UDFs). Modern engines vectorize what they can; fall back to iterator for what they can\'t. Real-world: DuckDB is essentially "SQLite for analytics" — in-process, single-node, vectorized, columnar in-memory. Adoption exploding in data engineering, notebooks, embedded analytics. ClickHouse is server-based columnar-storage-plus-vectorized-execution with additional SIMD hand-tuning. Snowflake and BigQuery apply vectorized SIMD execution at petabyte scale via distributed architectures. The specific engineering: for analytical workloads, choose a vectorized engine; the 10-100× performance improvement over Volcano-based engines is measurable and reproducible. Postgres for OLTP; DuckDB/ClickHouse/Snowflake for analytics; understanding which fits which is Expert-tier competence.

The SIMD acceleration (iii) is the specific mechanism that pushes vectorized engines from "fast" to "hardware-limited." SIMD (Single Instruction, Multiple Data) instructions process multiple values per CPU cycle. AVX-2 (introduced 2013): 256-bit registers, 4 int64 or 8 int32 or 16 int16 values per operation. AVX-512 (introduced 2017, wider deployment 2020s): 512-bit registers, 8 int64 or 16 int32 values per operation. Applied to hot loops in vectorized engines: SUM(column) becomes 8 additions per instruction; column1 + column2 becomes 8 additions per instruction; filter predicates like column > 42 become 8 comparisons per instruction. Real code from ClickHouse: __m512i acc = _mm512_setzero_si512(); for (i=0; i<n; i+=8) acc = _mm512_add_epi64(acc, _mm512_loadu_si512(col+i)); — 8 int64 additions per iteration. Combined with vectorized batching (i.e., 8 SIMD operations per batch = 8192 rows processed per outer function call), the amortization is enormous. Measured on TPC-H style queries: 3-10× additional speedup over pure vectorized without SIMD. Combined with vectorization: 100-300× over Volcano. The specific requirements: (a) COLUMNAR STORAGE OR BUFFERS — SIMD requires contiguous same-type values; columnar layout is natural fit. Row-major requires gather, defeats SIMD. (b) SIMD-FRIENDLY DATA TYPES — fixed-width numeric types are ideal; strings, complex types are harder. Some SIMD algorithms for strings exist (SIMD comparison for equality, SIMD parsing) but complex. (c) HARDWARE AVAILABILITY — AVX-512 in Intel Xeon Skylake+ (2017+), AMD EPYC Zen4+ (2022+), ARM SVE. Not universal; check target hardware. (d) COMPILER OR HAND-TUNED — modern compilers auto-vectorize many loops (with -O3 -mavx512f); complex algorithms often benefit from intrinsics (_mm512_add_epi64) or hand-written assembly. ClickHouse extensive hand-tuning; DuckDB balance of auto-vectorization and targeted intrinsics. Real-world usage: aggregations (SUM, COUNT, AVG, MIN, MAX), filters (column comparisons), arithmetic (column1 + column2 with scalars), hash computations. Combined effect: analytical query on 1B rows in seconds instead of minutes. The specific rule: SIMD is worth the engineering when the workload is analytical (large scans over columns) and columnar layout is possible; not worth it for OLTP (small result sets don\'t benefit from batching, let alone SIMD). Understanding when SIMD-accelerated engines apply is the specific competence for choosing modern analytical infrastructure.

Row-at-a-time is a compiler you can\'t optimize past. Vectorized amortizes overhead across batches. SIMD pushes to hardware limits. 300× speedup lives in these choices.
§ 04 — Query execution explorer

Three execution models.
Three query types.

Below: each of three execution models (Row-at-a-time Volcano · Vectorized batch · SIMD columnar) evaluated against three query types (OLTP point lookup · Analytical aggregation · Complex multi-join). Watch how each execution model fits or fails each query — the diagonals show exactly which model produces the best result for which specific query class, and the off-diagonals show where each model is over-engineered, under-provisioned, or a fundamental mismatch. This is the matrix Expert engineers implicitly consult when choosing analytical infrastructure.

QE.SIM // m.57 lab
Query type →
// EXECUTION BEHAVIOR · under current query type
// METRICS · PERFORMANCE / OPERATIONAL PROFILE
Latency-
Throughput-
CPU efficiency-
Cache friendly-
Ecosystem-
Overall fit-
// VERDICT
Loading...
...
§ 05 — Where query engine choices decay

Every slow query
is a plan
or a model failure.

The failure modes of query engines are the specific mechanisms by which "the database is slow" turns into "we\'ve migrated three times and nothing works" or "this query used to be fast, now it takes forever." Each of these anti-patterns is a real production pattern; Expert engineers avoid them by matching execution model to workload, maintaining fresh statistics, understanding query plans, and choosing engines by workload characteristics. Recognizing them saves months of "why is this slow" debugging.

// FIVE QUERY ENGINE ANTI-PATTERNS

i
The row-at-a-time for analytical workloads
"We run analytical dashboards on Postgres. Aggregations over 100M rows take 30 seconds; our users complain constantly. Every dashboard reload waits half a minute. We\'ve maxed out the CPU and IO. We\'ve indexed everything. Nothing helps."

Postgres uses the Volcano iterator model — row-at-a-time execution. For 100M-row aggregations, that\'s 100M virtual function calls per operator, per row. Instructions cache misses, branch mispredictions, and function-call overhead dominate; 90%+ of CPU time spent on execution mechanics, not actual computation. Indexes don\'t help for full aggregations. Adding hardware doesn\'t help proportionally (the overhead is per-row, not per-byte). The fix: use a vectorized engine for analytical workloads. Options: (a) DUCKDB — in-process; run alongside Postgres; connect via Foreign Data Wrapper or export data. Excellent for interactive analytics on 1B-row tables from a single machine. Same 100M-row aggregation: 30s → 300ms. (b) CLICKHOUSE — server-based, columnar, distributed. Excellent for larger analytical workloads. (c) DATA WAREHOUSE — Snowflake, BigQuery, Redshift for distributed analytical scale. Each is a specific fit for specific workload size and operational preference. Postgres 12+ mitigates via JIT and parallel workers; helps 3-10× but doesn\'t reach vectorized-engine performance. The general principle: match execution model to workload; row-at-a-time for OLTP; vectorized for analytics. Anti-pattern §05.i.

ii
The vectorized for OLTP point lookups
"We built our transactional API on ClickHouse because it\'s fast. Point lookups (SELECT ... WHERE id=42) take 50-100ms each — much slower than Postgres would take (1-2ms). Batch overhead dominates when we only need one row."

Vectorized engines optimize for batches of ~1024 rows. When you need one row, the batch overhead dominates. Setup cost per query is fixed (parsing, planning, execution setup); for a large batch it amortizes; for one row it dominates. Additionally: columnar storage requires reading many columns to reconstruct one row. Optimization for the wrong workload. ClickHouse explicitly documents this — "not designed for OLTP workloads." Snowflake documents similar — analytical warehouse, not transactional store. The fix: use OLTP-optimized engine for point lookups. Postgres, MySQL, SQLite: 1-2ms point lookups on well-indexed tables. Use vectorized engine only for analytical queries; keep transactional workloads on OLTP engine. Composite architectures: Postgres for OLTP, ClickHouse or DuckDB for analytics. The general principle: match engine to workload; vectorized engines pay the batch overhead cost that OLTP workloads can\'t amortize. Anti-pattern §05.ii.

iii
The stale statistics silently killing plans
"This query used to run in 500ms; now it takes 2 minutes. We didn\'t change anything — no code changes, no schema changes. But statistics show the query is now doing a nested loop join over 100M rows instead of a hash join."

The cost-based optimizer relies on statistics (row counts, histograms, distinct counts) to choose join algorithms and join order. Statistics get stale as data changes; the optimizer\'s decisions become wrong. Common trigger: a table grew from 1000 rows to 10M rows since last ANALYZE; optimizer thinks it\'s still small; picks nested loop; nested loop is 10M×10M = 10^14 operations. Query stalls. Result: "we changed nothing and this query is now 1000× slower." Standard incident pattern; happens frequently in production. The fix: (a) MAINTAIN FRESH STATISTICS — Postgres auto-vacuum runs ANALYZE periodically but not aggressively enough for rapidly-growing tables. Trigger manual ANALYZE on large tables after bulk loads. (b) MONITOR PLAN CHANGES — track query plans; alert on unexpected plan changes. Postgres extension pg_stat_statements helps. (c) USE QUERY HINTS OR PLAN GUIDES for critical queries where you need deterministic plans. Postgres has pg_hint_plan extension; other DBMSs have native hints. (d) FIX STATISTICS TARGETS — ALTER TABLE ... SET STATISTICS TARGET for columns with skewed distributions. (e) USE EXPLAIN ANALYZE routinely to see chosen plans. The general principle: statistics must be fresh for cost-based optimization to work; stale statistics silently produce catastrophic plans; monitoring plan changes is Expert-tier operational discipline. Anti-pattern §05.iii.

iv
The Cartesian product from missing predicates
"This query has been running for 6 hours. It\'s a 3-table join but the WHERE clause forgot to include the join predicate for one pair. Now we have a Cartesian product: 10M × 10M × 10M = 10^21 rows being processed."

SQL is declarative; missing join predicates produce Cartesian products silently. The query is syntactically valid; the optimizer produces a plan; execution starts; only after hours (or never) do you realize the plan involves an infeasible amount of work. Standard cause: complex query with many tables; developer forgets one JOIN condition; result set explodes. Real-world: 3 tables × 10M rows each without proper join = 10^21 row combinations. Even at 1M ops/sec, that\'s 3×10^7 years to complete. Query occupies infrastructure indefinitely. The fix: (a) REVIEW QUERY PLANS BEFORE PRODUCTION — EXPLAIN shows the plan; if it says "Cartesian" or "Nested Loop over 10^18 rows," something is wrong. (b) SET QUERY TIMEOUTS — statement_timeout in Postgres kills queries running past a threshold. Prevents indefinite occupation. (c) ROW-LIMIT ESTIMATES — the optimizer\'s row estimate warns of Cartesian products before execution. (d) STATIC ANALYSIS — some tools (Great Expectations, dbt) catch missing join predicates in review. (e) EDUCATE DEVELOPERS on reading query plans. The general principle: Cartesian products from missing predicates are the most common "query never completes" bug; query plans reveal them before execution; monitoring and timeouts prevent indefinite resource occupation. Anti-pattern §05.iv.

v
The optimizer refuses to use the good plan
"The optimizer keeps choosing a full table scan even though our index is perfect for this query. We\'ve tried everything — rebuilt the index, run ANALYZE, checked statistics. Nothing works. The plan is still wrong."

Cost-based optimizers work by estimating costs and comparing. Sometimes the estimates are wrong (statistics wrong, distributions non-uniform, correlations missed) OR the cost model has systematic biases (e.g., random_page_cost too high in Postgres, making index scans look expensive) that make the "right" plan look worse than the "wrong" one. When the optimizer picks poorly and you can\'t fix statistics, the remaining options are query hints or query rewrites. The fix: (a) EXTRACT PLAN with EXPLAIN ANALYZE; compare estimated vs actual rows. Large discrepancy = statistics problem or model limitation. (b) FIX STATISTICS if that\'s the cause — ALTER TABLE ... SET STATISTICS TARGET, create extended statistics for correlated columns. (c) FIX COST MODEL — tune random_page_cost, cpu_tuple_cost, etc. for your hardware. Defaults assume magnetic disks; SSDs need different values. (d) USE HINTS if statistics/model can\'t be fixed — pg_hint_plan extension in Postgres, native hints in Oracle/SQL Server. Last resort. (e) REWRITE QUERY — sometimes minor rewrites (adding a redundant predicate, using CTE with materialization hint, breaking a subquery out) produce better plans. (f) UPGRADE OPTIMIZER — newer Postgres, DuckDB, ClickHouse versions have better planners. The general principle: query optimizers are heuristic; sometimes wrong; understanding when they\'re wrong and using appropriate escapes (hints, rewrites, statistics tuning) is Expert-tier operational competence. Anti-pattern §05.v.

The composite pattern across all five is that query engine failure modes have specific causes and specific fixes. Row-at-a-time on analytics hits Volcano overhead ceilings; vectorized on OLTP pays unnecessary batch overhead; stale statistics silently corrupt plans; missing predicates produce Cartesian products; optimizer sometimes picks wrong even with fresh statistics. Each anti-pattern reflects a specific engineering understanding gap that Expert-tier competence addresses by: (a) matching execution model to workload (Volcano for OLTP; vectorized for analytics); (b) maintaining fresh statistics as operational discipline; (c) reading query plans routinely; (d) knowing when to use hints or rewrites; (e) monitoring plan changes over time. Getting query engine choices right is the specific engineering discipline that prevents the "why is our database slow" investigation that consumes months of debugging effort.

Every slow query is a plan failure or a model mismatch. Fresh statistics, right execution model, monitored plans, understood cost models. The composite discipline for analytical infrastructure.
§ 06 — Eight words for the query engine conversation

Vocabulary,
for the compiler case.

The terms that show up in every query planner discussion, every "why is this slow" investigation, every choice between analytical engines.

Query Plan
/ˈkwɪəri plæn/
The concrete tree of operators (Scan, Filter, Join, Aggregate) with specific algorithms chosen that will execute a query. Logical plans use relational algebra (independent of algorithms); physical plans specify the algorithms (HashJoin vs MergeJoin, IndexScan vs SeqScan). Viewable via EXPLAIN.
AST
/eɪ-ɛs-tiː/
Abstract Syntax Tree: the parsed representation of SQL text as a tree structure. Root is the top-level statement (Select, Insert, etc.); children are clauses (Where, From, GroupBy); leaves are identifiers and literals. Basis for semantic analysis and logical planning.
Cost-Based Optimizer
/kɒst-beɪst/
Query optimizer that uses statistics to estimate costs of alternative plans and picks the cheapest. Selinger 1979 established the pattern. Uses dynamic programming for join ordering. Every modern database has one. Requires fresh statistics to work well.
Cardinality
/ˌkɑːdɪˈnælɪti/
The number of rows in a result (or intermediate result). Cost-based optimization depends on estimating cardinality accurately. Wrong estimates → wrong plans. Cardinality estimation is one of the hardest problems in query optimization; still an active research area.
Vectorized Execution
/ˈvɛktəraɪzd/
Processing batches of ~1024 rows per operator call instead of one row at a time. Amortizes function call overhead; enables SIMD; cache-friendly. DuckDB, MonetDB, ClickHouse, Snowflake use this. 10-30× speedup over Volcano on analytical workloads.
Hash Join
/hæʃ dʒɔɪn/
Join algorithm that builds a hash table on the smaller side, then probes with the larger side. O(n+m) — one pass through each side. Preferred for unsorted equi-joins when smaller side fits in memory. Alternative to nested loop (small inputs) or merge join (sorted inputs).
Predicate Pushdown
/ˈprɛdɪkət/
Optimization that moves filters as close to the storage layer as possible. SELECT * FROM (view) WHERE id=5 should filter at storage, not materialize the view first. Rule-based optimization. Universal in modern engines. Enables index usage and reduces data movement.
Materialized View
/məˈtɪəriəlaɪzd vjuː/
A view whose result is stored (materialized) rather than computed on each query. Trades storage + refresh cost for query time. Standard optimization for expensive aggregations. Postgres, ClickHouse, Snowflake support. Incremental refresh for large tables.
§ 07 — Knowledge check

Five questions.
The compiler intuition.

Test the query engine understanding. Click an answer; explanation drops in instantly.

QUESTION 1 OF 5
Loading question...
Score: 0 / 5
5 / 5

Query engines earned.

Perfect. Parse-plan-execute pipeline, cost-based optimization with fresh statistics, vectorized SIMD for analytics — the specific engineering discipline. Next: M.58.

§ 08 — The recap

Three ideas to
carry forward.

The composite understanding that turns "the database is slow" into "we chose our engine based on workload measurements and know exactly which layer is the bottleneck."

i

The pipeline is a compiler

SQL text → tokens → AST → semantic analysis → logical plan → optimized physical plan → execution. Each stage has specific engineering. Parse errors are cheap; planning errors are silent and produce 1000× slowdowns; execution model choice provides the final 10-100× multiplier. Understanding all three layers is the specific competence.

ii

Three execution models, three regimes

Row-at-a-time Volcano for OLTP (Postgres, SQLite, MySQL — mature, correct, sufficient). Vectorized batch for analytics (DuckDB, MonetDB, Vectorwise — 10-30× faster than Volcano). SIMD columnar for extreme analytics (ClickHouse, Snowflake, BigQuery — 100-300× faster than Volcano). Match engine to workload; compose across a system.

iii

Statistics are operational discipline

Cost-based optimization depends on fresh statistics. Stale statistics silently produce catastrophic plans (nested loop instead of hash join = 1000× slowdown). Run ANALYZE aggressively on rapidly-growing tables. Monitor plan changes. Use EXPLAIN routinely. Statistics maintenance is Expert-tier operational competence for any cost-based engine.

↓ UP NEXT · PHASE J CONTINUES

M.58 — Vector
databases & ANN.

The next Expert module. Beyond relational queries, modern AI workloads require nearest-neighbor search over high-dimensional embeddings — vector databases and approximate nearest-neighbor (ANN) algorithms. HNSW, IVF, product quantization. Pinecone, Weaviate, Milvus, pgvector.

Continue to Module 58 →