Module 37 / 46 · Phase G — Microservices in Practice · 50 min

Polyglot
persistence.

One database served every workload adequately. None served any workload optimally. The pattern that lets each bounded context pick the data store its work actually needs — and the coordination costs of running six databases instead of one.

// What you'll know by the end

  • Why workloads have shapes, not preferences
  • Six store types and their sweet spots
  • CDC as the glue that makes polyglot honest
  • The coordination cost and when to pay it
§ 01 — Workloads have shapes

One database.
Five bad fits.

In M.32 you drew bounded contexts around services and gave each its own team. The corollary — that each context probably wants its own data store, too — is the argument this module makes. Because different bounded contexts don't just have different code; they have different query shapes. A product-search workload wants to find "wireless headphones under $100 with 4+ stars, sorted by relevance." A session workload wants to look up session:abc123 in under a millisecond, a million times a second. An analytics workload wants to scan two billion order rows and sum revenue by region. These aren't preferences — they're structurally different access patterns, and they push a relational database in structurally different bad directions. Polyglot persistence is the operational discipline of matching each workload to a store type built for its shape.

// SIX WORKLOADS · ONE POSTGRES · HOW EACH ONE FARES
user-profile-svc// account details · settings · preferences
Read-heavy OLTP with joins. SELECT ... FROM users JOIN preferences JOIN addresses WHERE user_id=?. Modest cardinality per query, sub-100ms latency needs, strict consistency (user just updated their address; the next read should see it).// PATTERN: transactional joins · ACID · ~1K reads/sec
POSTGRES:
EXCELLENT
product-search-svc// browse · faceted filter · relevance
Full-text + faceted queries with relevance scoring. "wireless headphones under $100 with 4+ stars, sorted by relevance to my query." Postgres has full-text search (ts_vector), but no relevance scoring at scale, no faceted aggregation, no fuzzy matching. Every workaround compounds.// PATTERN: inverted index · relevance · faceted
POSTGRES:
POOR
session-svc// login tokens · shopping carts
1M+ lookups/sec, sub-ms latency, ephemeral data. GET session:abc123 → {user_id, cart, expires}. Session state doesn't need SQL — it's key→value with a TTL. Postgres row overhead, WAL writes, and full transactional machinery make each op ~100× more expensive than needed.// PATTERN: KV lookup · sub-ms · high QPS · TTL
POSTGRES:
POOR
analytics-svc// revenue rollups · cohorts · dashboards
Massive aggregations over billions of rows. SELECT region, SUM(revenue) FROM orders WHERE ts BETWEEN .... Postgres is row-oriented — every scan reads irrelevant columns from disk. What ClickHouse does in 200ms takes Postgres 40 minutes and consumes so much I/O it hurts every OLTP query on the same instance.// PATTERN: columnar scan · analytical · GB reads
POSTGRES:
AWFUL
payment-ledger-svc// financial records · balances · audit trail
Append-only writes + strong consistency + audit. INSERT INTO transactions ..., SELECT ... WHERE account_id=?. ACID is a genuine requirement here — a failed write must not leave a phantom charge. Postgres is exactly the shape this needs: strict serializability, WAL for durability, tooling for point-in-time recovery.// PATTERN: ACID · audit · high integrity
POSTGRES:
EXCELLENT
event-history-svc// clicks · pageviews · time-series metrics
Very high-write, time-ordered, cheap to lose one. INSERT event ... at 100K writes/sec, mostly appending. Postgres indexes get destroyed by the write volume; B-tree maintenance costs dominate. Time-series databases store this data ordered by time from day one, at 10× the write throughput.// PATTERN: high-write · time-ordered · low-value each
POSTGRES:
POOR
// THE FIT SUMMARY

Postgres handles 2 of 6 workloads excellently (user-profile, payment-ledger — both transactional with joins and consistency). It handles 4 of 6 workloads poorly to awfully — and those 4 include the ones with the highest volume and cost. The 2 workloads Postgres excels at are worth building around; the other 4 need different shapes. That's the whole polyglot argument in one table.

The reason a relational database is a poor fit for these workloads isn't a limitation to overcome — it's a design intent to respect. Postgres was designed for OLTP with joins and ACID guarantees, and it's superb at that. It wasn't designed to answer "which products match this fuzzy search query with facets and relevance scoring" (Elasticsearch's job), or "what's the sum of revenue by region for a billion rows" (ClickHouse's job), or "give me this session key in 100µs a million times a second" (Redis's job). Each of those workloads has a data store built around its specific access pattern, storing data in a shape that makes the target query cheap. Elasticsearch stores an inverted index; ClickHouse stores columns instead of rows; Redis stores everything in RAM. Matching workload to shape is what polyglot persistence is — the recognition that a single hammer doesn't drive every kind of nail well.

The counter-argument — "but running one database is so much simpler than running six" — is real and often decisive. Every additional store is a permanent operational commitment: backups, disaster recovery, monitoring, on-call runbooks, upgrade cycles, security patches, and hiring for a new set of skills. Six stores is not 6× the work, but it's 3–4× — and that overhead never goes away. The polyglot decision must earn its cost. §04's lab lets you feel exactly which workloads move the needle enough to justify their own store and which are fine as tables in your OLTP database; §05 covers the coordination pain that makes polyglot fail when it fails. But the core insight stays: fit-for-purpose beats one-size-fits-all whenever the fit matters more than the operational simplicity. The interesting engineering is knowing when that's true.

One database is the operational simplicity you always want. Six databases is the workload fit you sometimes need. Neither is universally correct.
§ 02 — Six shapes for six kinds of work

A store for
every shape.

The database ecosystem has, over 20 years, converged on six canonical store types — each shaped around a specific class of access pattern, each optimally cheap for some queries and expensive-to-impossible for others. Understanding the six is the substrate for every polyglot decision: you're not choosing "MongoDB vs Postgres" as products, you're choosing "document-oriented shape vs relational shape" as data structures. Every specific product is an implementation of one of these shapes (sometimes two), tuned for scale, ergonomics, or operational fit. Below are the six; the honorable-mention seventh (graph) sits at the end for completeness.

// SIX STORE TYPES · SIX SHAPES · WHERE EACH FITS AND FAILS

// I · RELATIONAL / OLTP
Relational
PostgreSQL · MySQL · Aurora · CockroachDB

Row-oriented storage, ACID transactions, joins, secondary indexes. The default choice for anything transactional with relationships between entities. Superb for OLTP workloads under ~1TB per instance; scales horizontally with pain but scales vertically for surprisingly long. When in doubt, this is where data starts.

Fits: transactional writes · joined reads · strong consistency · financial records
Fails: full-text search at scale · analytical scans over billions · sub-ms lookups
// II · DOCUMENT
Document
MongoDB · DynamoDB (document mode) · Couchbase · FerretDB

Schema-flexible aggregates stored as JSON-ish documents. Each record is a self-contained tree that maps naturally to a domain aggregate (order + line items + shipping address in one document). Reads that fetch a full aggregate become one indexed lookup; joins become impossible or awkward. Preferred when the domain is aggregate-shaped and schema evolution is frequent.

Fits: aggregate reads · schema evolution · nested data · content-heavy documents
Fails: multi-aggregate joins · strong consistency across documents · analytical rollups
// III · KEY-VALUE / CACHE
Key-Value
Redis · Memcached · DynamoDB (KV mode) · Cloudflare KV

Get by key, set by key, expire by key. Everything else is optional. Redis extends this with data structures (lists, sorted sets, streams) and it's still fundamentally a KV store. In-memory ones (Redis, Memcached) offer sub-millisecond latency at 100K+ ops/sec per instance. Cache, session state, rate-limit counters, real-time leaderboards — anywhere you want the fastest possible lookup by a single ID.

Fits: session state · rate-limit counters · caches · leaderboards · sub-ms lookups
Fails: any query that isn't "give me this key" · durable primary storage (usually) · complex analytics
// V · COLUMNAR / OLAP
Columnar
ClickHouse · BigQuery · Snowflake · Redshift · Druid

Data stored column-by-column instead of row-by-row, plus aggressive compression and vectorized query execution. Analytical scans over billions of rows finish in seconds because they only read the columns the query touches. Trade: writes are expensive (columnar formats aren't friendly to random inserts) and single-row lookups are slow. Perfect complement to a relational OLTP store — the source of truth handles writes; the columnar store handles the "how much revenue by region" questions.

Fits: analytical rollups · reporting · dashboards · time-series aggregations at scale
Fails: OLTP · single-row lookups · transactional writes · high update rates
// VI · TIME-SERIES
Time-Series
Prometheus · InfluxDB · TimescaleDB · VictoriaMetrics · QuestDB

Data stored in time-ordered blocks with aggressive compression on similar values. Optimized for the "N million data points per second, mostly append-only, queried by time range plus tags" workload. Metrics, IoT sensor data, financial ticks — anywhere the primary access pattern is "give me all events from timeframe X, aggregated by tag Y." Time-partitioned by design; drops old data cheaply via partition eviction, unlike relational stores where large deletes are expensive.

Fits: metrics · high-write ordered data · time-range queries · downsampled rollups
Fails: arbitrary joins · complex transactions · text search · anything not indexed by time

Honorable mention: graph databases (Neo4j, Amazon Neptune, TigerGraph, JanusGraph). Graph stores are the right shape for domains dominated by traversals — "find all users connected to user A within 3 degrees who also purchased product X." Recommendation engines, fraud rings, knowledge graphs, and social networks are the classical fits. They're a smaller market than the six above because most domains don't have workloads centered on traversal — but where they exist, the fit is dramatic (queries that are days in SQL are seconds in Cypher). If you don't have a traversal-heavy workload, you don't need a graph database; if you do, nothing else comes close.

Two composite observations worth naming. First, most modern products blur these categories deliberately. PostgreSQL has a JSON column type (poor-man's document), a ts_vector (partial search), TimescaleDB as a first-party time-series extension, and pg_partman for partition-based data lifecycle — Postgres is a legitimately serviceable "80% good enough" for many secondary workloads. MongoDB has strong secondary indexes and some join capability. Redis has a search module. The shapes remain distinct; the products increasingly cover multiple shapes with progressively worse fit as you move away from their core. Second, the "which shape do I need?" question is genuinely a first-order design decision — architects who default to "we'll use Postgres for everything" pay a compounding penalty on their off-fit workloads that shows up as latency, cost, and operational firefighting years later. Making the shape choice consciously — even when the answer is "one Postgres for now" — is the discipline this section teaches.

Different workloads have different shapes. A relational database is a shape. So is a search index. So is a columnar warehouse. Matching workload to shape is what polyglot persistence is.
§ 03 — CDC · the glue that makes polyglot honest

One source
of truth.

The naive way to run polyglot is to have every write go into every store. The service inserts an order into Postgres, then indexes it into Elasticsearch, then invalidates the Redis cache, then emits a message to the analytics warehouse. This works — until any one of those writes fails and the stores diverge. Which one is now the truth? The one you retried? The one you didn't? The answer is that in a system with N stores updated in-line by application code, the truth is undefined during any partial-failure window, and eventually one of those windows will be big enough to produce user-visible incorrectness. Change Data Capture (CDC) solves this by picking one store as the source of truth (usually the relational OLTP store) and deriving all others from its change log. The derived stores are eventually consistent with the source; the source is the authoritative record. The truth is defined; only its propagation is asynchronous.

// THE CDC PIPELINE · POSTGRES SOURCE → KAFKA → DERIVED STORES

order-svc writes only here POSTGRES source of truth WAL · logical replication DEBEZIUM CDC connector KAFKA change log write WAL event ELASTICSEARCH product search CLICKHOUSE analytics rollups REDIS order-detail cache consumer consumer ... query-time reads back to services Source of truth: Postgres. Derived stores: eventually consistent. Every store gets updated the same way — through Kafka.
// PROPERTY 1 · ONE WRITER
Application code only writes to Postgres. That's the source of truth. Elasticsearch, ClickHouse, Redis, and every other derived store are updated only via the CDC stream. Divergence between stores becomes impossible because they can't be written to independently.
// PROPERTY 2 · DEBEZIUM READS THE WAL
Debezium subscribes to Postgres's Write-Ahead Log (via logical replication), converts each row change to an event, and publishes it to a Kafka topic. Same mechanism for MySQL binlog, MongoDB oplog, DynamoDB streams. The database's own durability log becomes the change source.
// PROPERTY 3 · CONSUMERS UPDATE STORES
Each derived store has a Kafka consumer that applies changes to it. The Elasticsearch consumer indexes new orders; the ClickHouse consumer inserts denormalized rows; the Redis consumer invalidates cache keys. Each consumer scales independently; each failure recovers independently by resuming from its Kafka offset.
// PROPERTY 4 · EVENTUAL CONSISTENCY
Derived stores lag the source by milliseconds to seconds, depending on consumer load and network. This is a real constraint: a search index that lags Postgres by 200ms means users may search and not find something they just created. Usually acceptable; sometimes it drives explicit "read your writes" workarounds (M.22 callback).

The CDC pipeline callbacks a lot of what Phase F built. The Postgres → Kafka → derived-stores flow is the same shape as the outbox pattern from M.31: writes to the source of truth are the atomic unit; the Kafka topic is the "official" broadcast of what happened; consumers derive their own local materialization. The consistency guarantees are the same too — at-least-once delivery from Kafka means consumers must be idempotent (they might see the same order twice on retry), and the eventual-consistency window means derived stores are always slightly behind the source. The stronger the sync you want, the more expensive it gets; most polyglot systems accept ~100ms to a few seconds of lag on derived stores and design their UX around that.

Two things about CDC in production that are easy to underestimate. Schema evolution is genuinely hard: when the source schema changes, every consumer must handle both old and new shapes during the migration window (Kafka retains old messages; new consumers read them; new producers emit new format). The dance between adding a column, deploying producers, updating consumers, and finally cleaning up old-format handlers is 4-6 weeks of work per breaking change in a mature system. Backfills are the other gotcha: adding a new derived store means playing history back — either by reading the whole Kafka topic from the beginning (needs infinite retention), or by doing a one-time bulk load from Postgres followed by resuming from the CDC stream (needs careful sequencing to avoid missing or duplicating changes). Neither problem is a dealbreaker; both are the specific cost of polyglot that architects should budget for. §05 covers the anti-patterns that emerge when teams underestimate these costs.

Change Data Capture is what makes polyglot honest. Without it, you have five copies of the truth and no source of it.
§ 04 — Workload-to-store matcher

Same workload.
Three architectures.

Below: five workloads from a real e-commerce system — user profile, product search, session state, analytics rollup, payment ledger. Pick an architecture — single Postgres (one store handles everything), pure polyglot (services write directly to specialized stores), hybrid with CDC (Postgres as source of truth, specialized stores derived via Change Data Capture). Then pick a workload and see how each architecture handles it. Watch which architecture excels at which workload, which pays the cost of over-serving, and where CDC pays for its own complexity.

POLYGLOT.SIM // m.37 lab
Workload →
// DATA FLOW · service → store · current workload highlighted
// METRICS · WORKLOAD FIT
Store type-
Query latency-
Throughput fit-
Consistency-
Ops burden-
Workload fit-
// VERDICT
Loading...
...
§ 05 — Where polyglot decays

The coordination
tax.

Every polyglot architecture that fails has one of five specific failure modes. The success stories are quiet — the Postgres runs, the Elasticsearch indexes, the ClickHouse aggregates, the CDC pipeline keeps them in sync, and nobody thinks about it. The failure stories are loud: quarterly all-hands about "data drift" between stores; two-week outages when one derived store falls too far behind; the search index that shows product prices from three weeks ago. The five patterns below are the ways teams pay the coordination tax; recognizing them from the design phase is where operational maturity actually shows up.

// FIVE ANTI-PATTERNS · HOW POLYGLOT DECAYS INTO OPERATIONAL BURDEN

i
The shared derived store
"order-svc, catalog-svc, and pricing-svc all write to the same Elasticsearch index."

Multiple services writing to a single "shared" store is the distributed-monolith pattern from M.32, in the data layer. Schema changes require coordination across all writers; incidents blame every writer; the shared store has no clear owner. The fix: each service owns its own store — or, if genuinely one canonical index is needed, one service owns it and others send events for it to consume. Ownership must be clear.

ii
The dual-write divergence
"We write to Postgres AND Elasticsearch in the same transaction. What could go wrong?"

Writing to two stores from application code is not atomic. Whatever order you pick, one succeeds and the other might not. The stores drift; users see search results that don't match reality; incidents accumulate. The fix: pick one store as source of truth; derive the other via CDC. Application code writes to exactly one place. The single-writer invariant is what makes polyglot correct; without it, polyglot is guaranteed inconsistency.

iii
The CDC lag catastrophe
"The search index is 6 hours behind Postgres and nobody noticed."

CDC consumers fall behind — sometimes quietly. Kafka lag on the derived store's consumer group grows during a load spike; nobody's alerted because "lag alerts" weren't configured; the derived store slowly diverges. The fix: monitor consumer lag on every CDC-fed store; alert when it exceeds a threshold that matches your UX tolerance (usually seconds to a minute). Lag is the polyglot-specific SLI; treat it like latency for HTTP services.

iv
The backfill nightmare
"We want to add a new derived store. Six months of data to backfill. How?"

Adding a new derived store means playing history back into it before turning it on. Options: (a) infinite Kafka retention and replay from start — usually infeasible; (b) bulk-load from source + resume CDC — needs careful cursor management to avoid missing or duplicating events; (c) shadow-mode for weeks while catching up — needs isolated compute. The fix: standardize a backfill runbook, budget the time, don't underestimate the coordination. Every "we should also put this data in a new store" idea implicitly commits to this cost.

v
The ops-burden spiral
"We adopted five stores. Now we have five on-call rotations and no polyglot experts."

Each store is a permanent operational commitment: backups, DR drills, monitoring dashboards, upgrade rituals, security patches, capacity planning, and — most expensively — humans who understand it. Teams that adopt polyglot without budgeting for the operational headcount find themselves running five stores badly. The fix: adopt polyglot incrementally; each new store must earn its ops cost. Sometimes the answer is "keep it in Postgres for now, migrate when the pain justifies the platform investment."

The composite advice: polyglot persistence rewards discipline and punishes optimism. Teams that treat it as "we'll use MongoDB for this thing because it feels right" without a specific workload argument and without CDC discipline end up with the ops-burden spiral. Teams that treat it as "we identified this specific workload doesn't fit our OLTP database, and we've designed the CDC path to keep the derived store honest" ship polyglot successfully. The pattern is not "adopt many databases"; the pattern is "make deliberate, workload-driven, well-plumbed exceptions to your default." The default should be a single, well-run relational database (usually Postgres); the exceptions should be earned by measurable workload fit and paid for by explicit operational budget. When those conditions are met, polyglot is a superpower. When they aren't, it's a permanent tax.

The dominant industry pattern in 2025 reflects this: hybrid-with-CDC has quietly become the standard architecture for anything beyond a small startup. Postgres (or Aurora, or CockroachDB for larger scale) as the OLTP source of truth; Debezium or a managed CDC service as the change stream; Kafka as the backbone; then derived stores as needed — Elasticsearch for search, ClickHouse or Snowflake for analytics, Redis for cache. Pure polyglot (where each service picks its own primary store with no shared source of truth) works for teams organized around aggressive service ownership but has largely been out-competed by hybrid-CDC in scenarios requiring cross-store reporting or search over transactional data. The industry converged on "one source of truth, many derived views" — which is exactly the shape M.29's CQRS predicted for read-heavy systems. Different vocabulary, same underlying architectural idea.

Every additional store is a permanent operational commitment — backups, DR, monitoring, upgrades, security patches, humans who understand it. Don't buy stores you don't need.
§ 06 — Eight words for the polyglot conversation

Vocabulary,
for the store map.

The terms every "should we adopt Elasticsearch?" architecture doc, every CDC pipeline design, every "why is our search 6 hours behind?" incident review assumes you already know.

Polyglot Persistence
/ˈpɒl.i.ɡlɒt/
The pattern of using multiple data-store types in one system, each matched to a specific workload's access pattern. Coined by Neal Ford (2006) and popularized by Fowler (2011). Assumes bounded-context ownership (M.32) and usually needs CDC to keep stores honest.
OLTP
/oʊ ɛl tiː piː/
Online Transaction Processing — the classical "many small transactions per second" workload of business applications. Relational databases (Postgres, MySQL, Aurora) are shaped for this: row-oriented storage, ACID, joins, secondary indexes.
OLAP
/oʊ læp/
Online Analytical Processing — massive aggregations over billions of rows for reporting, dashboards, business intelligence. Columnar stores (ClickHouse, BigQuery, Snowflake) are shaped for this: column-oriented storage, aggressive compression, vectorized execution.
CDC · Change Data Capture
/siː diː siː/
The mechanism of reading a database's change log (WAL, binlog, oplog) and emitting each change as an event. Debezium is the reference open-source implementation. Turns a database's durability log into a stream of change events consumable by other stores.
Source of Truth
/sɔːrs əv truːθ/
The store that holds the canonical, authoritative record of data. In hybrid-CDC architectures, usually a relational OLTP database. All other stores are derived — updated from the source via CDC, never written to directly by application code.
Derived Store
/dɪˈraɪvd/
A store populated by consuming changes from the source of truth, not by direct application writes. Typical examples: Elasticsearch for search, ClickHouse for analytics, Redis for caches. Eventually consistent with the source. Regenerable from the CDC stream (in principle).
Inverted Index
/ɪnˈvɜːr.tɪd/
The data structure that powers full-text search — maps each token to a list of documents containing it. Enables efficient "find docs matching this query" instead of scanning every document. Elasticsearch, OpenSearch, Meilisearch, Vespa are all inverted-index engines at their core.
Debezium
/dɪˈbiː.zi.əm/
The reference open-source CDC platform — a set of Kafka Connect source connectors that read Postgres WAL, MySQL binlog, MongoDB oplog, and others, and emit changes as Kafka events. The near-default CDC choice for self-hosted polyglot systems.
§ 07 — Knowledge check

Five questions.
Match shape to workload.

Test the polyglot intuition. Click an answer; explanation drops in instantly.

QUESTION 1 OF 5
Loading question...
Score: 0 / 5
5 / 5

Matched.

Perfect. Store shapes, CDC discipline, anti-patterns — all yours. Phase G has one module left: the pattern-language for how services actually talk to each other. Sync REST, async events, choreography, orchestration — the finale that pulls the whole phase together.

§ 08 — The recap

Three ideas to
carry forward.

The data-layer extension of bounded contexts, with CDC as the discipline that keeps it honest.

i

Workloads have shapes

Full-text search wants an inverted index. Analytical scans want columnar. Session state wants sub-ms KV. Financial records want ACID. Postgres serves some excellently and others awfully; matching shape to workload is what polyglot persistence is.

ii

CDC makes it honest

One store is the source of truth. All others are derived via Change Data Capture — Debezium reads the WAL, publishes to Kafka, consumers update the derived stores. Application code writes to exactly one place. The single-writer invariant is what makes polyglot correct.

iii

Every store is a permanent commitment

Backups, DR, monitoring, upgrades, security, on-call — six databases is not 6× the work but it's 3-4×. Adopt polyglot incrementally; each new store must earn its ops cost with measurable workload fit. Default to one; make exceptions deliberately.

↓ UP NEXT · PHASE G FINALE

M.38 — Inter-service
communication.

You have services (M.32), gateways (M.33), meshes (M.34), discovery (M.35), tracing (M.36), and stores (M.37). Now: how do the services actually talk to each other? Sync REST vs async events. Choreography vs orchestration. Request-response vs pub-sub. The pattern-language that pulls all of Phase G together — and the phase-completion banner that closes the microservices chapter.

Continue to Module 38 →