An unbounded stream doesn't have "the last hour" — you have to define it. Tumbling, sliding, and session windows are the three ways to slice a stream into groups that can be aggregated. Watermarks decide when a window can close. Allowed lateness decides what happens to events that arrive after. The whole vocabulary that makes stream aggregations tractable.
M.39 established that streams are unbounded — events arrive continuously, forever, with no "end of input" to trigger aggregation. That single fact makes a superficially simple question — "how many requests did we get in the last hour?" — into one of the more subtle problems in distributed data systems. In a batch world, "the last hour" is trivial: you have a timestamped input file; you filter timestamp >= now - 1h; you count. In a stream world, "the last hour" doesn't exist as a bounded set — you have to actively define it, as events flow past. Every stream aggregation over any time-based grouping — rolling averages, per-minute counts, session summaries, hourly rollups — is a windowing problem. M.40 is the vocabulary that makes those problems solvable rather than approximable.
The tradeoff has a specific shape. Waiting longer for late events gives more correct aggregations but delays the result — a 5-minute window that waits 10 minutes for late data emits results 15 minutes after the window's nominal end. Not waiting emits results faster but silently drops late-arriving events, which produces subtly wrong aggregations that nobody notices until a downstream consumer discovers a discrepancy. Neither extreme is correct in general — the right answer is workload-specific, driven by how late events actually arrive in practice, how tolerant the downstream is of small errors, and how time-sensitive the aggregation is. Making that tradeoff explicit, with knobs the engineer can turn, is what the windowing model provides. Systems that hide it (implicit processing-time, aggressive default watermarks, silently dropped late data) accumulate correctness bugs; systems that expose it force the engineer to decide deliberately.
Each attempt fails in a specific way that reveals what a real windowing system must provide: event-time semantics for grouping (not processing-time), watermarks to decide when a window can close (not just wall-clock triggers), allowed lateness to specify how late is "still on time", and update or side-output policies for events that arrive beyond allowed lateness. These four mechanisms — event-time, watermarks, allowed lateness, late-data handling — are M.40's core vocabulary, and every mature streaming system (Apache Flink, Beam/Dataflow, Kafka Streams) exposes them explicitly rather than hiding them behind defaults.
The historical grounding matters here because the vocabulary emerged from specific engineering experiences. MillWheel (Google, 2013) introduced the low-level primitives — persistent state, timers, exactly-once semantics — that windowed stream processing needs to be correct. The Dataflow Model paper (Akidau et al., Google, 2015) is the definitive treatment: it introduced the unified vocabulary of windows, triggers, watermarks, and accumulation modes that Apache Beam and Flink both adopted. Before Dataflow, every streaming system invented its own windowing terminology and made different tradeoffs implicitly; after Dataflow, there's a shared vocabulary and the tradeoffs are named and orthogonal. Tyler Akidau's "Streaming 101" and "Streaming 102" O'Reilly essays translated the paper into engineering practice. The current mainstream — Apache Flink — implements the full model with fine-grained control over each dimension. Kafka Streams provides a lighter subset (tumbling and sliding windows, watermark-like time semantics) that's often enough for simpler topologies. The rest of this module builds up the vocabulary in the order that matters for engineering decisions: the three window types first (§02), then the time model and watermark machinery (§03), then a hands-on lab of how they interact (§04), then the specific failure modes to recognize (§05).
Every stream-aggregation problem starts with the choice of window type — the shape that turns "the last N events" or "the last M minutes" into a bounded group the processor can aggregate. The Dataflow Model settled on three canonical shapes: tumbling windows (fixed size, non-overlapping), sliding windows (fixed size, overlapping), and session windows (dynamic size, driven by activity gaps). Each shape fits specific workloads and fails others. Choosing the right one is a specific design skill; using the wrong one silently distorts the output.
Fixed-size, non-overlapping windows. Each event belongs to exactly one window. The simplest and most efficient windowing type. Configuration: window size — that's it. Every event contributes to one count, one sum, one aggregation. State cost per key is O(1) at any moment — only the current window's aggregate. The default choice when you don't have a specific reason to pick something else.
Fixed-size, overlapping windows. Configuration: window size AND slide interval. If size=5min and slide=1min, a new window opens every minute and closes 5 minutes later — meaning every event belongs to ~5 windows simultaneously. Produces smooth, rolling aggregations: "average requests over the last 5 minutes, updated every minute" is a sliding-window query. State cost is O(size/slide) per key at any moment — 5× the tumbling equivalent for size:slide of 5:1. Higher compute cost is the price of smoothing.
Dynamic-size windows, defined by activity gaps. Configuration: gap timeout (e.g., 30 minutes). A session starts when an event arrives, extends whenever another event arrives within the gap timeout, and closes when the gap timeout elapses without a new event. The specific fit for user-session semantics: a browsing session, an in-app session, a customer interaction that has natural start/stop boundaries driven by human behavior rather than clock time. Session windows are the most expressive of the three types and also the most state-hungry — the processor must maintain per-key state for every active session, and sessions can grow arbitrarily large if activity continues.
The tension between the three types isn't primarily about correctness — any of them can produce correct aggregations for a given input. The tension is about semantic fit and cost. Tumbling windows are cheap and match "period-based reporting" semantics naturally. Sliding windows are expensive but match "rolling averages" semantics naturally. Session windows are the most expensive and match "human activity" semantics naturally. Choosing the wrong shape doesn't produce wrong numbers — it produces numbers that don't answer the question being asked. A tumbling window aggregating "hits per 5-minute bucket" is a perfectly valid computation; it just doesn't smooth out bursts the way a sliding average would. A sliding window computing "hits over the last 5 minutes, updated every second" produces a smooth curve; it costs 300× more state than tumbling. A session window slicing user activity into per-session bundles is expressive; it exposes the processor to unbounded state growth if sessions never close. Deliberate shape-matching per workload is the specific engineering discipline.
The tooling landscape mirrors the abstraction. Apache Flink implements all three window types with fine-grained control over slide, gap, and session-merging semantics; it's the reference open-source implementation of the Dataflow Model. Kafka Streams provides tumbling, hopping (its name for sliding), and session windows via the groupByKey().windowedBy() API. Google Cloud Dataflow / Apache Beam is where the model originated — the API deliberately makes windows, triggers, and accumulation modes orthogonal, so switching between types is a one-line change. The specific vocabulary in this section — tumbling, sliding, session — is portable across all these systems, and the "which shape fits this workload?" question is the same regardless of the tool. The next section adds the time-model machinery that makes these window types actually produce correct outputs in the face of out-of-order data.
Every streaming event has (at least) two timestamps that matter: event-time — when it actually happened in the real world (the credit-card swipe, the page load, the sensor reading) — and processing-time — when the streamer observed the event. Under normal conditions the two are close; a well-behaved event flows through in tens or hundreds of milliseconds. Under real conditions they diverge — sometimes by minutes (network delays, load spikes), sometimes by hours or days (mobile clients uploading buffered data, replay after outages, backfill jobs). Choosing which clock to use for window boundaries is the single most consequential correctness decision in a streaming pipeline, and getting it wrong produces the specific class of bugs where "the aggregation looks fine but is subtly wrong."
Under normal conditions the two axes are almost parallel — a small constant offset for network latency. Under real conditions they can diverge dramatically: mobile clients cache events offline and upload in bursts (event-time 3 hours ago, processing-time now); services under load queue events (event-time 5 minutes ago, processing-time now); backfill jobs replay historical data (event-time yesterday, processing-time now); network partitions delay in-flight events (event-time seconds ago, processing-time whenever the partition heals). Windowing by processing-time treats all these events as arriving "now"; windowing by event-time correctly places them in the past window they actually belong to. Processing-time is easier to reason about but semantically wrong for anything that answers "what happened during X period?" Event-time is correct but requires machinery to handle out-of-order arrivals.
Watermark advances quickly. Window at [10:00, 10:05) is closed at ~10:05:02 — the processor decides all data for the window has arrived. Fast emit: downstream sees the count within seconds. Drops late data: the event at event-time 10:03 that arrives at 10:06 is either dropped silently or diverted to a side-output. Fits real-time dashboards where "approximately correct now" beats "exactly correct later"; unfits billing, forensics, anything that must not silently lose data.
Watermark advances slowly. Window at [10:00, 10:05) is not closed until 10:15 — the processor waits 10 minutes for late data. Correct emit: virtually all late data has arrived by then; the count is accurate. Slow emit: downstream doesn't see 10:00-10:05 results until 10:15, adding 10 minutes of latency to every window. Fits billing, compliance, financial reporting; unfits anything with real-time consumption.
The mechanism that makes both extremes tunable is the watermark. Formally: a watermark is a signal — a timestamp in the event-time domain — that says "the processor believes no events with event-timestamp less than X will arrive from this point on." When the watermark advances past a window's end time, the processor knows it's safe to emit the window's aggregate. The watermark is an assertion about the future, and like all such assertions, it can be wrong: if a late event arrives after the watermark advanced past its event-time, that event belongs to a window the processor already considered "closed." How the system handles this exception is the "allowed lateness" and "late data" policy. Watermarks can be generated in different ways: bounded-out-of-orderness watermarks assume events won't be more than N seconds late; punctuated watermarks are emitted by upstream sources at known points ("I promise no events with timestamp before X will come from me"); heuristic watermarks estimate lateness from observed data. Each strategy trades off correctness against speed differently, and mature systems let engineers configure the strategy per pipeline.
allowedLateness(Duration) — window stays "open" for late updates for N minutes past watermark. Late events within this window update the emitted result. Beyond it → dropped or side-output.sideOutputLateData(tag) — events arriving beyond allowed lateness are emitted to a separate stream instead of being dropped. Enables reconciliation, audit trails, and downstream correction.trigger(...) — determines when a window emits results: at watermark, periodically during window, on every event, or a combination. Beam's most flexible dimension; Flink exposes similar controls.The composite mental model for §03 is: the streamer maintains an evolving belief about "how far along in event-time we are" — the watermark — and every window aggregation policy is defined in terms of that watermark. Window opens when the first event with matching event-time arrives. Window "considered closed for on-time data" when the watermark advances past window-end. Window "considered closed for late data" when the watermark advances past window-end + allowed-lateness. Beyond that, late events are diverted (side-output) or dropped. These four events — open, on-time-close, late-close, drop-or-divert — are the state machine every stream aggregation navigates, and expressing that state machine explicitly is what modern streaming APIs (Flink, Beam, Kafka Streams) do. The §04 lab lets you feel this state machine directly by watching windows open, close, and emit under different data patterns.
Below: the same stream of events analyzed with each of the three window types (tumbling · sliding · session) against three different arrival patterns (steady · bursty · late-arriving). Watch the same events get grouped differently by each window shape, watch what happens when bursts arrive, watch how late-arriving events are handled (or aren't). The 9 combinations illustrate why "which window type do we use?" is a design decision, not a default.
The failure modes of windowing are subtle because the pipeline usually keeps running — no errors, no timeouts, no dashboards turning red. The bugs are downstream: aggregations that are wrong by a small percentage, alerts that don't fire when they should, revenue reports that don't quite match the underlying source of truth. Recognizing these five patterns during design review — or better, during code review of windowing configuration — is what mature stream-engineering practice looks like.
Watermark advances too quickly relative to actual event lateness; a meaningful percentage of events arrives after the watermark has passed their window and gets silently dropped. The pipeline reports success; the aggregation is subtly wrong; the missing 8% is bucketed nowhere. The fix: measure actual lateness distribution first, THEN pick watermark aggressiveness. Enable side-output for late data so "dropped" is observable rather than silent. Add an allowed-lateness period that captures the 99th percentile of observed lateness. Watermark tuning is empirical; setting it to a default and hoping is the bug.
Windowing by processing-time (when the streamer saw the event) instead of event-time (when the event happened) silently corrupts every time-scoped aggregation. Users see revenue reports where "9 AM revenue" doesn't match the underlying transactions; SREs see request-rate metrics that spike at deploy time (because deploys clear the backlog); fraud detection scores transactions based on when they arrived, not when they happened. The fix: use event-time semantics explicitly for any aggregation that must answer "what happened during period X?" Processing-time is only correct for questions like "what did the streamer observe during period X?" — which is a much narrower set of questions than it appears.
Session windows extend indefinitely as long as events arrive within the gap timeout. For workloads where activity is genuinely continuous — server heartbeats, IoT sensor readings, high-frequency trading events — session windows never close, state grows without bound, and the processor eventually OOMs. This is the specific stream-state failure previewed in M.39.v and expanded in M.41. The fix: session windows require the workload to actually have natural gaps; for workloads without them, use tumbling or sliding windows with fixed sizes. Or use session windows with a maximum session duration (Flink supports this) so runaway sessions get force-closed.
Sliding-window state cost scales as O(size/slide) — every event contributes to (size/slide) simultaneous windows. Size=1h, slide=1s means every event is in 3,600 windows at once; the processor maintains 3,600× the state of a tumbling equivalent. The fix: choose slide such that size/slide is small (usually < 10). If you truly need 1-second granularity over a 1-hour window, consider whether the underlying computation can be done more efficiently — e.g., an incremental aggregation (Flink's AggregateFunction) or a probabilistic sketch (HyperLogLog for cardinality, Count-Min for frequencies) rather than raw event retention. Sliding windows are the most cost-sensitive window type; unstated size:slide ratios are how teams accidentally blow up their compute budget.
Joining two streams within a windowed context requires both streams' watermarks to be aligned — the join operator must wait for both watermarks to advance past the window before considering the window closed. If one stream is slower or has more lateness, the join operator either (a) waits for the slower stream (adds latency and grows state), or (b) times out on the slower stream and closes windows before its events arrive, silently missing joins. Both failure modes produce "join miss rates" that seem inexplicable. The fix: measure per-stream lateness separately; set allowed lateness to accommodate the slower stream; monitor watermark alignment as an operational metric. Windowed joins are the highest-complexity operation in streaming, and watermark discipline is the specific correctness lever.
The composite pattern across all five is that windowing correctness is empirical, not theoretical — the right watermark, the right allowed-lateness, the right window type all depend on the actual behavior of the incoming stream, not the ideal behavior of a diagram. Systems that treat these knobs as configuration to be set once at pipeline creation and never revisited accumulate the specific failures above. Systems that measure lateness distributions, watermark lag, per-stream freshness, and late-data drop rates as first-class operational metrics — and tune windowing parameters based on those measurements — produce reliable stream aggregations. This is where the streaming engineering discipline actually lives: the code shape is easy (Flink's API is straightforward); the correctness comes from empirical understanding of the specific data.
The upcoming Phase H modules deepen the topics this section previewed. M.41 (Stateful Stream Processing) makes the state-management problem the central topic — checkpoints, state backends (RocksDB, memory, filesystem), state migrations, exactly-once state semantics; the anti-pattern §05.iii (session windows without gap semantics) is a specific case of the broader state-management discipline M.41 develops. M.42 (Change Data Capture) covers how to source streams from operational databases; the CDC events flowing into your streaming pipeline have their own event-time semantics that must align with downstream windowing. M.43 (Real-Time Analytics) covers how to make stream state queryable at analytical latencies — the interactive-query layer over the streaming state that pipelines like the ones in this module produce. The batch/stream foundation from M.39, the windowing model from M.40, and the state management from M.41 together are the specific mental model that makes the rest of Phase H tractable.
The terms every "should this be tumbling or sliding?" design review, every "why did our aggregation drop 8%?" postmortem, every Flink watermark-tuning RFC assumes you already know.
[00:00, 00:05), [00:05, 00:10), ...size=5min, slide=1min → 5 concurrent windows per event.Test the vocabulary. Click an answer; explanation drops in instantly.
Perfect. Tumbling, sliding, session; event-time vs processing-time; watermarks and allowed lateness — all yours. The vocabulary of stream aggregations is settled. Next up: stateful stream processing (M.41), where the state these windows carry becomes the central topic — checkpoints, state backends, exactly-once semantics.
The vocabulary that makes stream aggregations tractable.
Tumbling for period-based rollups (cheap, non-overlapping). Sliding for rolling averages (expensive, smooth). Session for activity-driven groupings (natural fit for human behavior, requires real gaps). Choose the shape that matches the semantic question, not the technical default.
Event-time is when things happened; processing-time is when the streamer saw them. Event-time is correct for questions about the world. Processing-time is only correct for questions about the processor. Substituting the two silently corrupts most time-scoped aggregations.
Watermark aggressiveness trades latency for correctness. The right setting depends on actual observed lateness in your stream — not on defaults. Measure lateness distributions, tune allowed lateness to cover the 99th percentile, monitor drop rates. Correctness lives in the data.