Identity
A streaming application is persistent. Change the SQL and deploy again, and the new version either reuses the state the old one built or rebuilds it from the beginning of the input. Deciding which needs two things from every version of the plan: a way to say “this is the same node”, and a way to say “it stores the same thing”. Streamform gives every node both.
Node ids
Every node has a stable id: <model>::<kind>::<ordinal>.
customer_metrics::source::0
customer_metrics::aggregate::0
customer_metrics::project::0
customer_metrics::sink::0
The ordinal counts operators of that kind within the model. Identity is positional on purpose, and independent of what the node does: change the aggregate’s key or its accumulators and it is still customer_metrics::aggregate::0, the same node, changed. That is what lets an upgrade say “the aggregate changed its key” rather than “one node removed, one node added, rebuild”.
What moves an id: renaming the model, or inserting a second operator of the same kind before an existing one. What does not: whitespace, column order, a changed key, an added WHERE, a changed accumulator.
Digests
Every stateful node has a state digest: the SHA-256 of its fingerprint text (aggregate key=[customer_id] accumulators=[sum(amount): decimal(38,9)] retention=unbounded). Every node has a schema digest: the SHA-256 of its output columns by name and canonical type, in order. Text output shows the first twelve characters; the full 64 are what is compared.
Both are computed from Streamform’s own renderings, never from a dependency’s output, so upgrading the compiler’s dependencies or the Rust toolchain cannot move a digest. A digest moves exactly when meaning moves. The digest of the customer_metrics example, 4aead023aa01…, is pinned by a test; changing it is a deliberate act.
| Edit | id | state digest |
|---|---|---|
| Reorder projection columns, reformat the SQL | same | same |
SUM(amount) becomes COUNT(*) | same | different |
GROUP BY customer_id becomes GROUP BY customer_id, country | same | different |
| Rename the model | different | same text, but under a new id |
The manifest
streamform inspect-plan --json prints all of it for the whole application as a versioned JSON document, the plan manifest, and streamform build writes the same document to build/plan.json. A reader refuses a manifest from another format version or with fields it does not know. See Preparing for upgrades for the workflow and the plan identity reference for the document.
What this is for
streamform plan compares the running manifest with the desired one and classifies every change before anything is deployed: SAFE, COMPATIBLE, STATE MIGRATION REQUIRED, BACKFILL REQUIRED, STATE INCOMPATIBLE. Identity is what lets it say “the aggregate in customer_metrics changed its key” instead of “one node removed, one added”, and it exists early on purpose: identity that is bolted on after users have running state cannot be changed without invalidating that state. See Upgrading a running application and the plan diff reference.