Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Commands

Every command takes --project DIR (default: the current directory) and compiles the whole project to one streaming plan before doing its job. Exit codes follow one convention: 0 success, 1 the command ran but the project or the fixtures are not valid, 2 the command itself could not run (the project does not load, a model is unknown, a service is unreachable).

streamform check          Load and validate a Streamform project
streamform explain        Print the streaming plan for a model
streamform test           Run every fixture in tests/ through the simulator and assert its changelog
streamform graph          Print the model dependency graph: sources, models, and sinks
streamform build          Generate the application for a backend, without submitting it
streamform apply          Build the application and submit it to a target's Flink SQL Gateway
streamform inspect-plan   Print the stable identity of the plan: node ids, digests, schemas, and contracts
streamform plan           Compare the running plan with the desired plan and classify every change

streamform check

streamform check [--project DIR] [--strict] [--json]

Loads the project, validates every file, builds the dependency graph, compiles every model, validates targets, and checks every fixture’s expectations against its model’s output columns. Prints a summary of models (with their materialization and file), sources, and fixtures, then each diagnostic with its file and a help: line, then a count.

--json prints the check report instead — check_version: 1, loaded, passed, strict, the project, the three tables as records, and diagnostics with severity, path, message, help — and nothing else on stdout. When the project cannot be loaded the report is still printed, with loaded: false and the reason as its one diagnostic, before the exit 2. streamform schema check-report prints its schema.

order-pipeline (schema version 1)

models
  big_customers            upsert(customer_id)      models/big_customers.sql
  clean_orders             append                   models/clean_orders.sql
  customer_metrics         upsert(customer_id)      models/customer_metrics.sql

sources
  orders                   kafka/json   topic=orders             2 columns

fixtures
  big_customers            tests/big_customers.yml  4 given, 3 expect
  clean_orders             tests/clean_orders.yml   3 given, 2 expect
  customer_metrics         tests/customer_metrics.yml 3 given, 2 expect

ok: 3 models, 1 source, 3 fixtures, 0 errors, 0 warnings

A diagnostic:

error: model `downstream` reads ref('other_model'), but there is no models/other_model.sql
  --> models/downstream.sql
  help: ref() names another model in this project

--strict treats warnings as errors. Exit 0 with no errors, 1 with errors, 2 if the project cannot be loaded at all.

streamform explain

streamform explain [MODEL] [--project DIR] [--json]

Prints one model’s plan as its operator chain, or every model in dependency order when no model is given. Each operator shows its definition, then boundedness, changelog, and state; a keyed operator adds its fingerprint and the first twelve characters of its state digest. A model reading another model shows from=<model> on its first operator.

--json prints the model’s slice of the plan manifest (manifest_version, model, nodes), the same document inspect-plan --json prints for the whole application. Exit 2 for an unknown model or a model that does not compile, with the compile error.

streamform test

streamform test [--project DIR] [--model MODEL] [--json]

Runs every fixture, or only those for --model, through the simulator and asserts the exact changelog. Prints per fixture: the model, PASS or FAIL, events processed, records emitted; a failure adds the fixture path and a side-by-side listing of expected and actual with the first difference named. A project with no fixtures prints ok: no fixtures under tests/ and exits 0.

Exit 0 when every fixture passes, 1 when any fails, 2 when the project cannot be tested (it does not load, check would report errors, or --model names a model that does not exist). See Testing with fixtures.

--json prints the test report instead: report_version: 1, passed, and one entry per fixture with model, path, passed, events_processed, changes_emitted, and rows — one per position, each with expected, actual, and problem (null when they match; the authority on whether the row passed). Values on both sides are rendered after typing against the model’s output schema, so a matching row looks matching: decimals are strings ("28", exact, trailing zeros trimmed as in fixtures), timestamps RFC 3339, integers and booleans themselves. An update’s actual carries before, the row as it was. A fixture the simulator refuses has error and no rows. With errors check would report, no document is printed and the exit is 2. streamform schema test-report prints its schema.

streamform graph

streamform graph [--project DIR] [--format text|dot] [--json]

Prints the application as a tree: each source, the models that read it, each model’s materialization, and declared sinks. --format dot emits Graphviz. --json prints the graph document: graph_version: 1, sources, models (each with materialized, key, sink, and what it reads), edges from source:<name> or model:<name> to model:<name>, the topological order, and cycles. A reference cycle is an error for the text and DOT forms (exit 2); the JSON form prints the document with the cycle in cycles and exits 1. streamform schema graph prints its schema.

order-pipeline

orders                   source                kafka topic=orders
└─ clean_orders          append
   └─ customer_metrics   upsert(customer_id)
      └─ big_customers   upsert(customer_id)   → kafka topic=big-customers

streamform build

streamform build --backend flink [--target NAME] [--project DIR] [--stdout]

Compiles the project (refusing anything check would fail), picks the target (required when the project declares more than one), validates the application for the backend, and writes build/flink/application.sql and build/plan.json. --stdout prints the SQL and writes nothing. A model with neither a sink nor a reader produces a warning. Exit 1 when the backend cannot represent the application (the capability error is printed), 2 when the project does not pass check or the target is unknown. See Running on Apache Flink.

streamform apply

streamform apply [--target NAME] [--project DIR]

Builds the application and submits each statement in order to the target’s Flink SQL Gateway: opens a session, runs each CREATE TABLE and CREATE TEMPORARY VIEW, runs the EXECUTE STATEMENT SET, prints the job id, closes the session.

plan: nothing is recorded as applied to target `local`; the application starts fresh
applying customer-metrics to target `local` (http://localhost:8083)

ok: source orders
ok: model customer_metrics
ok: sink customer_metrics
ok: statement set → job 9d1f2c4b7a3e5f6081c2d3e4f5a6b7c8

ok: applied
ok: applied plan recorded in .streamform/applied/local.json

Before applying, apply compares the desired plan with the target’s applied record and prints the verdict (plan: SAFE against target local, applied 2026-08-27T10:12:03Z); when the verdict is above COMPATIBLE it prints the whole diff and applies anyway, saying so, because it cannot stop or restore a job yet. After the last statement is accepted it writes the applied record, .streamform/applied/<target>.json, atomically; a rejected apply leaves the previous record.

If no model declares a sink there is nothing to run: nothing to apply: no model declares a sink. If Flink rejects a statement, apply prints the statement and Flink’s reason, stops, and exits 1; no job is started and the record is unchanged. An unreachable gateway is exit 2; so is a record that could not be written after a successful apply, and the message says the apply succeeded. The gateway is plain HTTP.

streamform inspect-plan

streamform inspect-plan [MODEL] [--project DIR] [--json]

Prints the stable identity of one model, or of every model in dependency order followed by the application digest. Per model: materialization, declared key, output node. Per node: id, operator, canonical definition, output schema with its digest, boundedness and changelog contract, and state (stateless, or keyed with key, accumulators, retention, and state digest). --json prints the plan manifest: the whole application, or one model’s slice when a model is given. Exit 2 for an unknown model or a model that does not compile. See Upgrading a running application and the plan identity reference.

streamform plan

streamform plan [--target NAME] [--against FILE] [--project DIR] [--json]

Compares the running plan with the desired plan and classifies every change. The desired plan is compiled from the project, through the same checks as build. The running plan is the target’s applied record (.streamform/applied/<target>.json, written by apply) or, with --against, any file holding a plan manifest or an applied record, such as a build/plan.json from an earlier build. --target defaults when the project declares exactly one.

plan: customer-metrics against target `local`
  running: applied 2026-08-27T10:12:03Z by streamform 0.4.0, job 9d1f2c4b7a3e, application digest 7b0d9542cf02
  desired: application digest a5771fc8a5b2

customer_metrics  changed
  customer_metrics::aggregate::0  Aggregate  changed
    accumulators: [sum(amount): decimal(38,9)] → [sum(amount): decimal(38,9), count(*): int64]
    state digest: 4aead023aa01 → a5771fc8a5b2
    BACKFILL REQUIRED: accumulator `count(*)` added; it starts empty for every existing key, replay history to fill it
  BACKFILL REQUIRED: replay history through the new plan

verdict: BACKFILL REQUIRED (1 model to backfill)
note: the flink backend does not carry state across a changed query; on this target the action is a rebuild, and the classification tells you what a rebuild costs

Per model, each changed node with the fields that differ (old → new), its classification (SAFE, COMPATIBLE, STATE MIGRATION REQUIRED, BACKFILL REQUIRED, STATE INCOMPATIBLE) and the reason; then the model’s verdict, the application’s verdict with counts, and a note when the target’s backend is Flink and the verdict is above SAFE. With nothing recorded and no --against, every model is new and the verdict is SAFE. --json prints the diff document (diff_version: 1). A hint in streamform.yml that names no running node is printed as a warning.

Exit 0 when the verdict is SAFE or COMPATIBLE, so the application can continue from its existing state; 1 for any other verdict; 2 when the project does not compile or the running side cannot be read (a record or manifest from another format version, unknown fields, a file that is neither). See Upgrading a running application and the plan diff reference.

streamform schema

streamform schema --list
streamform schema <document> [--version N]

Prints the JSON Schema (draft 2020-12) of a document this build reads or writes. The schema is generated from the same types the binary uses, so it cannot disagree with what check accepts or what plan --json prints. --list names every document with the version this build speaks and where the document appears:

project         1  streamform.yml
sources         1  sources.yml
fixture         1  tests/*.yml
plan-manifest   1  inspect-plan --json, explain --json, build/plan.json
applied-record  1  .streamform/applied/<target>.json
diff            1  plan --json
check-report    1  check --json
test-report     1  test --json
graph           1  graph --json

--version N is refused, with exit 2, when this build speaks another version of the document; use it in scripts to assert the version you were written against. Every schema carries an $id of the form urn:streamform:spec:<document>:v<N>.

Editors that use the YAML language server (VS Code with the YAML extension, Zed, Neovim) complete and validate a project file from the schema when the file names it:

# yaml-language-server: $schema=.streamform/schema/project.json
version: 1
name: customer-metrics

Write the file once with streamform schema project > .streamform/schema/project.json.