Streamform

Your SQL, running as a streaming application.

Streamform turns the SQL models a data team already writes into stateful streaming applications: tested on your laptop, compiled for Apache Flink, deployed in your own cloud. No Java, no Flink expertise, one binary.

Install v0.4 Your first project

for data and analytics engineers in SQL, YAML, and Git on Apache Flink, in your cloud free to use, in production too

topic ordersappend only

models/customer_metrics.sqlupsert

SELECT customer_id,
       SUM(amount) AS total_spend
FROM source('orders')
GROUP BY customer_id

table customer_metricscurrent

Changelog: insert customer 42 at 60, then updates to 130, 140, and 90 after the refund.

You write the query you would write anyway. Streamform works out that the input never ends, that the result is a table keyed by customer, what state that takes, and the Flink job that keeps it current, including the refund.

What you get, in the order you use it.

A streaming application is the query plus its input, its state, its output, its tests, and its upgrades. Streamform gives you each of those as a file you can read or a command you can run, and this is the order they come in.

models/*.sql
Write it in SQL.

A model is a SELECT. It reads a topic with source() or another model with ref(). That is the whole authoring surface: no DSL, no Java, no job class.
If you can write the query, you have written the application.
models/customer_metrics.sqlthe example project
```
SELECT customer_id,
       SUM(amount) AS total_spend
FROM source('orders')
GROUP BY customer_id
```
sources.yml and streamform.yml
Say the three things SQL cannot.

Where the input comes from and whether it ends. How the result is materialized: a stream of facts, or a table kept current by key. Which column carries event time, and how late data may arrive.
Three declarations stand in for the Flink expertise: input, result, time.
sources.yml, streamform.ymltrimmed
```
sources:
  orders:
    connector: kafka
    topic: orders
    event_time: ordered_at
    watermark: 5m

models:
  customer_metrics:
    materialized: upsert
    key: [customer_id]
```
streamform test
Test the data before anything runs.

A fixture gives a model some events and states the exact changes it must emit. A deterministic simulator runs the whole chain in-process, on a laptop or in CI. No Kafka, no cluster, no waiting.
Streaming data quality is asserted before deploy, not discovered after.
streamform testreal output, trimmed
```
big_customers

PASS

4 events processed
3 changelog records emitted

customer_metrics

PASS

2 events processed
2 changelog records emitted
1 late event dropped

ok: 4 fixtures passed
```
streamform build
Compile to a streaming backend.

The project compiles to one backend-neutral streaming plan. Apache Flink runs it today; a native runtime is on the roadmap. What comes out is plain files you can read, diff, and commit.
Your SQL is not tied to an engine. The plan is what the engine runs.
streamform build --backend flinkreal output
```
ok: 7 statements for target `local`
    written to build/flink/application.sql
ok: plan manifest written to build/plan.json
```
streamform apply
Deploy in your own cloud.

apply submits the application to your Flink through the SQL Gateway, against your Kafka, wherever they run, and writes down what it applied. Nothing is hosted and no account is involved.
It runs where your data already is.
streamform applyreal output, trimmed
```
applying order-pipeline to target `local`

ok: source orders
ok: model clean_orders
ok: model customer_metrics
ok: model big_customers
ok: sink big_customers

ok: applied
```
streamform plan
Know what the next change costs.

State outlives the code that built it. plan compares the running application with the new one, node by node, and classifies every change from SAFE to STATE INCOMPATIBLE. CI can refuse on the exit code.
Upgrading a stateful application is a decision. plan puts the facts in front of it.
streamform planone filter edited upstream
```
customer_metrics  changed
  BACKFILL REQUIRED: input definition changed
  upstream at `clean_orders::filter::0`

big_customers  SAFE

verdict: BACKFILL REQUIRED
```

Without it, a team pays one of three ways.

The logic for a real-time metric is usually already written, as SQL, by the people who own the data. Getting it to run continuously is where it stops being theirs.

Build the application by hand.

A Flink or Kafka Streams job in Java or Scala. The business logic lives inside the job, next to state handling, serialization, and checkpoints, and only the engineers who wrote it can change it safely.

the logic leaves SQL for good

Leave the analytics team out.

The people who own the models and the metrics cannot ship any of it as streaming, so real time becomes a special project with its own backlog, and the dashboards stay a day behind.

the data owners cannot ship it

Buy a streaming platform.

A managed streaming-SQL service. It works, and it is priced by throughput; the runtime and the artifacts belong to the vendor, and there is still no way to test a change on a laptop before it is live.

priced by throughput, owned by the vendor

With Streamform, each of those stays where it belongs.

the logicstays in SQL, in the models the data team already owns
the testsrun in CI and on a laptop, against the exact changelog
the artifactssit in your repository as plain files you can read and diff
the applicationruns in your cloud, on your Flink and your Kafka

Someone who knows SQL can build it and deploy it.

Real-time data for agents.

An agent answering a customer is only as good as what it knows right now. A batch pipeline tells it who the customer was last night. A Streamform model keeps the customer's state current as orders and refunds happen, and publishes every change to a topic the agent's tools read, so the answer reflects the refund from five minutes ago.

ordersthe events your systems already produce, on Kafka
customer_stateone model: total spend, order count, last order, kept current per customer
customer-statethe topic its changes are published to; a tool reads the latest row, or a store keeps it
the fixtureproves the refund reaches the table, before any agent depends on it

The same model serves a dashboard, an alert, or an API. An agent is one more reader, with a lower tolerance for stale data.

models/customer_state.sqlreal, and its fixture passes

SELECT customer_id,
       SUM(amount)     AS total_spend,
       COUNT(*)        AS orders,
       MAX(ordered_at) AS last_order_at
FROM source('orders')
GROUP BY customer_id

streamform.ymlwhere the changes go

models:
  customer_state:
    materialized: upsert
    key: [customer_id]
    sink: { connector: kafka, topic: customer-state }

tests/customer_state.ymla refund, asserted (trimmed)

given:
  - customer_id: 42
    amount: 60
    ordered_at: "2026-08-30T10:00:00Z"
  - customer_id: 42
    amount: -60                  # the refund
    ordered_at: "2026-08-30T10:05:00Z"
expect:
  - { op: insert, customer_id: 42, total_spend: 60 }
  - { op: update, customer_id: 42, total_spend: 0 }

The whole workflow is five commands.

Every one of them compiles the project to the same plan, so what you test is what runs, and what runs is what the next change is compared against.

streamform checkValidate the project: every model, source, key, and fixture, with errors that point at the file.
streamform testRun every fixture through the simulator and assert the exact changelog.
streamform buildGenerate the application for the backend, as artifacts you can read and commit.
streamform applySubmit it to your Flink through the SQL Gateway, and record what was applied.
streamform planCompare the running application with the new one, and classify every change before it deploys.

Install it, then run the example.

One archive for macOS or Linux, checksums beside it. The example project is three SQL files, and its fixtures pass on your laptop in well under a second.

Download v0.4.0 Read the docs

terminalApple silicon shown

$ curl -fsSL https://github.com/glyf-data/streamform-releases/releases/latest/download/streamform-aarch64-apple-darwin.tar.gz | tar xz
$ sudo mv streamform /usr/local/bin/
$ curl -fsSL https://github.com/glyf-data/streamform-releases/releases/latest/download/streamform-examples.tar.gz | tar xz
$ cd streamform-examples/order-pipeline
$ streamform test
…
ok: 4 fixtures passed

Your SQL, running as a streaming application.

What you get, in the order you use it.

Write it in SQL.

Say the three things SQL cannot.

Test the data before anything runs.

Compile to a streaming backend.

Deploy in your own cloud.

Know what the next change costs.