Zipline watches your database, your Kafka topics, or a folder of files for changes - every insert, update, and delete - and streams them somewhere else, in order, without dropping any. Think of it like a conveyor belt that never drops a package: if a sink stumbles or a connector restarts, your changes wait safely in a queue instead of disappearing, and delivery just picks back up where it left off. It runs on your own infrastructure as a single self-hosted binary - not as someone else's hosted black box.

Zipline ships as two core binaries: zipline, the CLI you run from your terminal, and ziplined, the runtime that actually moves data. Both are single, statically-linked binaries - no JVM, no sidecars - and every connection between them is mutual TLS from the very first command.

Install the CLI

Homebrew is the fastest path on macOS and Linux:

brew install zipline/tap/zipline

curl -fsSL https://get.zipline.run | sh

Run the runtime

ziplined is the runtime - the process that holds the hub, runs your connectors, and serves the control plane the CLI talks to. The quickest way to try it locally is Docker:

docker run -d --name ziplined zipline/ziplined:latest

See Networking & ports for exactly which ports to open once you're past a local trial.

Verify

zipline --version
zipline v0.4.0 (a1b2c3d) - built 2026-07-01

Eleven words cover almost everything below. Skim once now - you'll see all of them again in Your first pipeline.

The whole story in four verbs: apply, enroll, bind, stream. Here's what each one looks like end to end - capturing changes from Postgres and routing them to Kafka.

1. Enable CDC on your source

Zipline reads change data capture (CDC) - the database's own change stream - instead of polling your tables. For Postgres, that means logical replication has to be switched on:

-- must read 'logical'
SHOW wal_level;

-- if it doesn't, set it in postgresql.conf and restart:
-- wal_level = logical

Running SQL Server instead? MSSQL needs its own CDC turned on, per database and per table - more on that in Sources & sinks.

2. Write a pipeline spec

A pipeline spec is plain YAML: one source, one or more routes to a sink. Save this as orders.yaml:

# orders.yaml - a pipeline spec
name: orders

source:
  type: postgres
  placement: external
  config:
    host: orders-db
    port: 5432
    database: orders
    user: zipline
    publication: zipline_pub
    slot_name: zipline_orders
  secret_refs:
    - { field: password, ref: orders_db_password }

routes:
  - name: to-kafka
    sink:
      type: kafka
      placement: runtime
      config:
        brokers: ["kafka:9092"]
        topic_prefix: cdc.orders

3. Apply it

apply hands that spec to the control plane running inside your runtime. It validates the config and records it as desired state - that's all:

zipline pipeline apply -f orders.yaml

4. Enroll a runtime instance

Before anything can bind to your pipeline, a runtime instance has to enroll - prove who it is over mutual TLS and receive a certificate. An admin mints a one-time token, and the instance redeems it:

# admin, once
zipline token issue --handle source-orders

# the instance, using that token
zipline connect rt.internal:9090 --token <token> --name orders-src

From here on, the instance authenticates with its certificate - no more passing tokens around.

5. Bind it

bind is the step that actually starts delivery - it attaches the enrolled instance to your pipeline's source slot:

zipline pipeline bind orders --instance src-7f3a

Delivery begins from right now. This pilot streams CDC-from-now - it doesn't snapshot the rows that already existed in the table before you bound it. Backfill is on the roadmap.

6. Watch it stream

status shows you whether it's actually healthy - per pipeline, and per slot:

zipline pipeline status orders

NAME     STATE     INSTANCE     INCIDENTS
orders   running   src-7f3a     0

SLOT     ROLE     ROUTE        STATE     PLACEMENT   FRESHNESS
source   source   -            healthy   external    lag 1.2s
sink     sink     warehouse    healthy   in-proc     lag 0.4s

Freshness/lag is the number to watch - how far behind the source your sink currently is. If it climbs and doesn't come back down, that's your first sign something downstream is struggling to keep up.

A source reads change events from where your data lives; a sink writes them somewhere else. Today, four built-in connectors cover both roles - any of them can be either end of a pipeline:

Set the type on each source and sink - it's just the connector name; the direction comes from where it sits:

# The type is just the connector name.
# Direction comes from the slot it sits in.

# valid in the  source:  slot
type: mssql
type: postgres

# valid in a route's  sink:  slot
type: postgres
type: kafka
type: file

MSSQL needs CDC turned on per database and per table before Zipline can read it; Postgres needs logical replication (wal_level = logical) - see Enable CDC on your source.

A pipeline isn't limited to one destination. One source can feed several routes at once - the same change events delivered to Postgres, Kafka, and a file sink in parallel, the fan-out you saw in the picture back in What is Zipline?. Adding a route doesn't touch the source at all - you're just telling the hub about one more place to deliver to.

Once events leave the hub, each route moves independently. If one sink slows down - a warehouse under load, a flaky network - that route's retention grows to cover it, but the other routes keep flowing at full speed. A slow sink can pin how much history the hub has to hold; it can't stall a fast one.

Between every source and its routes sits the hub - a durable log that buffers change events so a slow or offline sink can never make you lose data. The source writes once; each route reads from the hub at its own pace, whenever it's ready.

That durability is also what makes replay possible: if a sink falls behind or restarts, it picks back up from where it left off in the log instead of from scratch. Nothing is dropped while it catches up.

Delivery is at-least-once. Paired with a sink that safely absorbs a duplicate write, that adds up to effectively-once delivery for that target: no event is ever silently lost, and a retry lands safely instead of corrupting the destination.

A profile is a reusable, named connector configuration - the credentials and connection details for one particular Postgres instance or Kafka cluster, say. Define it once, then reference it by name from any pipeline instead of repeating a DSN or broker list everywhere.

Placement is where a connector instance actually runs. Two choices, and you can mix them freely across one pipeline:

Most people start in-process and only reach for external placement when they want a connector isolated on its own host - for resource limits, network segmentation, or just to keep it out of the runtime's blast radius.

Credentials never go in your specs. In the very first pipeline you wrote, the source's password came in through secret_refs, not as plain text. That's the rule everywhere: secret_refs points a config field - say password - at a named secret, and the value itself is never written inline.

The runtime resolves that reference locally, from secret material on the node, at the moment the connector runs. Nothing sensitive travels in the spec you apply or lives in version control.

Because a profile can carry secret_refs too, one shared profile can keep a single set of credentials out of every pipeline that references it.

Everything dials into the runtime - the runtime never calls out to a connector, even for sinks. That makes whitelisting simple: open these ports inbound, and nothing else.

Notice external sinks have no fixed port to open: the runtime automatically provisions a dedicated, isolated data channel per connector instance and advertises it to the connector over the control channel - the connector just dials in. That's a feature, not a gap: zero-config, self-resolving data channels mean you can run many sink connectors on a single host with no port to pick and nothing to collide.

Here is the whole system in one picture: a single self-hosted runtime holds the control plane and the engine, with connectors attaching over the data channel and your two ways in - the terminal and the browser - on top. Tap any box to see what it does.

One thing worth calling out: the zipline CLI and the Web Console are both thin clients of the same control plane, in-process inside your runtime - there's no separate hosted service behind the console. The Console is served by the Gateway: it handles browser auth, then forwards your clicks to the exact same API the CLI calls. Anything you can do with zipline pipeline apply, you can do from the Console, and the reverse.

zipline pipeline status <name> is the first command to reach for - it shows you whether a pipeline is actually healthy, not just whether you applied it:

zipline pipeline status orders

NAME     STATE     INSTANCE     INCIDENTS
orders   running   src-7f3a     0

Two numbers matter more than any other: freshness and lag - how far behind the source your sink currently is, in plain wall-clock time. A pipeline that's keeping up shows a lag of a second or two; a lag that keeps climbing and doesn't come back down is your first sign something downstream can't keep pace.

A pipeline - or a single slot within it - is always in one of three health states:

Running means delivery is flowing normally. Paused means an operator deliberately held it - see Pause vs disable vs unbind. Degraded means it's still delivering, but something's wrong - a sink erroring, lag climbing, an incident open. Degraded is your cue to look at status in detail before it turns into an outage.

Four verbs stop or steer delivery - they differ in whether resuming leaves a gap in what gets delivered.

Sometimes a single event just won't deliver - malformed, too large, or the sink rejects it outright. Zipline gives you two explicit, deliberate ways to move past it. Neither happens automatically; you choose.

A dead-letter queue (DLQ) diverts an undeliverable event to a side sink instead of blocking the route behind it - you keep the event, just off to the side, so you can inspect and reprocess it later.

skip is blunter: it deliberately drops exactly one corrupt or stuck frame so the route's cursor can advance past it. Reach for it when a frame is truly unrecoverable and a DLQ isn't set up - it's a last resort, not a first response.

Every connection in Zipline - CLI, Console, every connector instance - is mutual TLS. Nothing talks to the runtime without a certificate, and the runtime never accepts a plaintext command.

Getting that first certificate goes through an enroll token: an admin mints one, and it's good for exactly one redemption. Once an instance enrolls with it, the token is spent - it can't be reused. Its role is fixed by the certificate authority at enrollment time, not chosen by whoever holds it, so a token minted for a connector stays a connector: it can never escalate itself to operator access.

On the client side, a context is just the CLI's saved pointer to a runtime address and a certificate directory - one context per environment (dev, staging, prod) so you're never retyping either on every command, and never one typo away from running a command against the wrong cluster.

Credentials themselves are kept out of your specs entirely - see Secrets.

Next steps