You declare a pipeline as YAML and hand it to zipline pipeline apply. That doesn't start anything by itself - it tells the runtime the pipeline you want. The runtime persists that desired spec, and a reconciler keeps converging the running system toward it on its own, until you change the spec again.

A pipeline is one source slot plus one or more route→sink slots. A slot only moves data once a connector instance is bound to it. External connectors enroll first - redeem a token, receive a certificate - and only then can they bind; in-process connectors skip enrollment entirely (see Placement & profiles).

Verbs

Everything you do to a pipeline funnels through zipline pipeline …:

plan is a dry-run diff - see what applying a spec would actually change before you commit to it. bind takes --instance and, for a sink slot, --route; leave out --route and it binds the source instead.

States you'll see

A pipeline is always in one of pending, configured, running, degraded, stalled, or paused. Each slot within it is independently unbound, offline, live, paused, or disabled - a pipeline can read running overall while one of its routes sits disabled.

Four verbs stop delivery. They differ in what happens to the binding, what happens to the WAL, and - the part that actually matters when you're deciding which one to reach for - whether resuming leaves a gap.

The two real advisories: pausing the source holds the whole pipeline in place, so upstream lag grows for as long as it's paused. Disabling the last live sink on a pipeline has no live route consuming it, so the durable log keeps growing exactly as it would under a stalled route - see WAL retention & the safety valve before you reach for disable as a long-lived "off" switch.

One source can fan out to any number of routes, and each route is an independent, ordered consumer with its own cursor. A route that's slow - or paused, or stuck behind a poison event - never stalls a faster sibling route on the same pipeline; they only share the source.

Routes aren't fixed at apply time. Add or remove one from a live pipeline with zipline pipeline route add / route remove.

The baseline is at-least-once: a sink may see the same event more than once, most commonly right after a restart or a replay. Zipline doesn't hide that from you - it hands you a durability story that's honest about what it actually is.

Layer an idempotent sink on top - one that applies the same write twice without corrupting the result, like an upsert keyed on a stable primary key - and the combination is effectively-once for idempotent sinks. That guarantee comes from the sink's own idempotent apply, not from anything the runtime does on your behalf; Zipline's job is to make sure the duplicate lands, not to suppress it.

Delivery is per-row, not per-transaction. If a source transaction touches several rows, a sink can transiently observe a partial prefix of it before the rest arrives - it converges to the full, correct result on replay, but it isn't atomic in the interim.

commit_mode decides when the source is allowed to consider a batch delivered and let its retention advance:

• on_delivery - the source only advances once every route's sink has confirmed delivery. Simple and safe, but the slowest sink gates upstream retention for every route on the pipeline.
• on_buffer - the source advances as soon as the event is buffered locally, independent of any sink. That releases upstream pressure fast, but it does not bound the local WAL - a slow sink still backs up, just inside Zipline instead of at the source.

One limit holds under either mode: an open transaction upstream pins the log until it commits or rolls back. There's no commit_mode setting that gets you out from under a transaction the source hasn't closed yet.

A connector runs in one of two places. runtime placement means in-process - no enroll, no bind, it's just part of the hub. external placement means a separate process that enrolls and binds like any other connector - and it's the only option for a custom connector you've written yourself, or for MSSQL.

When a connector actually supports both, placement: is a required field on that slot - there's no silent default one way or the other. You have to say which one you mean.

Moving a live connector between hosts, or between in-process and external, doesn't have to cost you a resync. zipline pipeline swap stage starts the new binding alongside the old one, keeping the same cursor; watch pipeline status for the staged instance to report ready, then swap commit to cut over - or swap cancel to back out and keep the original binding live the whole time.

A profile is a reusable, named connector configuration - define credentials and connection details once, then reference them by name from any slot instead of repeating them. One gotcha carries over from the Reference: a source profile can't be shared across more than one pipeline (see Reference → Profiles & secrets).

The hub - the ziplined process - is the runtime. It owns the durable log, orders and durably records every event, hosts in-process connectors directly, and embeds the control plane the CLI and console both talk to. There's no separate scheduler or coordinator process sitting in front of it: you edit one spec, and the hub's own reconciler converges the running system to match.

State it plainly: the hub is single-node today. There's no clustering and no leader election yet - one ziplined process is the whole control plane and the whole data path for everything bound to it. Plan your deployment around that fact rather than around a HA model that doesn't exist yet (more in HA & failover).

Four connectors ship today. Each has a fixed direction, and one of them - MSSQL - has a fixed placement too.

That's the closed set that ships with Zipline. If you need something else, the connector interface is open - operators can build and register a custom connector with zipline connector … rather than waiting on the catalog to grow.

Four surfaces, from scriptable to visual. The web console's gateway exposes /healthz, a plain HTTP health check in front of the browser UI. The runtime itself has no HTTP health endpoint - its health is surfaced through the CLI instead: zipline status, pipeline status <name>, and instance list all give you a live read without leaving your terminal. And for a persistent view, there's the web console itself on :8080. If you would rather stay in the terminal, there are two options, and they are not the same thing. zipline tui is the operator cockpit: it connects to the control plane and covers every pipeline and instance at once. ziplined run --tui belongs to a single runtime daemon, showing that one process monitoring itself, with its logs redirected to zipline-ziplined.log.

When something's actually wrong, it surfaces as an incident at one of three severities:

Scale is horizontal on connectors, not on the hub - run external zipline-connector processes on however many other hosts you need, each enrolled and bound to its own slot. Need to move a running one without losing its place? pipeline swap migrates a live route to a new instance with no cursor loss (see Placement & profiles).

The hub itself protects against being overrun with two node-level limits: runtime.max_pipelines (default 256) caps how many pipelines it will run concurrently, and an in-flight memory budget caps how much unacknowledged data it will hold in memory at once.

Said plainly, twice, because it shapes everything else in this section: the hub is single-node today. No clustering, no leader election, no hub-level replication. If the node running ziplined goes down, nothing is standing by to take over automatically.

What you do get is real crash recovery: on restart, the runtime replays the un-acknowledged tail of its durable log and resumes every pipeline from its last persisted cursor. If it finds its own local state corrupted rather than merely behind, it fails closed rather than guessing - it won't start serving a pipeline against state it can't trust. External connectors reconnect on their own with backoff once the hub is back.

A primary/replica MSSQL setup is read-offload, not failover: pointing reads at a replica reduces load on the primary, but there is no automatic replica-to-primary promotion built in. If the primary goes away, that's a manual database-level failover, not something Zipline does for you.

There's no dedicated upgrade runbook today - no blue/green rollout tooling, no rolling upgrade across a cluster (there is no cluster; see HA & failover). What you have instead are the same real primitives everything else in this page is built from, and they're enough to upgrade safely by hand:

1. Send SIGINT or SIGTERM to ziplined - it drains gracefully rather than dying mid-write.
2. Replace the binary.
3. Start it again.
4. The runtime replays its durable log from where it left off, exactly like it would after a crash.

Any duplicate delivery right at that boundary is the same at-least-once behavior described in Delivery guarantees - an idempotent sink absorbs it the same way it absorbs any other retry.

Two distinct mechanisms handle events that won't go through cleanly - reach for the wrong one and you'll either lose data you didn't need to, or stay stalled longer than you needed to.

DLQ - sink-side poison

A route can declare a dead-letter dlq: sink. When a batch fails delivery to the primary sink, it diverts to the DLQ instead - and once it's there, the route's cursor advances, so one bad batch doesn't block everything behind it.

dlq_failure decides what happens if the DLQ itself fails to accept the batch:

• halt_route (default) - the route stalls, the WAL is retained, and a Critical incident opens. Nothing is lost; you have to intervene.
• drop_and_continue - the batch is dropped and the cursor advances anyway, with a Critical incident opening so you know it happened. This one is data loss, on purpose, in exchange for the route not stalling.

Skip - corrupt frame

Sometimes the problem isn't the sink rejecting a valid event - it's a frame in the log that can't be decoded at all. That's not DLQ-able; there's nothing coherent to divert. The route simply stalls and waits for a human. zipline pipeline skip <name> <route> (or --all for every route) tells it to accept the loss of exactly that one frame and move on.

Both the control plane and the data plane run on mutual TLS - every CLI command, every console request, every connector's data channel presents a certificate and gets one checked in return. The one exception is the enroll plane, which is necessarily server-auth only (a brand-new connector has no certificate yet) and instead gated by a single-use token.

Certificates come from an internal CA the runtime keeps at <wal.dir>/ca - a long-lived root signing short-lived leaf certificates. The CA's fingerprint is pinned into every enroll token, so a connector can verify it's talking to the right hub without any certificate being pre-distributed to it first.

Enroll tokens are single-use: an admin mints one, and it's burned the moment it's redeemed - before the certificate is even signed, so a race to reuse it loses. The role baked into the resulting certificate (operator, sink, source, or gateway) is decided server-side at enrollment time, never by whatever the token holder claims - a connector's token can't talk its way into operator access. Certificates renew automatically while a session is active, so you're not manually rotating leaves on a timer.

Notice the direction column: everything dials the hub. Connectors are always the client, never a server the hub connects out to, and they reconnect on their own if the connection drops - so the only firewall rule you ever need for Zipline is inbound to: the three fixed connector planes above (Control, Source data, Enroll), plus the auto-assigned per-instance data channel for each external sink. (The Console port is browser-facing, not a connector plane.)

This is the sharpest edge in the whole system: a sink that's stuck under on_delivery prevents the source's retention from advancing, and the local durable log keeps growing behind it. Left alone long enough, it fills the disk.

Zipline doesn't let that run unbounded. There's a graduated safety valve on two limits - buffer_max_mb (default 4 GiB per pipeline) and free_disk_reserve_mb (default 10 GiB node-wide free-disk floor):

• Warn - an incident opens so you have advance notice.
• High - the slowest route's reader is paused; every other, healthy route on the same pipeline keeps flowing, and the source gets backpressure so it stops making things worse.
• Critical - the offending route is diverted to its DLQ or detached outright, with a loud incident, rather than letting the disk actually fill.

Remember that on_buffer only moves this problem - it releases the source fast, but it still doesn't bound the local log, so the same valve above is what actually catches it.

MSSQL: transaction-log release

MSSQL has an opt-in release-on-advance setting that lets Zipline signal SQL Server it's safe to reclaim transaction-log space it's consumed, so the source's own transaction log doesn't grow unbounded on Zipline's account. It needs db_owner permissions, plus CDC or replication publications already set up on the tables in question. Thresholds warn at 70% and go critical at 90% of log usage; it can't run at the same time as the SQL Server Log Reader Agent; and a long-open transaction on the source blocks reclaim entirely - nothing can be reclaimed past a transaction that hasn't closed. See Source prerequisites & cliffs.

Postgres: replication slot lag

A Postgres replication slot's lag warns at ≥50% and goes critical at ≥80% of its configured limit. The sharper failure mode: a stopped pipeline leaves its slot behind - Postgres keeps holding onto WAL for a slot nobody is draining, and it fills the source's own disk, not Zipline's. Check pg_replication_slots to find an orphaned slot, and pg_drop_replication_slot to remove one you've confirmed is no longer in use.

MSSQL: CDC / supplemental logging

An INSERT or DELETE always carries its full row, CDC or not. An UPDATE is where it matters: without change data capture enabled, SQL Server's log only records the changed byte range for that row - which is enough for most column types, but can decode wrong for DECIMAL, MONEY, and NUMERIC columns specifically. If you need accurate UPDATE values on those types, enable CDC on the tables that hold them. Expect roughly 1.5–2.5× log growth on tables where you turn it on - that's the real cost of the guarantee.

Postgres: REPLICA IDENTITY and the TOAST cliff

Postgres logical replication needs REPLICA IDENTITY DEFAULT plus a primary key on every table you're capturing - Zipline refuses a keyless table outright rather than silently capturing partial identity.

The sharper cliff is REPLICA IDENTITY FULL: if a table has a large, off-page (TOASTed) column that goes unchanged in a given UPDATE, that UPDATE FATAL-stops the stream right there - Postgres doesn't include the unchanged TOASTed value in the change record, and FULL's stricter identity requirements can't tolerate that gap. If you're on FULL, either exclude TOAST-prone columns from the publication, or plan to re-bootstrap the pipeline after you fix the schema.

Next steps