Connectors

Postgres

source sink
Available

Stream Postgres changes via logical replication, or land changes into a Postgres table.

Start streaming
DirectionSource + Sink
PlacementRuntime or external
DeliveryEffectively-once (idempotent sink)
CategoryCDC

What it does

Postgres is a dual-direction connector. As a source it streams change data capture off the database’s logical replication: every insert, update, and delete becomes an ordered change row. As a sink it upserts those rows into a target table; because the upsert is idempotent, delivery is effectively-once.

How it works

  1. 1

    Reads your Postgres logical-replication stream from a publication and a slot it owns.

  2. 2

    Turns every insert, update, and delete into a change row, in commit order.

  3. 3

    Routes each row to any sink you configure, ordered per key.

Use cases

  • Keep a read replica, warehouse, or search index continuously in sync with your primary Postgres.
  • Run a zero-downtime migration by streaming changes into the new database.
  • Fan out OLTP changes to Kafka, files, or other stores for event-driven consumers.

Highlights

Dual direction

One connector: use Postgres as a CDC source or an upsert sink.

Flexible placement

Run it inside the Zipline runtime or as an external process.

Idempotent sink

Effectively-once delivery, safe to retry, upserts never double-apply.

Prerequisites

  • Set wal_level = logical
  • Create a publication for the source (the connector creates and owns the replication slot: you just name it via slot_name)
  • Sink writes into a pre-existing table with a primary key

Configure it

YAML
source:
  type: postgres
  placement: external
  config:
    host: orders-db
    port: 5432
    database: orders
    user: zipline
    publication: zipline_pub
    slot_name: zipline_orders
  secret_refs:
    - { field: password, ref: orders_db_password }

source config

FieldTypeDefaultReqDescription
host string - req Postgres host to connect to.
port int - req Postgres port.
database string - req Database name.
user string - req Connecting user.
password string - req secret Connecting user’s password.
publication string - req Publication to read changes from.
slot_name string - req Replication slot to read from.
ssl_mode string prefer opt TLS mode for the connection.

sink config

FieldTypeDefaultReqDescription
host string - req Postgres host to connect to.
port int 5432 opt Postgres port.
database string - req Database name.
user string - req Connecting user.
password string - opt secret Connecting user’s password.
ssl_mode string prefer opt TLS mode for the connection.

The sink upserts into a table and primary key that already exist in the destination - Zipline does not create schema.

Pairs well with

SQL Server Kafka File

Learn more

Start streaming with Postgres

Follow the first-pipeline guide to wire Postgres into a running pipeline, or jump to the full config reference above.

Start streaming All connectors