Zipline connects your production databases to Kafka, PubSub, Snowflake, BigQuery, and S3 with exactly-once delivery and sub-second latency - no data loss, no duplicates. Apply the spec, then bind it to a runtime - no dashboards, no wizard.
routes:
- from: orders-db
to: [kafka-out] | Item | Part | Dir | Class |
|---|---|---|---|
| 01 | postgres | src · snk | CDC |
| 02 | mssql | src | CDC |
| 03 | kafka | snk | Bus |
| 04 | file | snk | File |
| 27 further parts continued on sheet 2 | |||
name: orders
connectors:
- name: orders-db
type: source/postgres
host: "${POSTGRES_HOST}"
port: "${POSTGRES_PORT}"
database: "${POSTGRES_DATABASE}"
user: "${POSTGRES_USER}"
publication: "${POSTGRES_PUBLICATION}"
slot_name: "${POSTGRES_SLOT_NAME}"
secret_refs:
- { field: password, ref: postgres-password }
- name: kafka-out
type: sink/kafka
brokers: "${KAFKA_BROKERS}"
topic_prefix: "${KAFKA_TOPIC_PREFIX}"
routes:
- from: orders-db
to: [kafka-out] ~ zipline pipeline apply -f orders.yaml✓ applied orders the spec is recorded. nothing streams yet.~ zipline pipeline bind orders --instance src-7f3abinding src-7f3a to orders...✓ bound orders <- src-7f3a state: pending route orders-db → kafka-out · running Connect your first database in under 30 seconds.
No credit card. No lock-in. Apache 2.0 licensed.