Lakehouse & Iceberg · object layout

The table your engine already reads.

Write change data straight into S3, Iceberg, Parquet, Delta Lake, GCS, or Azure Data Lake, so the table format your query engine already reads is the one Zipline keeps current.

Sheet 01

What lands, and where

Changes arrive as files under the prefix you name. Nothing rewrites a partition that is already closed.

Partition layout
s3://lake/orders/
  dt=2026-08-28/
    part-0004.parquet
    part-0005.parquet
  dt=2026-08-29/
    part-0006.parquet
    part-0007.parquet
One prefix per table · one directory per partition key
Sheet 02

File size, on purpose

Small files are the standard way to make a lakehouse slow. Zipline rolls a file at a target size rather than on every flush.

128 MB target file size · rolled on size or on age, whichever comes first
Sheet 03

The table your engine already reads

The layout is the one your table format expects, so nothing downstream changes. Your query engine reads the table it always read.

Different problem · one runtime

This page is orders.yaml.

orders.yaml YAML
name: orders
connectors:
  - name: orders-db
    type: source/mysql
    # required config: /connectors/mysql
  - name: iceberg-out
    type: sink/iceberg
    # required config: /connectors/iceberg
routes:
  - from: orders-db
    to: [iceberg-out]
zipline pipeline apply -f orders.yaml

Not every connector in this pair ships yet. The spec is what it will be · check the roadmap for dates.