Lakehouse Platform Engineering: Lakeflow

Lakeflow Architecture — Declarative Thinking, Pipeline Types & Execution Model

Build your first Lakeflow pipeline — define Bronze, Silver & Gold layers declaratively

Migrate an existing PySpark ETL notebook to a Lakeflow declarative pipeline

Configure pipeline modes — triggered vs continuous — for different SLA requirements

Data Quality with Expectations — Constraints, Warnings, Failures & Quarantine Patterns

Define multi-layer expectations across Bronze, Silver and Gold in a single pipeline

A pipeline passes DQ checks despite null primary keys — debug broken expectation logic

Implement quarantine pattern to isolate invalid records without halting the pipeline