.webp&w=3840&q=90)
Industry
retail-and-cpg
Skills
cloud-management
data-understanding
data-storage
batch-etl
programming
data-modelling
data-quality
data-wrangling
code-versioning
git-version-control
approach
Tools
google-cloud
spark
github
airflow
sql
Learning Objectives
Design and implement an ETL/ELT pipeline using Dataproc in GCP, following the Medallion Architecture.
Manage Secure Access and Credentials
Automate Deployment with CI/CD
Perform Unit Testing for Data Pipelines
Orchestrate Data Pipelines Efficiently
Overview
Prerequisites
- Understanding of Google Cloud Platform
- Knowledge of ETL/ELT Processes & Pipeline Management
- Familiarity with Big Query, Dataproc, PySpark & Python
- Basic Knowledge of CI/CD Pipelines
- Experience with Airflow