ETL Automation - Apache Airflow
ETL Automation - Apache Airflow
ETL Automation - Apache Airflow
Tune Protect Group
1 month
Automation
Tune Protect Group
1 month
Automation
Tune Protect Group
1 month
Automation
Colaborators
Project Overview
The historical table in MYSQL databases is updated manually which consumes a lot of time. Building an automated ETL Pipeline ensure efficiency in detecting new data entries in MSSQL and performing ETL before loading into MYSQL Historical Tables.
Objective
Automate ETL pipeline for Tune Protect EMEIA & Tune Protect Re.
Monitor the Pipeline to ensure each task runs without error.
Analyze the query performance of each table update.
Skills & Tools Used
Project Overview
The historical table in MYSQL databases is updated manually which consumes a lot of time. Building an automated ETL Pipeline ensure efficiency in detecting new data entries in MSSQL and performing ETL before loading into MYSQL Historical Tables.
Objective
Automate ETL pipeline for Tune Protect EMEIA & Tune Protect Re.
Monitor the Pipeline to ensure each task runs without error.
Analyze the query performance of each table update.
Skills & Tools Used
Project Overview
The historical table in MYSQL databases is updated manually which consumes a lot of time. Building an automated ETL Pipeline ensure efficiency in detecting new data entries in MSSQL and performing ETL before loading into MYSQL Historical Tables.
Objective
Automate ETL pipeline for Tune Protect EMEIA & Tune Protect Re.
Monitor the Pipeline to ensure each task runs without error.
Analyze the query performance of each table update.
Skills & Tools Used
Outcomes
Designed DAGs with a total of six tasks each, including two grouped tasks per DAG. The DAGs performed status and data checks before further transformation.
Other Projects
Available for Work
Available for Work