Spotify End-to-End ETL 🎵📈👷

This Data Engineering project consists of using the Spotify API to access both daily top 50 songs, and the songs I have played each day.

👷‍♂️ Made with:

Python
Airflow
Docker
PostgreSQL
AWS

🏆Main challenges overcomed include:

Dockerizing the application's components with docker compose for local development
Validating and transforming the data
Managing differences in timezones (local vs spotify's default)
Deploying in the cloud

📌AWS Diagram:

Data Analysis:

Link to notebook inside this repo