Spotify End-to-End ETL 🎵📈👷

This Data Engineering project consists of using the Spotify API to access both daily top 50 songs, and the songs I have played each day.

👷‍♂️ Made with:

• Python
• Airflow
• Docker
• PostgreSQL
• AWS

🏆Main challenges overcomed include:

• Dockerizing the application's components with docker compose for local development
• Validating and transforming the data
• Managing differences in timezones (local vs spotify's default)
• Deploying in the cloud

📌AWS Diagram:

Data Analysis:

Link to notebook inside this repo