I'm a passionate Data Engineer Consultant with over 5 years of experience in the data field, including 3 years specializing in the Azure stack and Databricks. I have worked across diverse industries such as finance, healthcare, and auto auctions, helping organizations unlock the true potential of their data. Currently based in Navi Mumbai, I'm seeking opportunities to create impactful data solutions while staying open to remote and hybrid roles.
- Proficient in Databricks, Azure Data Factory (ADF), and Azure Synapse Analytics
- Expertise in ETL Pipelines, Data Lakehouse, and Medallion Architecture
- Data modeling, transformation, and processing using SQL, PL/SQL, and Delta Lake
- Extensive experience with the Azure stack, including Azure DevOps and Unity Catalog
- Migrated complex workflows such as SSIS to ADF and SSRS reports to Power BI
- Developed dashboards and paginated reports in Power BI
- Skilled in generating ad-hoc reports and working with APIs
- Deployed data pipelines, notebooks, and warehouse objects using Azure DevOps CI/CD
- Migrated data marts from Azure Synapse to a Databricks Lakehouse
- Implemented Medallion Architecture to process Parquet, JSON, and Delta tables
- Leveraged Unity Catalog for data governance
- Orchestrated ETL workflows for over 50 dimensional and fact tables in Azure Synapse
- Ensured robust CI/CD for seamless deployment to production environments
- Designed and developed a new Data Warehouse using ADF, SQL, and Databricks
- Delivered requirement-based and ad-hoc reports
- Improving expertise in Databricks administration, init scripts, Git integration, and advanced Unity Catalog usage.
- Exploring MLFlow and advanced Azure services for scalable analytics solutions.
- LinkedIn: Kalpesh Naik
- GitHub: You're already here! 🚀
- Email: [email protected]
When I'm not optimizing ETL pipelines or architecting data solutions, you might find me exploring new technologies or staying updated on the latest trends in the data world! 😊