Job Summary

We are seeking a highly skilled Senior Data Engineer with expertise in Databricks and Microsoft Data Platform technologies. The ideal candidate should have extensive experience in designing enterprise-scale data solutions, developing data pipelines, and implementing modern data architectures using Databricks and Microsoft technologies such as Azure Data Factory (ADF), Azure Synapse Analytics, or Microsoft Fabric

Key Responsibilities

• Design and implement scalable data architectures and data pipelines. 

• Develop and optimize ETL/ELT solutions using Databricks. 

• Build data ingestion frameworks for batch and real-time processing. 

• Lead data integration projects using Azure Data Factory, Synapse Analytics, or Microsoft Fabric. 

• Implement Delta Lake and Lakehouse architectures. 

• Optimize Spark workloads and ensure efficient resource utilization. 

• Collaborate with Solution Architects, Business Analysts, and Data Scientists. 

• Define and enforce data governance, security, and quality standards. 

• Mentor junior data engineers and conduct technical reviews. 

• Support CI/CD implementation and DevOps practices for data platforms.

Required Skills & Qualifications

• 5+ years of experience in Data Engineering. 

• Mandatory hands-on experience with Databricks. 

• Strong expertise in Python and PySpark. 

• Experience with at least one Microsoft data platform: Azure Data Factory (ADF), or Azure Synapse Analytics, or Microsoft Fabric

• Strong SQL and data modeling experience.

• Experience with Delta Lake and Lakehouse architecture.

• Experience working with Azure cloud services.

• Knowledge of data warehousing concepts and dimensional modeling.

• Familiarity with Git, Azure DevOps, and CI/CD practices.

Preferred Qualifications

• Experience with real-time streaming using Spark Structured Streaming.

• Knowledge of Power BI and semantic models.

• Experience with data governance and security frameworks.

• Exposure to Snowflake, AWS, or other cloud data platforms.