Job Summary

We are looking for a Data Engineer with 3+ years of experience in designing, developing, and maintaining scalable data pipelines. The ideal candidate should have strong hands-on experience in Python, PySpark, and Databricks for building modern data engineering solutions.

Key Responsibilities

• Develop, maintain, and optimize ETL/ELT data pipelines.

• Build scalable data processing solutions using Python and PySpark.

• Design and implement data workflows in Databricks.

• Perform data ingestion, transformation, and validation from multiple data sources.

• Optimize Spark jobs for performance and cost efficiency.

• Work with structured and unstructured datasets.

• Collaborate with business analysts, data architects, and stakeholders to understand data requirements.

• Ensure data quality, governance, and security standards are maintained.

• Troubleshoot and resolve data pipeline issues.

Required Skills & Qualifications

• 3+ years of experience in Data Engineering.

• Minimum 2 years of hands-on experience with Python.

• Strong experience with PySpark.

• Mandatory experience with Databricks.

• Experience with SQL and database concepts.

• Understanding of ETL/ELT processes and data warehousing concepts.

• Experience working with cloud-based data platforms.

• Knowledge of Git and CI/CD practices.

Preferred Skills

• Experience with Azure Data Lake, AWS S3, or similar storage platforms.

• Exposure to Delta Lake.

• Knowledge of data modeling and performance tuning.

• Familiarity with Azure services.