AWS Databricks Developer / Data Engineer

We are looking for an experience AWS Databricks Developer / Data Engineer with strong hands-on expertise in Databricks, Apache Spark, PySpark, and building scalable ETL/ELT data pipelines. The ideal candidate should have experience in cloud-based data engineering, data modelling, and large-scale data processing.

✅ Primary Skills

🔹 Hands-on experience with Databricks development and large-scale data processing

🔹 Strong expertise in Data Engineering and ETL/ELT pipeline development

🔹 Proficiency in Apache Spark, PySpark, Spark SQL, Python, Scala, and Power BI

🔹 Hands-on experience with Delta Lake, ACID transactions, schema evolution, and performance optimization

🔹 Experience building batch and near-real-time data ingestion pipelines

🔹 Strong understanding of analytical and dimensional data modelling (Star and Snowflake schemas)

🔹 Experience with AWS cloud storage services

🔹 Knowledge of coding standards, documentation, and data engineering best practices

🔹 Ability to collaborate with architects, BI teams, and data science teams

⭐ Secondary Skills

✔️ Exposure to ML/AI workloads and feature engineering on Databricks

✔️ Familiarity with MLflow or similar model-tracking tools

✔️ Experience with Databricks CI/CD and Git-based version control

✔️ Understanding of data governance, security, and compliance

✔️ Familiarity with Agile/Scrum methodologies

✔️ Insurance domain experience is an added advantage