Lead Data Engineer

Obrimo Technologies

Bengaluru, Karnataka, India

We're Hiring: Technical Anchor – Databricks on AWS

Locations: Mumbai | Bangalore | Hyderabad | Pune

Experience: 8+ Years

Mandatory Experience: 4+ Years of hands-on experience with Databricks on AWS

🔹Role Overview

We are looking for an experienced Technical Anchor to lead the architecture, development, and delivery of our enterprise Data Lakehouse platform.

This role combines technical leadership with hands-on engineering expertise. The ideal candidate will drive technical strategy, design scalable data solutions using Databricks on AWS, and mentor a team of data engineers.

🔹 Key Responsibilities

Design and implement scalable cloud data architectures using Databricks Lakehouse on AWS, following the Medallion Architecture (Bronze, Silver, and Gold layers).
Build and manage robust batch and real-time data pipelines using PySpark, SQL, Delta Lake, and Delta Live Tables.
Implement data governance, access controls, and cross-account integrations using Databricks Unity Catalog.
Ensure compliance with AWS security standards, including IAM, KMS, VPC Peering, and PrivateLink.
Establish engineering best practices, conduct code reviews, and mentor data engineering teams.
Lead technical design discussions and collaborate with product owners and client stakeholders.
Optimize Apache Spark performance using Photon, serverless compute, and pipeline tuning techniques.
Monitor and optimize data processing costs while ensuring performance, scalability, and reliability.

🔹 Required Skills & Qualifications

8+ years of experience in Data Engineering, with at least 4+ years of hands-on experience building production solutions using Databricks on AWS.
Strong expertise in PySpark, Apache Spark, Python, and complex SQL queries.
Extensive knowledge of AWS services, including S3, EMR, Athena, Glue, and CloudWatch.
Experience with Infrastructure as Code (IaC) and cloud automation.
Practical experience implementing and managing Databricks Unity Catalog across multiple environments.
Strong understanding of Medallion Architecture, Delta Lake, and Delta Live Tables.
Proven experience in technical leadership, architecture design, code reviews, and mentoring data engineering teams.
Excellent communication skills with the ability to explain technical concepts to non-technical stakeholders.

🎯 Ideal Candidate: A hands-on technical leader with deep expertise in Databricks on AWS, cloud data architecture, performance optimization, and team leadership.

📩 Interested candidates can share their updated CV along with their current CTC, expected CTC, and notice period.

Score my resume against this job, free →