Lead II - Databricks Engineer

UST
Trivandrum, Kerala, India

Role Description

Lead Databricks Developer (Offshore) Looking for 2 strong Lead Databricks Developers with streaming expertise. Key Responsibilities

  • Design and develop end-to-end data pipelines using Databricks (PySpark, Delta Lake).
  • Build real-time streaming ingestion pipelines from Kafka and other event-driven sources.
  • Implement layered data architecture (Bronze, Silver, Gold) to support transformation and reporting. Develop data processing logic for transactional data ingestion, validation, and enrichment. Apply data quality, deduplication, and business rules for healthcare datasets.
  • Support cloud migration from on-premise systems to Databricks + Snowflake ecosystem. Optimize pipeline performance to meet SLA-driven processing timelines. Implement and support CI/CD pipelines and DevOps integration. Collaborate with business and technical stakeholders to deliver scalable solutions. Required Skills
  • Strong expertise in Databricks (Streaming & Batch) Advanced proficiency in PySpark / Spark SQL/Spark Streaming
  • Experience in real-time data streaming (Kafka) Strong understanding of data modeling and ETL/ELT pipeline design
  • Hands-on experience with AWS (S3, Airflow, Lambda) Experience with CI/CD, Git, and deployment automation Domain Experience
  • Experience in Healthcare Payer domain (Claims, Membership, Coverage) Exposure to FHIR / healthcare data standards preferred Understanding of data privacy, security, and governance requirements (PII/HIPAA) Leadership Expectations
  • Operate at a Lead level driving design, development, and delivery Mentor team members and ensure delivery quality Own modules independently and collaborate across distributed teams Experience
  • 8+ years in Data Engineering
  • 3+ years hands-on experience with Databricks Prior experience in Tech Lead/Senior roles Looking for 2 strong Lead Databricks Developers with streaming expertise

Skills

Apache Spark, PySpark, Databricks, Apache Kafka

Score my resume against this job, free →

Questions about this job

How do I apply for the Lead II - Databricks Engineer job at UST?

Use the apply link on this page: you apply directly with UST on LinkedIn. Before you do, score your resume against this job description on Fitboard. It shows which skills and keywords the job asks for that your resume doesn't show, so you can fix them first.

What skills does UST want for this Lead II - Databricks Engineer role?

The job description asks for Databricks, PySpark, Delta Lake, Kafka, AWS, S3, Airflow and Lambda. Make sure your resume shows the ones you have, in the words the job uses: applicant tracking systems look for them.

How can I get a job at UST?

Apply to the UST roles that match your experience, and tailor your resume to each job description: for this role that means showing Databricks, PySpark, Delta Lake. A resume that matches the job's own keywords is more likely to get past screening. Fitboard scores your resume against this exact job for free.

Is this Lead II - Databricks Engineer job remote or onsite?

This is based in Trivandrum, Kerala.

Is it free to apply? Does Fitboard charge anything?

Applying is free: you apply directly with UST, and this job page and its apply link cost nothing. Checking your resume against this job is free too, with 3 free scans when you sign in and no credit card needed.

Will you get shortlisted for this role? Check your match free →