Senior Data Platform Engineer
SoTalent
Bangalore Urban, Karnataka, India
Senior Data Platform Engineer – Cloud & DevOps
Location: Bangalore, Karnataka, India
Employment Type: Full-time
Work Model: Hybrid
Overview
Senior Data Platform Engineer responsible for building, automating and operating enterprise-scale data platforms across Databricks, Snowflake, Microsoft Fabric and Kubernetes. The role focuses on platform engineering, DevOps automation, reliability, security, observability and self-service enablement.
Key Responsibilities
- Design and support enterprise data platforms across Databricks, Snowflake and Microsoft Fabric.
- Build reusable capabilities for platform onboarding, provisioning, monitoring and governance.
- Manage Kubernetes-based data processing workloads and containerised services.
- Implement high availability, auto-scaling, disaster recovery and resilience.
- Develop CI/CD pipelines and automate platform provisioning and deployments.
- Implement GitOps and Infrastructure as Code using Terraform, Bicep and Helm.
- Establish monitoring, observability and alerting across platform components.
- Troubleshoot production incidents and drive root-cause analysis and remediation.
- Implement RBAC, secrets management, network security and governance controls.
- Support metadata, auditing, lineage and compliance requirements.
- Optimise platform performance, capacity and cloud costs using FinOps practices.Requirements
- Bachelor’s degree in Computer Science, Information Systems, Engineering or related field.
- 5–7 years of experience in Platform Engineering, DevOps, Cloud Engineering or Data Platform Operations.
- Hands-on experience with Databricks, Snowflake or Microsoft Fabric.
- Production Kubernetes and container management experience.
- Strong CI/CD, Infrastructure as Code and cloud-native architecture experience.
- Strong scripting skills using Python, PowerShell, Bash or similar.
- Experience with Agile and DevOps delivery models.Technical Skills
- Data Platforms: Databricks, Snowflake, Microsoft Fabric, Delta Lake, Data Lakehouse
- Kubernetes: Kubernetes, Docker, Helm, Kustomize
- DevOps: Azure DevOps, GitHub Actions, Jenkins, Git, CI/CD
- Infrastructure as Code: Terraform, Bicep, ARM Templates
- Cloud: Microsoft Azure; AWS/GCP exposure
- Observability: Prometheus, Grafana, Azure Monitor, Log Analytics, OpenTelemetry
- Security: RBAC, Entra ID, Key Vault, secrets management
- Programming: Python, SQL, PowerShell, BashPreferred
- Databricks, SnowPro, Microsoft Fabric, CKA or Terraform certifications.
- Experience with Apache Airflow, Kafka, Spark on Kubernetes, Kyuubi, ArgoCD, GitOps, service mesh or platform SRE practices.