Senior Data Engineer

Job Summary

We are seeking a highly skilled Senior Data Engineer with 7-10 years of experience in designing, developing, and managing enterprise-scale data platforms and data warehousing solutions. The ideal candidate will have extensive experience in building scalable ETL/ELT pipelines, architecting cloud-based data solutions on AWS, and leading teams in delivering reliable, secure, and high-performance data ecosystems.

The role requires strong expertise in Data Warehousing, Big Data technologies, AWS Data Services, Data Modeling, Performance Optimization, and People Management within a fast-paced banking and financial services environment.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines for batch and real-time data processing.
• Architect end-to-end ETL/ELT solutions for data ingestion, transformation, and consumption across enterprise systems.
• Build and optimize cloud-native data platforms using AWS services such as S3, EMR, Redshift, Glue, Athena, and Airflow.
• Define and implement enterprise data strategies, including data sourcing, data flow, storage, governance, and consumption frameworks.
• Collaborate with Data Analytics, Business Intelligence, Product, and Technology teams to understand and fulfill data requirements.
• Design and implement scalable data models supporting reporting, analytics, and business intelligence initiatives.
• Monitor, troubleshoot, and support production data pipelines to ensure high availability and reliability.
• Lead performance tuning initiatives for complex SQL queries, ETL jobs, and large-scale data processing workloads.
• Ensure data quality, validation, governance, security, and compliance standards are adhered to.
• Drive cloud cost optimization initiatives while maintaining performance and user experience.
• Present technical architecture, solutions, and recommendations to business and technology stakeholders.
• Mentor junior engineers and coordinate tasks across project teams.
• Prepare and maintain technical documentation, architecture diagrams, and operational runbooks.

Required Technical Skills Data Engineering & Data Warehousing

• Enterprise Data Warehousing Concepts
• Data Lake & Data Lakehouse Architecture
• ETL/ELT Design and Development
• Data Integration and Data Migration
• Data Quality & Validation Frameworks

Programming & Big Data

• Python
• Scala
• Apache Spark (PySpark/Spark SQL)
• SQL Query Optimization
• Distributed Data Processing

AWS Data Stack

• Amazon S3
• Amazon EMR
• Amazon Redshift
• AWS Glue
• Amazon Athena
• Apache Airflow
• IAM & AWS Security Best Practices

Data Modeling & Architecture

• Dimensional Data Modeling
• Star Schema & Snowflake Schema
• Data Architecture Design
• Metadata Management
• Data Governance Frameworks

Scheduling & Orchestration

• Apache Airflow
• Enterprise Job Scheduling Frameworks
• Workflow Automation

Monitoring & Production Support

• Data Pipeline Monitoring
• Incident Management
• Root Cause Analysis
• SLA Management

Security

• Data Security Principles
• Data Encryption & Access Controls
• Regulatory and Compliance Awareness

Experience Requirements

• Minimum 7+ years of overall Data Engineering experience .
• Strong experience in Enterprise Data Warehousing (7+ years) .
• Hands-on expertise in building scalable ETL pipelines using Spark, Scala, and Python (7+ years) .
• Strong SQL development and performance tuning experience.
• Experience in Data Modeling, Data Architecture, and Data System Design.
• Extensive experience working with AWS Data Services including EMR, Redshift, S3, Athena, Glue, and Airflow.
• Experience supporting production environments and handling critical incidents.
• Exposure to banking, financial services, fintech, or large enterprise environments preferred.

Leadership & Soft Skills

• Experience leading and mentoring data engineering teams.
• Strong stakeholder management and communication skills.
• Ability to present complex technical solutions to leadership and cross-functional teams.
• Strong analytical and problem-solving capabilities.
• Excellent documentation and technical writing skills.
• Ability to manage multiple priorities and deliver under tight timelines.

Preferred Qualifications

• BE/B.Tech/M.Tech from reputed Tier-1 Institutes.
• AWS Certifications (Solutions Architect, Data Engineer, Developer Associate, etc.).
• Certifications in Spark, Big Data, or Cloud Technologies will be an added advantage.