Senior Data Engineer

Bajaj Finserv

Pune Division, Maharashtra, India

Location Name: Pune Corporate Office - Mantri

Job Purpose

To design and deliver AI-powered Business Intelligence solutions by integrating Databricks Genie, PowerBI Co-Pilot & Semantic Models, and Metric Views on top of a robust end-to-end data engineering stack.

The role focuses on enabling self-service analytics using natural language (GenAI), ensuring business-friendly data consumption, and building governed, scalable, and high-performance data ecosystems across batch and real-time pipelines.

Duties And Responsibilities

Translate business problems into AI-driven BI solutions
Own end-to-end delivery: ingestion ? transformation ? semantic layer ? AI consumption
Design scalable data architecture aligned with lakehouse principles
Ensure data quality, governance, and metric consistency
Collaborate with business teams to onboard them onto Genie-based analytics
Optimize performance of queries, pipelines, and AI responses
Establish best practices for AI in BI (prompting, semantic tuning, governance)
Drive adoption of self-service BI with minimal dependency on tech teams

Key Decisions / Dimensions

Define semantic layer design and metric definitions
Decide GenAI prompting strategies and instruction frameworks
Prioritize data vs AI optimization trade-offs
Handle production issues with RCA and long-term fixes
Drive architectural decisions for lakehouse + AI integration

Major Challenges

Ensuring accuracy and trust in AI-generated insights
Driving adoption of GenAI-based BI over traditional dashboards
Maintaining semantic consistency across multiple tools
Balancing performance, cost, and scalability
Managing dependencies across data engineering, AI, and business teams

Required Qualifications And Experience

Azure Databricks – PySpark, SQL, Delta Lake
PowerBI – CoPilot, Semantic Model
Strong experience in Semantic Modeling & Metrics Layer design
Hands-on with Databricks Genie / GenAI-based BI workflows
Databricks One (Unified BI Experience)
Prompt Engineering / Instruction tuning for AI systems
Python (Pandas, PySpark, FastAPI)
Azure Data Factory (ADF) for ETL pipelines
Strong SQL and data modeling skills

Good to Have

Cosmos DB / MongoDB (NoSQL concepts)
Azure Data Explorer (KQL)

________________________________________

DATA STACK (MANDATORY FOR SCREENING)

SNo Data Platform / Concepts Associated Technologies

1 Databricks Lakehouse PySpark, SQL, Delta Lake

2 AI for BI Databricks Genie, Genie Rooms, Instructions, Agents, PowerBI CoPilot

3 Semantic Layer Semantic Models, Metric Views

4 ETL & Orchestration Azure Data Factory

5 Programming Python, C#/.NET

6 Cloud Platform Azure (Preferred)

7 DevOps CI/CD Pipelines, Git

Score my resume against this job, free →