Technical Lead – AI & Data Engineering

Job Title: Technical Lead – AI & Data Engineering Location: Hybrid – Ahmedabad Office / Remote Department: Data Engineering & AI Experience Level: 7+ Years (with at least 2+ years leading/managing technical teams)

About the Role We are looking for an experienced, hands-on Technical Lead – AI & Data Engineering to spearhead our data and AI initiatives from initial requirement gathering through to production deployment and monitoring. In this role, you will lead a cross functional team of data engineers and AI/ML developers, serving as the technical anchor for end-to-end delivery. You will drive architectural decisions across robust ETL/ELT pipelines, relational databases (PostgreSQL, SQL Server), and Power BI semantic models (DAX), while designing enterprise-grade AI and Generative AI systems (RAG workflows, vector search, Azure OpenAI). The ideal candidate pairs deep technical mastery with proven team-handling experience, translating business challenges into production-ready software with strict performance, cost, and reliability benchmarks.

Responsibilities

• Key Responsibilities
• 1. End-to-End Solution Architecture & SDLC Ownership
• Own the entire engineering lifecycle: lead requirement discovery, translate business requirements into technical roadmaps, design high- and low-level architectures, and oversee deployment to production.
• Interface directly with product owners, stakeholders, and business analysts to clarify functional expectations, identify edge cases, and define acceptance criteria.
• Establish engineering standards, production deployment strategies (blue/green, canary), rollback procedures, and post-launch maintenance workflows.
• • 2. Technical Leadership & Team Management
• Lead, mentor, and coach a team of junior-to-mid-level AI and data engineers, conducting 1-on-1s, technical goal setting, and skills development.
• Own resource planning, sprint estimation, sprint planning, task delegation, and delivery tracking in an Agile framework.
• Enforce rigorous code quality, schema management, and architectural integrity through strict PR reviews, testing standards, and Git workflows.
• • 3. Data Engineering & Pipeline Governance
• Architect, automate, and govern resilient ETL/ELT data pipelines using Python and Azure cloud services.
• Direct data cleansing, validation, normalization, and semantic data layer development to provide clean data for downstream analytics and AI.
• Guide database modelling, stored procedure optimization, indexing strategies, and execution plan analysis across PostgreSQL and SQL Server (SSMS).
• Oversee programmatic extraction and tabular aggregation directly from Power BI reports and semantic models using DAX.
• • 4. AI, GenAI & Model Performance Engineering
• Lead the design and integration of AI/GenAI architectures (RAG systems, prompt orchestration, vector search integrations with Azure OpenAI).
• Drive inference optimization strategies to maintain low latency and compute expenses: token usage control, caching, smart chunking, ONNX runtime conversion, quantization, batching, and hardware profiling (CPU/GPU).
• Implement enterprise MLOps/LLMOps monitoring for staging and production, tracking model throughput, latency, accuracy, regression, and data drift.
• • 5. CI/CD, DevOps & Operational Excellence
• Own production deployment automation via robust CI/CD pipelines (GitHub Actions or Azure DevOps).
• Define logging, telemetry, anomaly detection, and alerting standards to minimize production downtime and pipeline breakages.
• Qualifications
• Required Qualifications & Technical Skills
• Experience: 7+ years of hands-on technical experience across data engineering, analytics engineering, or applied AI/ML systems, including 2+ years of formal leadership or team management experience.
• Full Lifecycle Delivery: Proven track record of owning projects from initial requirement discovery and architecture design through deployment, production rollout, and operational monitoring.
• Required Skills
• Databases & Querying:
• Advanced proficiency in PostgreSQL and SQL Server Management Studio (SSMS), including database design, query optimization, indexing, and deep execution plan profiling.
• Strong working proficiency in writing and optimizing DAX queries against Power BI datasets and tabular models.
• • Programming & Pipeline Engineering:
• Expert-level Python skills (Pandas, Pytorch,NumPy, SQLAlchemy, PySpark/data manipulation frameworks).
• Production experience building automated, scheduled data pipelines (Azure Data Factory, orchestrators).
• • AI, GenAI & Performance Optimization:
• Hands-on experience architecting GenAI workflows (RAG, embeddings, vector databases, Azure OpenAI).
• Proven experience with model runtime acceleration, caching, quantization, and profiling techniques (e.g., ONNX Runtime, GPU/CPU profiling).
• • Statistical Rigor & Data Quality:
• Strong grasp of applied statistics, sampling, hypothesis testing, anomaly detection, and ML evaluation metrics (precision, recall, MSE, perplexity).
• • Cloud & DevOps:
• Hands-on expertise with the Azure ecosystem (Azure Data Factory, Blob Storage / ADLS, Azure OpenAI, Azure Keyault).
• Advanced Git/GitHub management, branching strategy enforcement, and automated CI/CD pipeline management (Azure DevOps / GitHub Actions).
• Preferred Skills
• Advanced hands-on experience in Generative AI and Agentic AI architectures, including multi-agent orchestration, tool calling, and autonomous workflow design.
• Experience with LLM frameworks and vector search systems such as LangChain, LlamaIndex, or Azure AI Search.
• Strong backend and API development expertise using FastAPI and Node.js, with comprehensive API documentation and