AI Quality and Evaluation Engineer

The Role

We are looking for an AI Quality and Evaluation Engineer to ensure our AI solutions are safe, reliable and effective. You will design and build evaluation frameworks, test datasets and automated processes to measure and improve the performance, quality and safety of AI systems, working closely with engineering, product and client teams.

ONLY CANDIDATES BASED IN BANGALORE SHOULD APPLY.

What You Will Do

• Design and execute evaluation frameworks for LLMs and AI applications
• Create and curate test datasets and benchmarks for different use cases
• Build automated evaluation pipelines and monitoring systems
• Analyse model outputs for accuracy, safety, bias and reliability
• Work with product and engineering teams to improve model performance
• Stay up to date with industry best practices in AI evaluation and responsible AI

What We Are Looking For

• 2 or more years of experience in software engineering, data science or AI evaluation
• Hands on experience with LLM evaluation tools and frameworks (e.g. LangChain, LlamaIndex, Ragas, DeepEval or similar)
• Strong analytical skills and understanding of evaluation metrics
• Proficiency in Python and data analysis libraries
• Good problem solving skills and a passion for building reliable AI systems

Nice to Have

• Experience with human in the loop evaluation methods
• Familiarity with red teaming, prompt safety and bias testing
• Exposure to MLOps and model monitoring
• Interest in AI ethics, governance and responsible AI

Why Join Us?

Work on real problems

Build AI solutions that make a difference for leading enterprises

Learn and grow

Work with a talented, supportive and diverse team

Make an impact

Help shape the future of AI in the region and beyond

Global exposure

Collaborate across our global team

People first

We value ownership, curiosity and a positive team culture

Interested?

We would love to hear from you. Send your resume to nitasha@koda.co.in

Company Description

KODA is a full-funnel digital marketing agency that specializes in serving growth oriented companies worldwide. Known for challenging conventional approaches, Koda is dedicated to finding innovative, tailored solutions that deliver measurable results. With a unique perspective on modern marketing, the team at Koda combines creativity and strategy to address complex challenges.