Artificial Intelligence Engineer
BlueRose Technologies
India
🚀 We’re Hiring: AI Engineer
📍 Location: Remote – Anywhere in India
💼 Experience: 3-5+ Years
We are looking for an AI Engineer to join our team and help test, evaluate, and improve advanced Agentic AI Systems.
🔹 Key Responsibilities
- Test, evaluate, and improve the performance and quality of AI agents.
- Design and execute evaluation datasets, Q&A sets, and multi-turn conversational scenarios.
- Define and measure agent evaluation metrics, including tool-calling performance, accuracy, reliability, and overall effectiveness.
- Benchmark agent performance and identify areas for improvement.
- Analyze large volumes of business documentation and data to quickly understand complex domains.
- Work on improving agent quality, reliability, and overall user outcomes.
🔹 Required Skills
- 5+ years of experience in AI/ML, Generative AI, or related fields.
- Strong proficiency in Python with a solid understanding of APIs and system integrations.
- Strong knowledge of Generative AI, LLMs, and Agentic Systems.
- Experience with AI/LLM evaluation, benchmarking, and performance measurement.
- Strong analytical and problem-solving skills.
- Ability to work with complex datasets and translate business requirements into effective evaluation scenarios.
📩 Interested? Share your profile/resume with us or apply directly.