Artificial Intelligence Engineer

BlueRose Technologies
India

🚀 We’re Hiring: AI Engineer

📍 Location: Remote – Anywhere in India

💼 Experience: 3-5+ Years

We are looking for an AI Engineer to join our team and help test, evaluate, and improve advanced Agentic AI Systems.

🔹 Key Responsibilities

  • Test, evaluate, and improve the performance and quality of AI agents.
  • Design and execute evaluation datasets, Q&A sets, and multi-turn conversational scenarios.
  • Define and measure agent evaluation metrics, including tool-calling performance, accuracy, reliability, and overall effectiveness.
  • Benchmark agent performance and identify areas for improvement.
  • Analyze large volumes of business documentation and data to quickly understand complex domains.
  • Work on improving agent quality, reliability, and overall user outcomes.

🔹 Required Skills

  • 5+ years of experience in AI/ML, Generative AI, or related fields.
  • Strong proficiency in Python with a solid understanding of APIs and system integrations.
  • Strong knowledge of Generative AI, LLMs, and Agentic Systems.
  • Experience with AI/LLM evaluation, benchmarking, and performance measurement.
  • Strong analytical and problem-solving skills.
  • Ability to work with complex datasets and translate business requirements into effective evaluation scenarios.

📩 Interested? Share your profile/resume with us or apply directly.

Score my resume against this job, free →

Get your ATS score for this role — free. Score my resume free →