AI Evaluation Specialists

AI Evaluation Specialists – Multiple Positions

Department: AI/LLM Operations

Employment: 3-Month Contract

Location: Noida Sector 62

Experience: 2+ Years, depending on the profile

We are hiring AI Evaluation Specialists across multiple domains to evaluate, benchmark, and improve next-generation AI/LLM models. Candidates will assess AI-generated outputs for accuracy, quality, reasoning, language, technical correctness, and adherence to defined evaluation guidelines.

1. Technical AI Evaluation Specialist – Code & Software Engineering

• Experience: 2+ years Software Development or strong CS background.
• Evaluate AI-generated code for correctness, performance, security, and best practices.
• Review text-to-code, image-to-code, UI, and software architecture outputs.
• Perform code reviews, SxS comparisons, debugging, and edge-case analysis.
• Skills: Python, JavaScript/TypeScript, Java, C++, Go, React/HTML/CSS, testing, Git.
• Strong technical writing and analytical skills required.

2. Speech & Audio AI Evaluation Specialist – International Voice

• Experience: 2+ years international voice experience in a GCC/Captive/In-house setup.
• Evaluate AI-generated Speech-to-Speech (S2S) and Text-to-Speech (TTS) outputs.
• Assess pronunciation, tone, emotion, cadence, naturalness, and conversational quality.
• Review accents, dialects, lip-sync, and audio/video synchronization.
• Mandatory: C2 / near-native English proficiency.
• Strong auditory attention and communication skills required.

3. Senior AI Evaluation & RLHF Specialist

• Experience: 2+ years in AI data annotation, model evaluation, or RLHF.
• Evaluate and rank AI responses using complex evaluation rubrics.
• Perform SxS comparisons covering factuality, reasoning, helpfulness, and instruction following.
• Identify model failures and provide structured corrective feedback.
• Create golden responses and edge-case examples for model training.
• Strong analytical ability and excellent written English required.

4. ASR & Transcription QA Specialist

• Experience: 2–3+ years in transcription, captioning, subtitling, or related services.
• Review and correct AI-generated speech-to-text outputs.
• Identify transcription errors, speaker changes, overlapping speech, and audio inconsistencies.
• Follow verbatim and clean-read transcription standards.
• Mandatory: 65+ WPM typing speed with 99% accuracy.
• Strong English listening and written comprehension required.

5. Spatial Reasoning Evaluation Specialist – Video RL

• Background: STEM degree with exposure to Computer Vision, Video ML, RL, robotics, or simulation.
• Evaluate AI-generated videos and autonomous-agent trajectories frame by frame.
• Assess temporal sequencing, physical plausibility, action accuracy, and goal completion.
• Review segmentation, bounding boxes, motion, and visual artifacts.
• Identify visual hallucinations, physics errors, and spatial inconsistencies.
• Python/scripting knowledge and strong analytical reasoning are preferred.

Common Requirements

• Strong attention to detail and analytical reasoning.
• Ability to follow detailed evaluation guidelines and maintain high-quality output.
• Excellent written and/or spoken English depending on the profile.
• Comfortable working on repetitive, high-accuracy evaluation tasks.
• Ability to provide clear, objective, evidence-based feedback.
• Contract Duration: 3 Months.