Lead I - ML Engineering

Role Description

Primary Role: Build/deployment of AI agents. Role Overview As an AI Agent Developer, you will build complex agentic architectures and establish evaluation flywheels to prevent regressions. You will primarily use LangChain and LangGraph for orchestration, and utilize LangSmith for tracing, monitoring, and automated testing. Key Responsibilities Build Agents: Develop stateful, cyclic multi-agent systems with persistent memory and human-in-the-loop controls. Optimize Observability: Instrument agent code to trace inner workings like tool calls, retrieval augmented generation (RAG) flows, and decision paths. Run Evaluations: Create programmatic test datasets and calibrate LLM-as-a-judge evaluators to score agent responses based on human preferences. Iterative Debugging: Inspect granular trace data to find recurring failure modes, fix prompt instructions, and deploy performance updates in one click. Production Monitoring: Set up dashboards and s for latency, cost, and hallucination rates to ensure reliable production deployments. Qualifications & Skills Experience: 5+ years of software engineering experience with a focus on LLM application development for at 2 years. Tech Stack: Proficiency in Python , Databricks, MS SQL, Core Frameworks: Deep knowledge of LangChain and LangGraph for orchestration. LangSmith Expertise: Hands-on experience using LangSmith Observability & Evals to test prompts, analyze trajectories, and manage datasets. Agent Concepts: Solid understanding of basic concept (model, tool, embedding, vector DB) and advanced agentic behaviors (working memory, subagents, tool-use loops, and complex reasoning). Problem Solving: Passion for the agent improvement loop: Build, Test, Deploy, and Monitor. Good to have Pay domain knowledge

Skills

Agentic AI, Vector Databases, LangChain, LangGraph, LangSmith, LLMs, Python, RAG, SQL, Prompt Engineering