Test Lead

We are looking for a hands-on Senior Test Engineer who can own quality across DataWeave's SaaS platform, spanning manual and automation testing, API and data validation, and testing of AI and ML driven features such as pricing insights, recommendations, and LLM powered outputs. This role requires strong testing fundamentals, comfort building and maintaining automation frameworks, and the judgment to design test strategies for features whose outputs are probabilistic rather than deterministic. You will partner closely with engineering, product, and data science to ensure releases are reliable, performant, and trustworthy at scale.

Roles and Responsibilities

• Design, execute, and maintain test plans and test cases for new features, covering functional, regression, integration, and exploratory testing across the DataWeave platform.
• Build and extend automation test suites (UI and API) using frameworks such as Selenium, Playwright, or Cypress, integrated into CI/CD pipelines for fast, reliable feedback.
• Own API testing end to end: functional, contract, negative, and performance testing of REST and internal service APIs using tools such as Postman, RestAssured, or equivalent.
• Define and execute test strategies for AI and ML driven features (LLM based insights, recommendations, classification, and NLP or computer vision pipelines), including evaluation of output accuracy, consistency, edge cases, hallucination checks, and regression across model or prompt changes.
• Validate large scale data pipelines and outputs for correctness, completeness, and consistency, working closely with data engineering and data science teams.
• Set up and maintain test data, environments, and mocks needed for repeatable manual and automated testing.
• Triage defects, drive root cause analysis with engineering, and track quality metrics (coverage, defect leakage, automation ROI) to improve release quality over time.
• Contribute to test strategy, tooling choices, and best practices, and mentor junior QA engineers on the team.
• Participate in sprint planning, story grooming, and release readiness reviews, representing the quality and testability point of view.Skills and Qualifications
• 4 to 7 years of overall QA/testing experience, with a strong mix of manual and automation testing in a SaaS or product engineering environment.
• Hands-on experience building and maintaining automation frameworks (Selenium, Playwright, Cypress, or similar) and integrating them into CI/CD (Jenkins, GitHub Actions, or similar).
• Strong experience in API testing (REST, JSON/XML payloads) using Postman, RestAssured, or comparable tools, including contract and performance testing.
• Practical exposure to testing AI/ML or LLM powered features, such as evaluating model outputs, prompt regression, accuracy and precision/recall style metrics, or building golden datasets for evaluation.
• Solid understanding of SQL and the ability to validate data in large datasets or data warehouses.
• Working knowledge of scripting or programming (Python, Java, or JavaScript) for automation and data validation scripts.
• Familiarity with Agile/Scrum delivery, defect tracking (Jira), and test management tools.
• Strong analytical mindset, attention to detail, and the ability to reason about edge cases in complex, data heavy, AI driven systems.
• Excellent communication skills and comfort working directly with engineering, product, and data science stakeholders.
• Proficiency in using at least one of the AI Developer Productivity Tools (like Claude, GitHub Copilot, Cursor..)
• Bachelor's degree in Engineering, Computer Science, or a related field.