✨ Our 2026 Hiring Report is out — see what 146,050 job postings say about getting hired.

Read the report →
JOB DETAILS

Test Analyst  - AI Assurance

CompanyHEXAWARE
LocationIndia
Work ModeOn Site
PostedNovember 18, 2026
About The Company
Hexaware is a global technology and business process services company driven by a singular purpose to create smiles through great people and technology. We combine AI-led engineering, deep industry expertise, and human ingenuity to help enterprises remove friction from their technology and business operations. Our Zero Friction Enterprise™ approach addresses challenges across legacy systems, cybersecurity, software delivery, IT operations, and licensing. Powered by Zerovity™, our AI delivery layer, and underpinned by Infinite Trust, we help organizations reduce complexity, improve efficiency, and accelerate innovation. The human expertise behind this approach comes from thousands of Hexawarians across offices spanning the globe. Investing in their skills and professional growth is central to how we deliver, while our focus on inclusion and opportunity extends beyond the workplace. Through our CSR initiatives, we support education, digital inclusion, healthcare, and livelihoods wherever we operate. That same sense of responsibility also informs our approach to sustainability, including our commitment to achieving net-zero emissions by 2040. Guided by our vision to be the world’s most loved digital transformation partner, we continue to pursue progress for the enterprises we serve, our people, and the communities around us. Zero Friction, Infinite Momentum.
About the Role

Role Overview
Strong understanding of GenAI system architecture, including Retrieval-Augmented Generation (RAG) and Agent-based systems, with hands-on experience in implementing Python-based AI evaluation frameworks such as Ragas, DeepEval, and similar tools. Proven ability to perform Model and Agent quality evaluations using platforms like Arize, LangFuse, and Confident AI, while driving Responsible AI (RAI) validation and reporting key RAI metrics. Experienced in golden dataset curation, synthetic dataset generation, and establishing AI observability and tracing through OpenTelemetry (OTEL) instrumentation to enable effective monitoring, diagnostics, and continuous quality improvement of AI systems.
Key Responsibilities
•  Ability to develop AI Assurance test strategy to assess AI systems for quality, safety, reliability and responsible AI behavior
•  Design AI evaluation strategy for LLMs, RAG and agent evaluation across functional and non-functional dimensions
•  Experience in Golden dataset curation & Synthetic dataset generation
•  Experience in implementing AI evaluation frameworks like Ragas, DeepEval, etc. 
•  Experience in using AI Evaluation tools like LangSmith, Arize, Langfuse, Galileo, Evidently, etc.
•  Experience in instrumentation to establish AI Observability / tracing using OTEL libraries
•  Experience in Integrating AI evaluation into CI/CD pipeline
•  Experience in Responsible AI validation

 

 Required Skills and Capabilities
•  Strong understanding of GenAI System Architecture of RAG & Agents
•  Proficiency in Python & AI evaluation SDKs
•  AI Evaluation Frameworks (Ragas, DeepEval) 
•  Model Evaluation 
•  Agent Evaluation 
•  Responsible AI (RAI) Validation 
•  RAI Metrics Reporting 
•  Golden Dataset Curation 
•  Synthetic Dataset Generation 
•  AI Observability 
•  AI Tracing 
•  OpenTelemetry (OTEL) 
•  LLM Evaluation 
•  AI Quality Engineering 
•  AI Governance 
•  AI Monitoring 
•  AI Testing & Assurance 
•  Root Cause Analysis (AI Systems)
•  Knowledge of regulatory considerations for AI systems
Role-Specific Requirements
•  Proficiency in Python & AI evaluation SDKs
•  Strong understanding of RAG & agentic system architecture
•  Ability to develop AI Assurance test strategy to assess AI systems for quality, safety, reliability and responsible AI behavior
•  Design AI evaluation strategy for LLMs, RAG and agent evaluation across functional and non-functional dimensions
•  Experience in Golden dataset curation & Synthetic dataset generation
•  Experience in implementing AI evaluation frameworks like Ragas, DeepEval, etc. 
•  Experience in using AI Evaluation tools like LangSmith, Arize, Langfuse, Galileo, Evidently, etc.
•  Experience in instrumentation to establish AI Observability / tracing using OTEL libraries
•  Experience in Integrating AI evaluation into CI/CD pipeline
•  Experience in Responsible AI validation
•  Knowledge of regulatory considerations for AI systems
Experience Requirements
More than 4 years of experience in AI Assurance Testing

 

Key Skills
PythonGenAIRAGAgent-based systemsRagasDeepEvalArizeLangFuseConfident AIResponsible AIOpenTelemetryGolden dataset curationSynthetic dataset generationAI observabilityCI/CD pipelineLLM evaluation
Categories
TechnologySoftwareData & AnalyticsEngineering
Job Information
📋Core Responsibilities
Develop and implement AI assurance test strategies to assess the quality, safety, and reliability of LLM, RAG, and agent-based systems. Integrate AI evaluation frameworks into CI/CD pipelines and perform continuous monitoring and diagnostics using observability tools.
📋Job Type
full time
📊Experience Level
2-5
💼Company Size
37122
📊Visa Sponsorship
No
💼Language
English
🏢Working Hours
40 hours
Apply Now →

You'll be redirected to
the company's application page