Back
Conversational AI (Voicebot) Quality Assurance
Full-time
This is a full-time, on-site position based in Mexico City.
You're not testing software functionality.
You'll ensure AI-powered voicebots deliver natural, accurate, and effective conversations by analyzing real user interactions, identifying quality issues, and working closely with voicebot designers to improve performance continuously.
Software QA candidates won't be considered.
Qualifications
You're a curious, analytical problem-solver who understands what makes a great conversation and can turn insights from real user interactions into actionable improvements.
Experience in Contact Center Quality, Customer Experience (CX), Conversational AI, or a related field.
Experience evaluating customer interactions, conversations, call recordings, or customer service quality.
Strong analytical and problem-solving skills, with the ability to identify patterns, investigate root causes, and prioritize issues based on severity and business impact.
Understanding of conversational AI, voicebots, chatbots, or contact center operations is a strong plus.
Familiarity with concepts such as LLMs, Prompts, ASR, TTS, NLU, Intent Recognition, Conversation Flows, and Knowledge Bases, or willingness to learn.
Ability to evaluate conversational quality, including response accuracy, intent recognition, dialogue flows, and overall user experience.
Excellent communication and collaboration skills, particularly when working with Voicebot Designers and cross-functional teams.
Business-oriented mindset, proactive attitude, and strong sense of ownership.
Main responsibilities
Monitor the quality and performance of AI-powered voicebots across live and pre-launch projects.
Review call recordings and real user interactions to evaluate conversational quality, response accuracy, intent recognition, dialogue flows, and overall user experience.
Identify incorrect responses, misunderstandings, repetitive questions, faulty dialogue flows, and other issues affecting the customer experience.
Monitor key quality and performance indicators to identify anomalies, recurring patterns, and potential delivery risks.
Investigate issues through conversation analysis, performance data, and recordings to identify their root causes.
Work closely with Voicebot Designers to recommend improvements to prompts, intents, conversation flows, response strategies, and exception handling.
Identify gaps between existing voicebot designs and actual user behavior, highlighting emerging user needs and scenarios not covered by current configurations.
Escalate critical quality issues and collaborate with the Product Delivery team to resolve them promptly.
Validate implemented improvements by reviewing subsequent interactions and monitoring performance indicators.
Prepare quality reports, root cause analyses, and actionable recommendations to support continuous voicebot optimization.
Contribute to improving QA standards, quality checklists, monitoring processes, and reporting practices.
For strategic accounts, identify recurring customer pain points and behavioral trends, providing insights to improve conversational design and business outcomes.
About AI Rudder
AI Rudder is an AI product company transforming how businesses communicate with their customers.
We launched in 2019 with one of the world's first enterprise-grade AI voice agents: built to hold real, intelligent conversations, not just play back scripts. Today, we serve over 500 enterprises across 20+ countries, processing 50 million calls every day and having delivered more than 8 billion calls to date. Our customers range from fast-growing startups to large enterprises.
When AI talks to people, it has to feel human. That's what we build.
