Back
Conversational AI Agent Analyst
Full-time
About the Team
Our team drives the end-to-end tools, systems, and AI-powered experiences that enable world-class support for millions of Roku customers and the advisors who help them. We own the strategy, roadmap, and delivery of the platforms that power customer support at scale — including our CRM, contact center technologies, agent tools, knowledge systems, automation capabilities, and LLM-based customer service bots.
About the Role
Roku is seeking a Conversational AI Agent Analyst to join our Product Management team. In this role, you will focus on generating data-driven insights and guiding the roadmap for our customer-facing LLM-powered Bot products. You will analyze key performance metrics and customer feedback to identify opportunities for improving the bot experience. Your findings will directly influence product enhancements and overall customer satisfaction.
What You'll Be Doing
Analyze Bot Performance: Evaluate the performance of existing and new bot features by tracking and interpreting key bot KPIs, including Bot CSAT scores, Bot Containment Rate, Bot Accuracy Rate, Bot Recontact Rate, and more. Identify trends, gaps, and areas for improvement.
Drive Product Improvements: Work closely with product managers, customer service operations, and our 3rd party bot vendor to prioritize enhancements and fixes. Use your findings to shape the product roadmap and ensure development efforts align with user needs and business goals.
Product Quality Ownership: Own creation of test cases and hands-on testing of any new bot releases. Work with cross-functional stakeholders to ensure test cases are executed, triaged, and driven to resolution.
Reporting & Monitoring: Work with internal data science teams and bot vendor to maintain and update dashboards, ensuring they provide necessary insights.
Bot System Administration: Be the single point of contact for bot system administration tasks, ensuring system configurations are updated and optimized.
We're Excited If You Have
3+ years of experience in business analysis, product analytics, data analysis, or quality assurance, preferably working with customer-facing technology or AI-powered tools
Familiarity with conversational AI systems, LLMs, Bots, or virtual assistants, and an understanding of customer service metrics
Proven ability to analyze large datasets and derive meaningful insights. Comfortable working with metrics and KPIs to evaluate product performance
Proficiency with Excel, BI tools, or data visualization software. Experience writing SQL queries is a plus
Excellent written and verbal communication skills. Strong storytelling skills with data
Experience working in cross-functional teams and the ability to influence product decisions through data-driven storytelling
A proactive and curious mindset with strong problem-solving skills and enthusiasm for AI tools and techniques
