Back
AI Product Analyst
Full-time
This role is for one of Weekday's clients. We are looking for an AI Product & Evals Analyst to drive evaluation, quality measurement, product analytics, and enterprise reporting for AI-powered voice and conversational systems. The role sits at the intersection of Product, AI Engineering, Data Analytics, and Customer Success, defining how AI performance is measured, monitored, and continuously improved.
Key Responsibilities
Design and manage AI evaluation frameworks to measure the quality and performance of voice and conversational AI systems.
Define conversation-level and interaction-level metrics to monitor AI effectiveness, accuracy, and user experience.
Analyze business KPIs such as automation rates, containment, customer satisfaction, conversion, and cost optimization.
Build and maintain enterprise dashboards, reports, and visualizations using modern BI tools.
Perform data analysis using SQL and Python to identify trends, detect regressions, and generate actionable insights.
Leverage AI tools, LLMs, and workflow automation to streamline analytics, reporting, and root cause analysis.
Collaborate with Product, AI Engineering, Customer Success, and Operations to improve product quality and enterprise adoption.
What Makes You a Great Fit
2-5 years in Product Analytics, Data Analytics, AI Quality, or similar analytical roles.
Strong proficiency in SQL and Python for data analysis and investigative analytics.
Experience designing dashboards with Power BI, Tableau, Looker, or similar platforms.
Strong understanding of conversational systems and AI-powered applications, with excellent communication and storytelling skills.
Exposure to Conversational AI, LLMs, prompt engineering, voice technologies (STT), or AI evaluation methodologies is an added advantage.
