Conversational AI Business Quality Analyst

Classification: Contract Contract Length: 12-months Position Summary The Conversational AI Business Quality Analyst ensures the quality, performance, and customer experience of Virtual Agent (VA) and IVR solutions. This role combines business analysis, QA, and contact center technology expertise to validate conversational journeys end-to-end and drive continuous improvement. The analyst partners with Operations, IT, Product, and Conversation Design teams to support the full lifecycle-from requirements and testing through go-live and optimization. The ideal candidate is detail-oriented, analytical, and comfortable working independently in a fast-paced environment. Experience with contact center technologies, conversational AI, and healthcare is highly valued. Responsibilities Audit 100% of Virtual Agent interactions using standardized scoring criteria. Validate adherence to design, quality, and compliance standards. Identify NLU/ASR issues, intent misclassifications, UX friction points, and logic errors. Confirm correct task completion, backend logic, authentication workflows, and escalation paths. Document and escalate compliance risks. Document defects with clear expected vs. actual behavior and supporting evidence. Classify issues (e.g., NLU gap, design flaw, logic error, compliance). Create and manage Azure DevOps stories and prioritize using established frameworks. Support remediation discussions with product, IT, and design teams. Perform regression testing to validate fixes. Requirements Bachelor's degree in Computer Science, Analytics, Information Systems, Business, Engineering, or related field. 3+ years in business or product analysis. Contact center experience required; 2+ years in QA preferred. Experience with Agile environments and data visualization tools (Power BI, Tableau) preferred. Experience with VA/IVR platforms (Dialogflow, Google CCAI, Genesys, Avaya, CCaaS) strongly preferred. Healthcare industry experience a plus. Strong SQL and advanced Excel skills. Familiarity with NLP concepts and conversational design. Understanding of IVR flows, call routing, and contact center KPIs. Experience with BigQuery, SSMS, or similar data environments. Basic scripting/text manipulation (e.g., Python) is a plus. Exposure to test automation tools (Selenium, QTP) preferred.

Created: 2026-03-04