Quality review – Assembled

Plans and permissions:

Add-on: AI agents or Copilot, billed separately
Review and score cases: Admin, Manager, or Team Lead role, or a custom role with Quality review set to Review quality for all agents or Review quality for certain agents
Add and edit your own scores: Admin, Manager, or Team Lead role, or a custom role with QA scoring settings set to Can edit QA scoring settings
View the AI agent QA scorecard: Admin, Manager, or Team Lead role, or a custom role with AI agent QA scorecard set to Can view
Add a case to your workflow test cases: Admin, Manager, or Team Lead role, or a custom role with Workflows set to Can create, edit, and delete workflows
With Review quality for certain agents, you see only the tabs for the agent types your role allows.

Use Quality review to check how well your AI agents and Copilot respond, and to record evaluations your team can track over time. Evaluations show you trends, and whether you need to change your configuration or update your knowledge.

To open it, go to AI quality > QA review. The page is called Quality review.

Navigating Quality review

The Quality review page with the Email agent, Chat agent, Voice agent, and Copilot tabs, the filters, and the case list

Item	What it’s for
Tabs	Email agent, Chat agent, Voice agent, and Copilot show the cases for each. You see a tab for each product your company uses.
Date range, sort, and Add filter	Narrow the cases by date, evaluation status, channel, category, and more. On the AI agent tabs, you can filter by Case status.
Case list	Each case, with columns such as AI outcome and AI experience on the AI agent tabs.
Manage QA scoring	Opens QA scoring, where you add your own scores. See Add your own scores.

How evaluations work

Every AI agent conversation and every Copilot interaction appears in Quality review.
An evaluation is a set of ratings your team gives a case, such as whether the reply was accurate. Each kind of case has default criteria, and you can add your own scores.
Evaluations of Copilot cases, along with support team members’ feedback on Copilot replies, feed the AI agent QA scorecard.
AI agent cases are also scored automatically. See AI experience scores and outcomes.

Evaluate a case

Select a case in the list. The conversation opens, with the Overview and QA scoring tabs beside it.
Read the conversation and the AI’s replies. On AI agent cases, Overview shows the case’s status and AI experience score.
Select QA scoring, then select a rating for each score.
Optional: add notes in Comments.
Select Submit evaluation, or press Cmd + Enter on a Mac or Ctrl + Enter on Windows.

When you’ve submitted it, you see Feedback saved. To move to the next or previous case, press the right or left arrow key. To close the case, press Esc.

To use a Zendesk case to test your workflows, select Add test case at the top of the open case, and choose the workflow that should handle it. The case is added to your test case library for Workflow bulk testing.

A case open in Quality review, with the conversation and the QA scoring tab

Default evaluation criteria

The default criteria depend on the kind of case. Your own scores appear alongside them.

Score	Ratings	Shown for
AI agent reply accuracy	Correct, Somewhat incorrect, or Incorrect: whether the answer was accurate.	All cases
Knowledge retrieval	Correct source or Incorrect source: whether Copilot used the right knowledge articles.	Copilot cases
AI agent reply style	Excellent, Fair, or Poor: how well the reply follows your style settings in Brand studio.	Copilot and email agent cases
Path correctness	Correct or Incorrect: whether the email agent chose the right workflow.	Email agent cases
Escalation detection accuracy, Routing accuracy, and Gather data accuracy	Correct, Somewhat correct, or Incorrect	Voice agent cases

Add your own scores

Add scores for anything else you want to evaluate, such as tone or policy compliance.

Go to AI quality > QA review and select Manage QA scoring.
On QA scoring, select Add score.
Enter a Name and a Summary.
In Channel, choose the kinds of cases the score is for, such as Copilot draft replies or chat agent cases.
Under Ratings, add the options reviewers choose from, such as Correct, Somewhat incorrect, and Incorrect.
Optional: under Reasoning Tags, add tags reviewers can choose to explain a rating.
Select Create scorecard.

The score appears on the QA scoring tab of every case of the kinds you chose. To change a score, select it in the QA scoring list, make your changes, and select Update scorecard.

The QA scoring page with the Add score button

The rating buttons for the AI agent reply accuracy score: Correct, Somewhat incorrect, and Incorrect

Track results in the AI agent QA scorecard

The AI agent QA scorecard reports on Copilot’s replies. It combines evaluations from your QA team with feedback from your support team: the Helpful and Not helpful buttons, a thumbs up and a thumbs down, that support team members select on Copilot’s replies.

To open it, go to Report > AI agent QA scorecard.

The AI agent QA scorecard report with the Drafted replies and Agent guidance tabs

Item	What it shows
Drafted replies tab	Copilot’s drafted replies, including drafts with notes and drafts with agent guidance. Charts for reply accuracy, knowledge retrieval, reply style, and agent feedback.
Agent guidance tab	Copilot’s answers to support team members’ questions, and its case summaries. Charts for reply accuracy, knowledge retrieval, and agent feedback.
Date range, Type, and Add filter	The report shows the last two weeks by default. Type narrows it to kinds of interaction, and Source narrows it to Zendesk, Upload, or Slack.

Your own scores also appear in the report.

Get useful results

The scorecard only includes cases your QA team has evaluated or support team members have given feedback on. To get meaningful results:

Evaluate cases in Quality review regularly.
Encourage your support team to rate Copilot’s replies with Helpful or Not helpful.
Evaluate every kind of Copilot interaction your team uses.
Keep your evaluation criteria consistent.