Case category classification

Plans and permissions:

• Add-on: AI agents or Copilot, billed separately
• Set up and activate case categories: Admin role, or a custom role with Case classification settings set to Can edit case classification settings
• Case classification is turned on by our team. If you don’t see Classification in the Configure menu, contact our team.

Use case category classification to have Assembled categorize your cases by contact reason automatically. Your support team spends less time categorizing tickets and more time on the reply, and you get consistent categories you can use to understand why customers contact you. The voice agent also uses these categories to route calls.

To open it, go to Configure > Classification, and select the Category tab.

This article describes setup with Zendesk. Category classification also works with Kustomer and Salesforce, with the same steps and slightly different output options. Our team works with you through each step.

How category classification works

• Your categories can have several levels, such as a parent category and its subcategories.
• Each category has a description that tells the AI what belongs in it. Assembled can write these descriptions for you, from historical tickets that support team members put in each category.
• An evaluation tests the descriptions against those historical tickets, so you can measure accuracy before anything changes in your help desk.
• Once you activate it, Assembled categorizes incoming cases and writes the result to a Zendesk field or tags.
• Long-running steps, such as generating descriptions or running an evaluation, continue in the background, and you get an email when they finish.

How category changes sync with Zendesk

• Assembled doesn’t keep syncing categories from Zendesk after the import. If you change your categories in Zendesk, make the same change in Assembled.
• Changes you make in Assembled are written to Zendesk. Adding a category in Assembled adds it in Zendesk, and changing one replaces it.
• Deleting a category in Assembled doesn’t remove it from Zendesk.

Import your categories

Start from the categories you use today.

1. Go to Configure > Classification, and select the Category tab.
2. Under Category import source, choose how to import your categories:
   • Import from Zendesk custom field: choose the field in Zendesk custom field name, and Assembled builds the hierarchy from the :: separators in its values. We recommend this option.
   • Upload CSV: download the CSV template, and fill it in with a column for each level of your hierarchy. For each category, include a description and its true positives: the IDs of tickets correctly assigned to it, separated by commas, such as 12345, 67890.
3. Select Continue. The import can take a few minutes.

Review and edit your categories

Check that the imported categories and their hierarchy match how you categorize cases, and delete any that are stale or unused.

To open a category, select it. Categories at the lowest level of the hierarchy show Example cases for this category: historical tickets support team members put in that category. Assembled uses them to write and test the category’s description.

To change a category, select the menu next to its name:

Generate descriptions

1. When your list is final, select Confirm categories.
2. Select Autogenerate descriptions.

Assembled writes descriptions that keep each category distinct and cover all your cases, using the historical tickets in each category. It writes descriptions only for categories that don’t have one. This takes a while, so check back in about 30 minutes, or wait for the email.

Run an evaluation

An evaluation measures how accurately your descriptions categorize cases. Assembled categorizes the same historical tickets it used to write the descriptions, and compares its result with the category support team members chose. A match counts as correct. Evaluations don’t change your tickets or their fields.

1. Select Run evaluation.
2. Confirm in the Run evaluation window.

An evaluation can take from 20 minutes to overnight, depending on how many categories you have. You get an email when it’s done.

Read the results and improve accuracy

For example: an evaluation of 400 cases with 240 true positives has an overall accuracy of 240 ÷ 400 = 60%.

To improve a category’s accuracy:

• Change its example cases. If your team doesn’t always categorize these tickets correctly, replace the examples with tickets that are categorized correctly. Better examples lead to more specific descriptions.
• Add a rule. If a ticket property is already set when the ticket is created, such as a tag, you can use it as a rule for the category. The property must be set at ticket creation.

After you change a category, select Optimize description from its menu, then run another evaluation to check that accuracy improved. Keep refining until overall accuracy is at least 50 to 60%.

Activate category classification

We recommend running Assembled’s categories side by side with your current process first. Write them to a new Zendesk field, so you can compare them with the categories your support team chooses, before you write to your existing field.

1. Select Preview results to check how incoming cases would be categorized.
2. Select Activation settings.
3. Under Categorization output, choose where to write categories:
   • Import from an existing Zendesk custom field: writes to the field you imported your categories from.
   • Create a new Zendesk custom field: enter a Display name for a new field. Use this to compare side by side.
   • Zendesk ticket tags: adds a tag for each category. Enter a Tag prefix.
4. Select Enable category classification.

Assembled starts categorizing incoming cases right away. To stop, open Activation settings and select Disable classification. Categories already written to Zendesk stay there.