AI Scorecards

Overview

AI Scorecards are part of the Usable AI bundle and are designed to enhance quality management in your contact center by automatically generating a quality score for every interaction. Rather than relying on manual review, which is time-consuming, inconsistent, and difficult to scale, AI Scorecards evaluate each interaction against your defined criteria and produce a score along with a written justification for each question.

By automating scoring, AI Scorecards help reduce the time spent on quality tasks, make it easier to identify trends and outliers across your team, and deliver more consistent feedback for agent coaching and development.

Setting Up an AI Scorecard

AI Scorecards are configured in Admin → Configuration → Scorecards.

Create a New Scorecard

Select New Scorecard to begin.

Provide a descriptive Scorecard name, including a relevant date and version number is recommended to make it easier to manage scorecards over time (e.g., Call Center Scorecard v2.2 2025Q4).

Define Your Scoring Criteria

Add the questions you want the AI to evaluate for each interaction. Each question consists of the following:

• Question

  • Enter the evaluation question. The AI will assess each interaction against this question and provide a written justification explaining why it passed or failed.

• Weight

  • Assign a weight to each question. All question weights must total 100 across the scorecard.

• Auto-Fail

  • If enabled, failing this question will result in an overall score of 0/100 regardless of how other questions were scored. Use this for critical compliance or policy requirements where a single failure should invalidate the entire interaction.

Assign Queues

Select the “+ Assign Queue” button in the UI

You’ll be presented with this configuration/association menu. Select the queues this scorecard should apply to. Interactions handled through the assigned queues will be scored automatically using this scorecard's criteria.

Assign Communication Types

Select which interaction types should be scored. Available options are:

• Voice

• SMS

• Chat

Save the Scorecard

Select Save to save your configuration. The scorecard will now be active for the assigned queues and communication types.

Testing Your Scorecard

Once saved, you can test the scorecard against a previous interaction before relying on it for live scoring.

Run a Test

Enter a previous Queue Call Manager ID and select Test. The scorecard will evaluate that interaction using your configured criteria and return the results.

NOTE: In order for the AI scoring function to initiate, the system needs something to evaluate! What this means, is that for things like voice-based interactions (inbound/outbound calls) you must have transcription/recording enabled for the queue.

Without the underlying data to evaluate, you’ll see an error like this when you try to test your scorecard against a specific QCMID:

Review the Test Results

The test results display the Overall Score, Pass/Fail outcome, and a per-question justification explaining the AI's reasoning for each result. Use this to validate your criteria and refine your questions before rolling the scorecard out more broadly.

Reporting on AI Scorecard Results

AI Scorecard results are stored alongside your Queue Call Records in the AI Score field, making them available for reporting and review directly within your existing call data. Use this “AI Score” field in the Queue Activity data-source to track scoring trends, identify outliers, and surface coaching opportunities across your team.

Related Articles

• Sharpen Usable AI