ChatSignals

Bot Performance Metrics

Understand the key performance metrics in ChatSignals, including human intervention rate, resolution rate, conversation volume, and handling time.

The dashboard and analytics pages track several metrics to help you understand your bot's effectiveness. Here are the key metrics and what they mean.

Core Metrics

Human Intervention Rate

The percentage of conversations where a human agent needed to step in. This is your primary measure of bot effectiveness.

  • Lower is better: A rate under 30% means your bot is handling most conversations independently.
  • Trending down: Indicates your knowledge base and configuration improvements are working.
  • Spike upward: May indicate a new topic your bot doesn't have knowledge for, or a product issue generating unusual questions.

Resolution Rate

The percentage of conversations your bot successfully resolved without human help.

  • Higher is better: Aim for steady improvement over time.
  • Includes auto-closure: Conversations the bot closed after confirming the customer was satisfied.

Conversation Volume

The total number of conversations your bot has handled over the selected time period. Useful for:

  • Understanding demand patterns (busy days, seasonal trends).
  • Measuring the impact of product changes on support volume.
  • Justifying ROI of AI automation.

Message Volume

A comparison of AI-generated messages vs human-sent messages. Shows the proportion of responses handled by the bot vs your team.

Time-Based Metrics

Handling Time

Average time from first customer message to conversation resolution. Compare:

  • Bot-resolved conversations: Usually seconds to minutes.
  • Human-resolved conversations: Usually minutes to hours.
  • The difference shows the time savings your bot provides.

Using Metrics Effectively

Filtering

All metrics can be filtered by:

  • Channel: Compare performance across different bots.
  • Date Range: Track trends over days, weeks, or months.

What to Monitor Weekly

  1. Human Intervention Rate: Is it trending down?
  2. Knowledge Gaps: Are new gaps appearing? (Add knowledge to address them.)
  3. Conversation Volume: Any unexpected spikes?
  4. Sales Actions: Are conversion rates holding steady?

Cost to Serve

If you've configured your team's hourly rate in Settings > Organisation, ChatSignals calculates the estimated cost savings from bot-handled conversations. This appears on the dashboard as a monetary value showing your return on investment.