Analytics & Monitoring

Track your bot's performance, understand user behavior, and continuously improve response quality.

Dashboard Analytics

Access analytics from your bot's Analytics tab in the dashboard.

Overview Metrics

Query Volume

Track query patterns over time:

• Daily/hourly breakdown: Identify peak usage times
• Trends: Week-over-week growth or decline
• Anomalies: Unexpected spikes or drops

Top Queries

See the most frequently asked questions. Use this to:

• Ensure content covers common questions
• Identify gaps in documentation
• Optimize responses for popular queries
• Inform content strategy

Performance Monitoring

Latency Breakdown

Understand where time is spent in each query:

Latency Percentiles

Beyond averages, track percentiles for real user experience:

• P50 (median): Half of queries are faster than this
• P90: 90% of queries are faster
• P99: 99% of queries are faster (worst case)

Quality Metrics

Success Rate

Percentage of queries that:

• Found relevant context
• Generated a response (no errors)
• Completed within timeout

Target: Above 95%

Context Relevance

Average similarity score of retrieved chunks. Low scores may indicate:

• Content gaps in knowledge base
• Threshold set too low
• Queries outside your content scope

API Analytics

Access analytics programmatically via the Analytics API:

GET /api/analytics/bots/{agentId}/?period=week

Response:
{
  "totalQueries": 1250,
  "averageLatencyMs": 890,
  "successRate": 0.98,
  "topQueries": [...],
  "queriesPerDay": [...]
}

Alerting

Set up alerts for important metrics (Enterprise feature):

• Latency alert: P90 latency exceeds threshold
• Error rate alert: Success rate drops below threshold
• Volume alert: Unusual query volume (spike or drop)
• Quota alert: Approaching monthly query limit

Continuous Improvement

Weekly Review Process

1. Check top queries: Are common questions being answered well?
2. Review failed queries: Why did some queries fail?
3. Monitor latency trends: Is performance stable?
4. Identify content gaps: What questions aren't covered?
5. Update knowledge base: Add missing content

A/B Testing (Enterprise)

Test configuration changes with a subset of traffic:

• Compare different models
• Test RAG setting variations
• Evaluate system prompt changes
• Measure impact before full rollout

Exporting Data

Export analytics data for external analysis:

• CSV export: Download from dashboard
• API: Programmatic access to all metrics
• Webhooks: Real-time event streaming (Enterprise)

Privacy & Data Retention