Usage & Metrics¶

This guide explains how to monitor and analyze your agent usage, performance, and compliance metrics.

Overview¶

Ag2Trust tracks comprehensive metrics for every agent interaction:

Category	Metrics
Usage	Messages, tokens (input/output), active agents
Performance	Response time, tokens per second
Compliance	Toxicity flags, PII redactions
Feedback	User ratings, satisfaction trends

All metrics are customer-isolated and accessible via the Dashboard or API.

Dashboard Analytics¶

The Usage Analytics dashboard provides visual insights into your agent usage.

Summary Cards¶

Metric	Description
Messages	Total agent responses in the period
Total Tokens	Combined input + output tokens
Active Agents	Number of agents that responded
Avg Response Time	Mean LLM response latency

Charts¶

Message Volume Over Time - Daily message counts trending
Token Usage by Agent - Compare token consumption across agents

Time Period Selection¶

View analytics for different periods: - 7 Days - Recent activity - 30 Days - Monthly trends - 90 Days - Quarterly overview

Usage API¶

Summary Endpoint¶

Get aggregated usage statistics:

GET /api/stats/usage/summary?days=7

Response:

{
  "period_days": 7,
  "total_messages": 1250,
  "total_tokens": 485000,
  "total_input_tokens": 320000,
  "total_output_tokens": 165000,
  "active_agents": 5,
  "avg_response_time_ms": 1850
}

Parameter	Type	Default	Description
`days`	integer	7	Lookback period (7, 30, 90)

Usage by Agent¶

Break down usage per agent:

GET /api/stats/usage/by-agent?days=30

Response:

[
  {
    "agent_id": 42,
    "agent_name": "Support Bot",
    "message_count": 850,
    "total_tokens": 325000,
    "total_input_tokens": 210000,
    "total_output_tokens": 115000,
    "avg_response_time_ms": 1720
  }
]

Results are sorted by total_tokens descending.

Usage Timeline¶

Get daily usage for charts:

GET /api/stats/usage/timeline?days=30&tz=America/New_York

Response:

{
  "labels": ["2025-12-01", "2025-12-02", "2025-12-03"],
  "messages": [45, 52, 48],
  "tokens": [18500, 21200, 19800]
}

Parameter	Type	Default	Description
`days`	integer	30	Lookback period
`tz`	string	"UTC"	IANA timezone for date grouping

Timezone Support

Pass the user's browser timezone (e.g., America/Los_Angeles) to ensure events appear on the correct local date.

Usage by Model¶

Compare usage across LLM models:

GET /api/stats/usage/by-model?days=30

Response:

[
  {
    "model": "claude-sonnet-4",
    "message_count": 620,
    "total_tokens": 245000,
    "total_input_tokens": 160000,
    "total_output_tokens": 85000,
    "avg_response_time_ms": 1650
  },
  {
    "model": "gpt-4o",
    "message_count": 380,
    "total_tokens": 156000,
    "total_input_tokens": 102000,
    "total_output_tokens": 54000,
    "avg_response_time_ms": 1420
  }
]

Compliance Metrics¶

Track content safety and privacy compliance.

Compliance Summary¶

GET /api/stats/compliance/summary?days=30

Response:

{
  "period_days": 30,
  "total_messages": 1250,
  "toxic_messages": 3,
  "toxic_rate_percent": 0.24,
  "pii_redactions": {
    "total": 147,
    "by_type": {
      "EMAIL_ADDRESS": 62,
      "PHONE_NUMBER": 45,
      "PERSON": 28,
      "CREDIT_CARD": 12
    }
  },
  "messages_with_pii": 89,
  "pii_rate_percent": 7.12
}

Compliance by Agent Type¶

GET /api/stats/compliance/by-agent-type?days=30

Response:

{
  "period_days": 30,
  "agent_types": [
    {
      "agent_type_id": 5,
      "agent_type_name": "Customer Support",
      "total_messages": 850,
      "toxic_messages": 2,
      "toxic_rate_percent": 0.24,
      "pii_redactions_total": 112,
      "pii_per_message": 0.132
    }
  ]
}

Feedback Analytics¶

Track user satisfaction with agent responses.

Feedback Summary¶

GET /api/stats/feedback/summary?days=30

Response:

{
  "period_days": 30,
  "total_feedback": 234,
  "average_rating": 4.2,
  "rating_distribution": {
    "1": 5,
    "2": 12,
    "3": 28,
    "4": 89,
    "5": 100
  }
}

Feedback by Agent Type¶

Includes load correlation analysis:

GET /api/stats/feedback/by-agent-type?days=30

Response:

name=__codelineno-15-1 href=#__codelineno-15-1>{ "period_days": 30, "agent_types": [ { "agent_type_id": 5, "agent_type_name": "Customer Support", "total_feedback": 156, "average_rating": 4.3, "rating_by_load": { "low": { "instances_range": "1-2", "avg_rating": 4.5, "count": 45 }, "medium": { "instances_range": "3-5", "avg_rating": 4.2, "count": 78 }, "high": { "instances_range": "6+", "avg_rating": 3.8, "count": 33 } } } ] class=p>}

Load Correlation

The rating_by_load breakdown shows how satisfaction changes as running instances increase. Use this to identify optimal scaling thresholds.

Agent Type Comparison¶

Comprehensive comparison across all dimensions:

GET /api/stats/agent-types/comparison?days=30

Response:

{
  "period_days": 30,
  "agent_types": [
    {
      "agent_type_id": 5,
      "name": "Customer Support",
      "model": "claude-sonnet-4",
      "performance": {
        "total_messages": 850,
        "total_tokens": 325000,
        "avg_response_time_ms": 1720,
        "tokens_per_second": 95.5,
        "avg_tokens_per_message": 382
      },
      "compliance": {
        "toxic_rate_percent": 0.24,
        "pii_redactions_per_message": 0.132
      },
      "satisfaction": {
        "feedback_count": 156,
        "average_rating": 4.3,
        "rating_degrades_at_load": 6
      }
    }
  ]
}

Metrics Explained¶

Metric	Description
`tokens_per_second`	Output generation speed
`avg_tokens_per_message`	Average response length
`toxic_rate_percent`	Percentage of flagged content
`pii_redactions_per_message`	Average PII entities redacted
`rating_degrades_at_load`	Instance count where satisfaction drops

Data Collection¶

How Usage is Tracked¶

Every agent response records:

┌─────────────────────────────────────────────────────────────────────────┐
│                        AGENT RESPONSE                                    │
├─────────────────────────────────────────────────────────────────────────┤
│                                                                         │
│  → Token counts from LLM provider                                       │
│      input_tokens, output_tokens, total_tokens                          │
│                                                                         │
│  → Performance metrics                                                  │
│      response_time_ms (LLM latency)                                     │
│                                                                         │
│  → Model metadata                                                       │
│      model_used (e.g., 'claude-sonnet-4')                               │
│                                                                         │
│  → Ownership                                                            │
│      customer_id, agent_id, agent_type_id                               │
│                                                                         │
└─────────────────────────────────────────────────────────────────────────┘
                               │
                               ▼
                    agent_usage_events table

Compliance Tracking¶

The audit service separately tracks:

Toxicity scores from content moderation
PII redaction counts by entity type (email, phone, etc.)

Retention¶

Usage data retention follows your tier:

Tier	Usage Data Retention
Free	7 days
Pro	90 days
Enterprise	Custom (up to 7 years)

Best Practices¶

1. Monitor Token Costs¶

Track total_tokens trends to predict costs:

const stats = await fetch('/api/stats/usage/summary?days=30')
const tokensPerDay = stats.total_tokens / 30
const projectedMonthly = tokensPerDay * 30

2. Identify Slow Agents¶

Use avg_response_time_ms to find performance issues:

const byAgent = await fetch('/api/stats/usage/by-agent?days=7')
const slow = byAgent.filter(a => a.avg_response_time_ms > 3000)

3. Compare Model Performance¶

Use model breakdown to optimize provider selection:

const byModel = await fetch('/api/stats/usage/by-model?days=30')
// Compare tokens_per_second and avg_response_time_ms

4. Track Compliance Trends¶

Monitor toxicity and PII rates over time:

const compliance = await fetch('/api/stats/compliance/summary?days=30')
if (compliance.toxic_rate_percent > 1.0) {
  // Alert: High toxicity rate
}

5. Scale Based on Load Correlation¶

Use feedback load analysis to set scaling thresholds:

const feedback = await fetch('/api/stats/feedback/by-agent-type?days=30')
// If rating_degrades_at_load is 6, scale up before 6 instances

API Reference¶

Usage Endpoints¶

Method	Endpoint	Description
`GET`	`/api/stats/usage/summary`	Aggregated usage stats
`GET`	`/api/stats/usage/by-agent`	Usage per agent
`GET`	`/api/stats/usage/timeline`	Daily usage for charts
`GET`	`/api/stats/usage/by-model`	Usage per LLM model

Compliance Endpoints¶

Method	Endpoint	Description
`GET`	`/api/stats/compliance/summary`	Toxicity & PII stats
`GET`	`/api/stats/compliance/by-agent-type`	Compliance per agent type

Feedback Endpoints¶

Method	Endpoint	Description
`GET`	`/api/stats/feedback/summary`	Rating summary
`GET`	`/api/stats/feedback/by-agent-type`	Ratings with load correlation

Comparison Endpoint¶

Method	Endpoint	Description
`GET`	`/api/stats/agent-types/comparison`	Full comparison dashboard

Next Steps¶

Billing & Pricing - Understand agent-hour billing
Rate Limits - API rate limiting details
Data Retention - Configure retention policies