Providers¶

Providers are LLM (Large Language Model) services that power your agents. AG2Trust supports multiple providers, allowing you to choose the best model for each use case.

Supported Providers¶

Adding a Provider¶

Via Dashboard¶

1. Navigate to Settings > Providers
2. Click Add Provider
3. Select the provider type
4. Enter your API key
5. Give it a name (e.g., "OpenAI Production")
6. Click Save

Provider Configuration¶

Security¶

Credential Encryption¶

Provider API keys are encrypted using AWS KMS envelope encryption:

┌─────────────────────────────────────────────────┐
│              Encryption Process                  │
│                                                  │
│  1. Generate random DEK (Data Encryption Key)   │
│  2. Encrypt API key with DEK (AES-256-GCM)      │
│  3. Encrypt DEK with KMS master key             │
│  4. Store: encrypted_key + encrypted_DEK        │
└─────────────────────────────────────────────────┘

Key Points¶

• API keys are never stored in plain text
• Keys are decrypted only when starting an agent
• Decrypted keys passed to containers via environment variables (in-memory only)
• No .env files containing credentials

Credential Flow¶

sequenceDiagram
    participant D as Dashboard
    participant B as Backend
    participant K as AWS KMS
    participant A as Agent Container

    D->>B: Save provider (API key)
    B->>K: Encrypt API key
    K-->>B: Encrypted credential
    B->>B: Store encrypted

    Note over B,A: Later: Agent Start

    B->>K: Decrypt credential
    K-->>B: Plain API key
    B->>A: Pass via env var
    A->>A: Use for LLM calls

Model Selection¶

OpenAI Models¶

Anthropic Models¶

Choosing a Model¶

High volume + Simple tasks    → gpt-4o-mini
General purpose              → gpt-4o
Complex reasoning            → claude-opus-4
Code generation              → claude-sonnet-4
Cost sensitive              → gpt-4o-mini

Multiple Providers¶

You can configure multiple providers for different purposes:

Example Setup¶

Benefits¶

1. Quota separation: Don't let dev testing affect prod limits
2. Model specialization: Use the right model for each task
3. Failover: Switch providers if one has issues
4. Cost management: Track spending per use case

Rate Limits¶

Provider Rate Limits¶

Each LLM provider has their own rate limits:

AG2Trust Rate Limits¶

AG2Trust adds additional rate limiting for protection:

Monitoring Usage¶

Dashboard Metrics¶

Track provider usage in the Dashboard:

• Tokens consumed per agent
• Response times
• Error rates
• Cost estimates

Per-Agent Statistics¶

GET /api/agents/{id}/stats

Returns: - Total tokens used - Average response time - Messages processed - Errors encountered

Best Practices¶

1. Use Separate Keys for Environments¶

Provider: openai-production
  └── Production API key with higher limits

Provider: openai-development
  └── Development API key for testing

2. Monitor Token Usage¶

Set up alerts for: - Unusual token consumption spikes - High error rates - Slow response times

3. Choose Models Wisely¶

4. Rotate Keys Periodically¶

1. Generate new API key at provider
2. Add new provider config in AG2Trust
3. Update agent types to use new provider
4. Restart affected agents
5. Delete old provider config

Troubleshooting¶

"Invalid API Key"¶

1. Verify key is correct (no extra spaces)
2. Check key hasn't been revoked at provider
3. Ensure key has required permissions

"Rate Limit Exceeded"¶

1. Check your provider's rate limit tier
2. Reduce agent activity or add more agents
3. Consider upgrading your provider plan

"Model Not Available"¶

1. Verify model name is correct
2. Check model is available in your region
3. Some models require special access

Provider API¶

List Providers¶

GET /api/providers

Add Provider¶

POST /api/providers
Content-Type: application/json

{
  "name": "OpenAI Production",
  "type": "openai",
  "api_key": "sk-..."
}

Delete Provider¶

DELETE /api/providers/{id}