Supported Language Models
UsageGuard supports a wide range of Language Models (LLMs) from various providers, allowing you to leverage the power of different AI models while maintaining a consistent API and robust safeguards.
Overview
UsageGuard acts as a proxy for multiple LLM providers, offering a unified interface for interacting with various models. This page provides an overview of the supported LLMs, their capabilities, and any provider-specific considerations.
The list of supported models is continuously expanding. Check our changelog for the most recent additions.
Supported Providers and Models
UsageGuard offers a unified inference API for various language models.
| Display Name | Model Identifier | Model Pointer | Max Tokens | Context Window |
|---|---|---|---|---|
| Amazon Nova Micro 1.0 | us.amazon.nova-micro-v1:0 | amazon-nova-micro:latest | 5000 | 128000 |
| Amazon Nova Lite 1.0 | us.amazon.nova-lite-v1:0 | amazon-nova-lite:latest | 5000 | 300000 |
| Amazon Nova Pro 1.0 | us.amazon.nova-pro-v1:0 | amazon-nova-pro:latest | 5000 | 300000 |
| Open AI GPT-4o 2024-11-20 | gpt-4o-2024-11-20 | gpt-4o:latest | 16384 | 128000 |
| Open AI O1 Mini 2024-09-12 | o1-mini-2024-09-12 | o1-mini:latest | 65536 | 128000 |
| Anthropic Claude 3 Haiku | anthropic.claude-3-haiku-20240307-v1:0 | claude-3-haiku:latest | 4096 | 200000 |
| Meta Llama 3.2 1B Instruct | us.meta.llama3-2-1b-instruct-v1:0 | llama3-2-1b-instruct | 4096 | 128000 |
| Meta Llama 3.2 3B Instruct | us.meta.llama3-2-3b-instruct-v1:0 | llama3-2-3b-instruct | 4096 | 128000 |
| Open AI GPT 4o Mini | gpt-4o-mini-2024-07-18 | gpt-4o-mini:latest | 16384 | 128000 |
| Anthropic Claude 3.5 Sonnet v2 | us.anthropic.claude-3-5-sonnet-20241022-v2:0 | claude-3.5-sonnet-v2:latest | 8192 | 200000 |
Deprecated Access
The following models are deprecated but may still be accessible through existing connections:
| Display Name | Model Identifier | Model Pointer | Max Tokens | Context Window |
|---|---|---|---|---|
| Meta Llama3 70b Instruct | meta.llama3-70b-instruct-v1:0 | llama3-70b-instruct:latest | 2048 | 8000 |
| Meta Llama3 8b Instruct | meta.llama3-8b-instruct:latest | meta.llama3-8b-instruct-v1:0 | 2048 | 8000 |
| Amazon Titan Text G1 - Express | amazon.titan-text-express-v1 | titan-text-express:latest | 8192 | 8192 |
| Amazon Titan Text G1 - Lite | amazon.titan-text-lite-v1 | titan-text-lite:latest | 4096 | 4096 |
| Amazon Titan Text G1 - Premier | amazon.titan-text-premier-v1:0 | titan-text-premier:latest | 3072 | 3072 |
| Open AI GPT-4o 2024-08-06 | gpt-4o-2024-08-06 | gpt-4o-2024-08-06 | 4096 | 128000 |
| Open AI GPT 3.5 Turbo | gpt-3.5-turbo-0125 | gpt-3.5-turbo | 4096 | 16000 |
| Anthropic Claude 3.5 Sonnet | anthropic.claude-3-5-sonnet-20240620-v1:0 | claude-3.5-sonnet | 4096 | 200000 |
| Anthropic Claude 3 Sonnet | anthropic.claude-3-sonnet-20240229-v1:0 | claude-3-sonnet | 4096 | 200000 |
| Open AI GPT 3.5 Turbo Instruct | gpt-3.5-turbo-instruct | gpt-3.5-turbo-instruct | 4096 | 4000 |
| Mistral 7b Instruct | mistral.mistral-7b-instruct-v0:2 | mistral-7b-instruct:latest | 8192 | 32000 |
| Mistral 8x7b Instruct | mistral.mixtral-8x7b-instruct-v0:1 | mistral-8x7b-instruct:latest | 8192 | 32000 |
| Open AI GPT-4o 2024-05-13 | gpt-4o-2024-05-13 | gpt-4o-2024-05-13 | 4096 | 128000 |
| Mistral Large | mistral.mistral-large-2402-v1:0 | mistral-large:latest | 4096 | 32000 |
| Mistral Small | mistral.mistral-small-2402-v1:0 | mistral-small:latest | 4096 | 32000 |
Switching Between Models
One of the key benefits of UsageGuard is the ability to easily switch between different LLMs without changing your application code. To switch models:
- Create a new or edit existing connection
- Enable the new model(s) to the connection
- Update your API calls to use the new connection ID (if new connection, add
x-connection-idheader) - UsageGuard will handle the rest, including any necessary request transformations
Newly released models are added automatically to your existing connection, you can start using them immediately by sending the new model id in your request.
You can alwayes choose to disable a model from a connection, this will prevent it from being used in your requests.
Best Practices
- Model Selection: Choose the appropriate model based on your specific use case and performance requirements.
- Cost Management: Monitor your usage and leverage UsageGuard's cost control features to manage expenses.
- Content Policies: Be aware of each provider's content policies and use UsageGuard's moderation features to ensure compliance.
- Performance Optimization: Use model-specific best practices for prompt engineering and request formatting to get the best results.
Troubleshooting
If you encounter any issues with a specific model or provider:
- Check the Status Page for any known issues or outages
- Review the API Reference to learn more about the model's specific parameters.
- Consult our Error Handling Guide for common issues and solutions
If you need further assistance, don't hesitate to contact our support team.
Next Steps
Now that you're familiar with the supported LLMs, you're ready to start leveraging these powerful models in your applications: