Supported Language Models

UsageGuard supports a wide range of Language Models (LLMs) from various providers, allowing you to leverage the power of different AI models while maintaining a consistent API and robust safeguards.

Overview

UsageGuard acts as a proxy for multiple LLM providers, offering a unified interface for interacting with various models. This page provides an overview of the supported LLMs, their capabilities, and any provider-specific considerations.

Supported Providers and Models

UsageGuard offers a unified inference API for various language models.

Display NameModel IdentifierModel PointerMax TokensContext Window
Amazon Nova Micro 1.0us.amazon.nova-micro-v1:0amazon-nova-micro:latest5000128000
Amazon Nova Lite 1.0us.amazon.nova-lite-v1:0amazon-nova-lite:latest5000300000
Amazon Nova Pro 1.0us.amazon.nova-pro-v1:0amazon-nova-pro:latest5000300000
Open AI GPT-4o 2024-11-20gpt-4o-2024-11-20gpt-4o:latest16384128000
Open AI O1 Mini 2024-09-12o1-mini-2024-09-12o1-mini:latest65536128000
Anthropic Claude 3 Haikuanthropic.claude-3-haiku-20240307-v1:0claude-3-haiku:latest4096200000
Meta Llama 3.2 1B Instructus.meta.llama3-2-1b-instruct-v1:0llama3-2-1b-instruct4096128000
Meta Llama 3.2 3B Instructus.meta.llama3-2-3b-instruct-v1:0llama3-2-3b-instruct4096128000
Open AI GPT 4o Minigpt-4o-mini-2024-07-18gpt-4o-mini:latest16384128000
Anthropic Claude 3.5 Sonnet v2us.anthropic.claude-3-5-sonnet-20241022-v2:0claude-3.5-sonnet-v2:latest8192200000

Deprecated Access

The following models are deprecated but may still be accessible through existing connections:

Display NameModel IdentifierModel PointerMax TokensContext Window
Meta Llama3 70b Instructmeta.llama3-70b-instruct-v1:0llama3-70b-instruct:latest20488000
Meta Llama3 8b Instructmeta.llama3-8b-instruct:latestmeta.llama3-8b-instruct-v1:020488000
Amazon Titan Text G1 - Expressamazon.titan-text-express-v1titan-text-express:latest81928192
Amazon Titan Text G1 - Liteamazon.titan-text-lite-v1titan-text-lite:latest40964096
Amazon Titan Text G1 - Premieramazon.titan-text-premier-v1:0titan-text-premier:latest30723072
Open AI GPT-4o 2024-08-06gpt-4o-2024-08-06gpt-4o-2024-08-064096128000
Open AI GPT 3.5 Turbogpt-3.5-turbo-0125gpt-3.5-turbo409616000
Anthropic Claude 3.5 Sonnetanthropic.claude-3-5-sonnet-20240620-v1:0claude-3.5-sonnet4096200000
Anthropic Claude 3 Sonnetanthropic.claude-3-sonnet-20240229-v1:0claude-3-sonnet4096200000
Open AI GPT 3.5 Turbo Instructgpt-3.5-turbo-instructgpt-3.5-turbo-instruct40964000
Mistral 7b Instructmistral.mistral-7b-instruct-v0:2mistral-7b-instruct:latest819232000
Mistral 8x7b Instructmistral.mixtral-8x7b-instruct-v0:1mistral-8x7b-instruct:latest819232000
Open AI GPT-4o 2024-05-13gpt-4o-2024-05-13gpt-4o-2024-05-134096128000
Mistral Largemistral.mistral-large-2402-v1:0mistral-large:latest409632000
Mistral Smallmistral.mistral-small-2402-v1:0mistral-small:latest409632000

Switching Between Models

One of the key benefits of UsageGuard is the ability to easily switch between different LLMs without changing your application code. To switch models:

  1. Create a new or edit existing connection
  2. Enable the new model(s) to the connection
  3. Update your API calls to use the new connection ID (if new connection, add x-connection-id header)
  4. UsageGuard will handle the rest, including any necessary request transformations

Best Practices

  • Model Selection: Choose the appropriate model based on your specific use case and performance requirements.
  • Cost Management: Monitor your usage and leverage UsageGuard's cost control features to manage expenses.
  • Content Policies: Be aware of each provider's content policies and use UsageGuard's moderation features to ensure compliance.
  • Performance Optimization: Use model-specific best practices for prompt engineering and request formatting to get the best results.

Troubleshooting

If you encounter any issues with a specific model or provider:

  1. Check the Status Page for any known issues or outages
  2. Review the API Reference to learn more about the model's specific parameters.
  3. Consult our Error Handling Guide for common issues and solutions

If you need further assistance, don't hesitate to contact our support team.

Next Steps

Now that you're familiar with the supported LLMs, you're ready to start leveraging these powerful models in your applications:

Was this page helpful?