Local Deployment

UsageGuard provide the option to deploy and run core UsageGuard components inside your infrastructure.

Requirments

You will need to be able to download, host and run the following:

Core Components

• OpenSearch (Documentation )
• PostgreSQL (Documentation )

Documents / Search Support

• AWS S3 or MinIO (Documentation )

• Pinecone or Qdrant (Documentation )

Local Inference

• Ollama (Documentation )

Services API Keys

You can always only work with local inference using Ollama as a provider, however you can also supply API keys for the services that you want to use like OpenAI or AWS Bedrock which will provide access to models from Anthropic, Mistral, Nova family of models and more.

You can also opt for using Qdrant as your vector database or provide API key for Pinecone (Serverless).

UsageGuard Installation

UsageGuard local deployments has two components each in its own docker image

• Reverse Proxy APIs (ug-ecr-reverse-proxy)
• Management Dashboards (ug-ecr-reverse-proxy-app)

Management Dashboards are optional however its highly recommended.

# Windows PowerShell
.\PullContainer.ps1 -Token "<paste-token-here>" -RepositoryName "ug-ecr-reverse-proxy"
.\PullContainer.ps1 -Token "<paste-token-here>" -RepositoryName "ug-ecr-reverse-proxy-app"

# Linux / macOS
./pull_container.sh "<your-token>" "ug-ecr-reverse-proxy"
./pull_container.sh "<your-token>" "ug-ecr-reverse-proxy-app"

for each UsageGuard container, mount appsettings.json into the container with your own settings or use enviroument variable to override existing settings.