Production-ready API gateway for self-hosted LLMs developed by the French Government, fully open-source forever.

Deploy OpenGateLLM quickly with Docker connected to our own free model and start using it:

make quickstart

Test the API:

curl -X POST "http://localhost:8000/v1/chat/completions" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer changeme" \
-d '{"model": "albert-testbed", "messages": [{"role": "user", "content": "Hello, how are you?"}]}'

The default master API key is changeme.

A user interface is available at: http://localhost:8501/playground

User: master Password: changeme

make create-user

With configuration file, you can connect to your own models and add addtionnal services to OpenGateLLM. Start by creating a configuration file and a .env dedicated:

cp config.example.yml config.yml
cp .env.example .env

Check the configuration documentation to configure your configuration file.

Vou can then set your environment variables in .env according to your needs.

You can run the services you need by running:

docker compose --env-file .env up <services_you_need> --detach 

For instance:

docker compose --env-file .env up api playground postgres redis elasticsearch --detach 

You can check our helmchart and instructions here: https://github.com/etalab-ia/albert-api-helm

Explore practical use cases:

Tutorial	Link
Chat Completions
Multi-Model Access
Retrieval-Augmented Generation (RAG)
Audio Transcriptions
Optical Character Recognition (OCR)

This project exists thanks to all the people who contribute. OpenGateLLM thrives on open-source contributions. Join our community!

Check out our Contribution Guide to get started.