Connect LibreChat to OpenLLM Buddy

Connect LibreChat to OpenLLM Buddy with a custom endpoint in librechat.yaml. Includes the Docker override, the .env key, and model setup.

Chat UI · Integration guide

What is LibreChat?

LibreChat is an open-source chat platform that supports many providers in one interface, with conversation branching, presets, agents, and multi-user auth. Visit LibreChat.

Why connect OpenLLM Buddy?

LibreChat's custom endpoints accept any OpenAI-compatible URL. OpenLLM Buddy slots in beside your other providers and shows up as its own endpoint in the model menu.

Before you start, create an API key in the console and deploy a model. If anything fails, run the curl test in Step 5 first: no tool can fix a broken endpoint.

Put your API key in .env

In your LibreChat folder, add the key to .env. The YAML file references it by name so the key never sits in the config.

.env

bash

OPENLLMBUDDY_API_KEY=YOUR_API_KEY

Add the endpoint to librechat.yaml

Create or edit librechat.yaml next to .env and add a custom endpoint.

librechat.yaml

yaml

version: 1.2.1
cache: true
endpoints:
  custom:
    - name: "OpenLLM Buddy"
      apiKey: "${OPENLLMBUDDY_API_KEY}"
      baseURL: "https://openllmbuddy-proxy.botbuddytech.workers.dev/v1"
      models:
        default: ["gemma4:26b"]
        fetch: false
      titleConvo: true
      titleModel: "current_model"
      modelDisplayLabel: "OpenLLM Buddy"

Keep the version line from the example file that ships with your LibreChat release if it differs. Set fetch: true if you want LibreChat to read the model list from /v1/models instead.

Mount the file (Docker only)

With Docker Compose, tell the API container to load the config, then restart.

docker-compose.override.yml

yaml

services:
  api:
    volumes:
      - type: bind
        source: ./librechat.yaml
        target: /app/librechat.yaml

Then run docker compose down && docker compose up -d.

Select the endpoint

Open LibreChat, click the endpoint menu at the top of a chat, and choose OpenLLM Buddy. Pick gemma4:26b and send a message.

Test the endpoint with curl

Confirm the key, URL, and model id work before debugging LibreChat. A short JSON reply means the endpoint is healthy.

Terminal

bash

curl -sS https://openllmbuddy-proxy.botbuddytech.workers.dev/v1/chat/completions   -H "Authorization: Bearer YOUR_API_KEY"   -H "Content-Type: application/json"   -d '{"model": "gemma4:26b", "messages": [{"role": "user", "content": "Reply with OK"}], "max_tokens": 16}'

Troubleshooting

Endpoint missing from the menu — the YAML was not loaded. Check the bind mount path and look for YAML errors in the API container logs.
Key shows as undefined — the variable name in .env must match the ${...} name exactly. Restart the containers after editing .env.
401 or 403 — the API key is wrong, revoked, or missing the Bearer prefix. Create a fresh key in the console.
404 on /chat/completions — the base URL must end in /v1, with no trailing /chat/completions and no extra path.
Model not found — the model id must match your deployment exactly. Run the curl test and copy the id you used.
Timeouts on the first request — a GPU that was idle may still be loading the model. Retry after a minute.
Connection refused from Docker or a remote host — use the public HTTPS base URL, never localhost.

Connection details

Use these values wherever LibreChat asks for an OpenAI-compatible provider.

Reference

text

Base URL:  https://openllmbuddy-proxy.botbuddytech.workers.dev/v1
Chat URL:  https://openllmbuddy-proxy.botbuddytech.workers.dev/v1/chat/completions
API key:   your key from the console
Model id:  gemma4:26b