Connect Dify to OpenLLM Buddy

Connect Dify to OpenLLM Buddy with the OpenAI-API-compatible model provider, then use it in chatbots, agents, and workflows.

No-code, automation & home · Integration guide

What is Dify?

Dify is an open-source platform for building LLM apps visually: chatbots, agents, RAG knowledge bases, and multi-step workflows. Visit Dify.

Why connect OpenLLM Buddy?

Dify's OpenAI-API-compatible provider takes a model name, key, and endpoint, so your OpenLLM Buddy deployment becomes a selectable model across every app you build.

Before you start, create an API key in the console and deploy a model. If anything fails, run the curl test in Step 4 first: no tool can fix a broken endpoint.

1

Install the compatible provider

In Dify, open Settings → Model Provider. Find OpenAI-API-compatible and install it if it is not already installed.

2

Add your model

Click Add Model on the provider and fill in the form.

  • Model Type: LLM
  • Model Name: gemma4:26b
  • API Key: your OpenLLM Buddy key
  • API endpoint URL: https://openllmbuddy-proxy.botbuddytech.workers.dev/v1
  • Model context size: your deployment's context length, for example 32768
  • Turn on Function calling support only if the model supports tool calls.

Save. Dify tests the connection and shows an error if the URL or key is wrong.

3

Use it in an app

Create a chatbot, agent, or workflow, and choose the model you just added in the LLM node or the model picker.

4

Test the endpoint with curl

Confirm the key, URL, and model id work before debugging Dify. A short JSON reply means the endpoint is healthy.

Terminal
bash
curl -sS https://openllmbuddy-proxy.botbuddytech.workers.dev/v1/chat/completions   -H "Authorization: Bearer YOUR_API_KEY"   -H "Content-Type: application/json"   -d '{"model": "gemma4:26b", "messages": [{"role": "user", "content": "Reply with OK"}], "max_tokens": 16}'

Troubleshooting

  • Cannot reach the endpoint from self-hosted Dify — the API container needs outbound internet access. Check Docker network and proxy settings.
  • Agent will not call tools — enable function calling on the model entry and use a model that supports it.
  • 401 or 403 — the API key is wrong, revoked, or missing the Bearer prefix. Create a fresh key in the console.
  • 404 on /chat/completions — the base URL must end in /v1, with no trailing /chat/completions and no extra path.
  • Model not found — the model id must match your deployment exactly. Run the curl test and copy the id you used.
  • Timeouts on the first request — a GPU that was idle may still be loading the model. Retry after a minute.
  • Connection refused from Docker or a remote host — use the public HTTPS base URL, never localhost.

Connection details

Use these values wherever Dify asks for an OpenAI-compatible provider.

Reference
text
Base URL:  https://openllmbuddy-proxy.botbuddytech.workers.dev/v1
Chat URL:  https://openllmbuddy-proxy.botbuddytech.workers.dev/v1/chat/completions
API key:   your key from the console
Model id:  gemma4:26b