Connect Raspberry Pi to OpenLLM Buddy

Start with a Pi running Raspberry Pi OS (64-bit recommended). You can work directly on the Pi with a keyboard and monitor, or connect over SSH from another computer on the same network.

1. Flash Raspberry Pi OS with Raspberry Pi Imager and boot the board.
2. Connect Ethernet or Wi‑Fi so the Pi has internet access.
3. Enable SSH in Imager (or run sudo raspi-config → Interface Options → SSH).
4. Update packages and install curl, Python, and jq for testing.

sudo apt update && sudo apt upgrade -y
sudo apt install -y curl python3 python3-pip python3-venv jq

Supported boards: Raspberry Pi 3, 4, 5, and Pi Zero 2 W. The Pi does not run the model locally — it only sends HTTPS requests to OpenLLM Buddy.

# From your laptop (replace pi with your Pi username if different)
ssh pi@raspberrypi.local

Replace raspberrypi.localwith your Pi's IP if mDNS does not resolve. See the official SSH documentation.

# Confirm the Pi can reach the internet
ping -c 3 1.1.1.1
curl -sI https://openllmbuddy-proxy.botbuddytech.workers.dev/v1 | head -n 1

OpenLLM Buddy authenticates requests with a Bearer token. Create a key in the console before configuring the Pi.

1. Sign in and open the API keys page.
2. Create a new key and copy it immediately — you will not see the full value again.
3. Store the key somewhere safe on your laptop until you paste it into the Pi in step 4.

Treat the key like a password. Do not commit it to git, paste it into public forums, or leave it in shell history on a shared machine. The Pi sends it as Authorization: Bearer YOUR_API_KEY on every request.

You can revoke and rotate keys anytime from the console if a key is exposed.

On the Pi, set OPENLLM_BUDDY_BASE_URL to your OpenLLM Buddy root URL ending in /v1. The OpenAI Python SDK and most clients append /chat/completions automatically.

https://openllmbuddy-proxy.botbuddytech.workers.dev/v1

Important: do not put the full chat-completions path in the Base URL. If you set the Base URL to /v1/chat/completions, requests often fail with 404 because the client adds /chat/completions again.

https://openllmbuddy-proxy.botbuddytech.workers.dev/v1/chat/completions

The full request URL is always https://openllmbuddy-proxy.botbuddytech.workers.dev/v1/chat/completions — but your env var should stop at /v1.

Export three variables in your shell session, or persist them so every login and script sees the same values. Replace YOUR_API_KEY with the key from step 2.

1. OPENLLM_BUDDY_BASE_URL — root URL from step 3 (https://openllmbuddy-proxy.botbuddytech.workers.dev/v1).
2. OPENLLM_BUDDY_API_KEY — your secret key from the console.
3. OPENLLM_BUDDY_MODEL — model handle (step 5 explains the value).

export OPENLLM_BUDDY_BASE_URL="https://openllmbuddy-proxy.botbuddytech.workers.dev/v1"
export OPENLLM_BUDDY_API_KEY="YOUR_API_KEY"
export OPENLLM_BUDDY_MODEL="gemma4:26b"

To keep variables after you close the terminal, append the same lines to ~/.bashrc (or ~/.profile on login shells), then run source ~/.bashrc.

# ~/.bashrc — OpenLLM Buddy on Raspberry Pi
export OPENLLM_BUDDY_BASE_URL="https://openllmbuddy-proxy.botbuddytech.workers.dev/v1"
export OPENLLM_BUDDY_API_KEY="YOUR_API_KEY"
export OPENLLM_BUDDY_MODEL="gemma4:26b"

For systemd services (step 9), store secrets in a root-only file instead of your home directory:

# /etc/openllmbuddy.env — readable only by root (chmod 600)
OPENLLM_BUDDY_BASE_URL=https://openllmbuddy-proxy.botbuddytech.workers.dev/v1
OPENLLM_BUDDY_API_KEY=YOUR_API_KEY
OPENLLM_BUDDY_MODEL=gemma4:26b

After creating /etc/openllmbuddy.env, run sudo chmod 600 /etc/openllmbuddy.env so only root can read the API key.

The model field in every chat request must match the handle for your chosen OpenLLM Buddy deployment. Set OPENLLM_BUDDY_MODEL to that value.

Common handles on OpenLLM Buddy:

• Gemma 4 26B → gemma4:26b
• Qwen 3.6 27B → qwen3.6:27b

For this guide (based on your selected model), use gemma4:26b.

1. Confirm the handle on the models page for your deployment.
2. Export OPENLLM_BUDDY_MODEL=gemma4:26b (already included in step 4).
3. Use the same string in curl -d JSON and in Python client.chat.completions.create(model=...).

A wrong model ID usually returns 400 or 404 from the API — not an auth error.

With environment variables loaded, send a chat completion from the Pi. A JSON response with an assistant message confirms network access, your API key, Base URL, and model ID are all correct.

curl "$OPENLLM_BUDDY_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $OPENLLM_BUDDY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "gemma4:26b", "messages": [{"role": "user", "content": "Reply with OK"}]}'

# Optional: print only the assistant reply (requires jq from step 1)
curl -s "$OPENLLM_BUDDY_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $OPENLLM_BUDDY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "gemma4:26b", "messages": [{"role": "user", "content": "Reply with OK"}]}' \
  | jq -r '.choices[0].message.content'

A successful response looks similar to:

{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "OK"
      }
    }
  ]
}

The second command prints only the assistant text when jq is installed (step 1). If you see 401 Unauthorized, recheck your API key in the console. 404 usually means the Base URL includes /chat/completions — revert to step 3.

For scripts and services, use a virtual environment so the OpenAI SDK does not conflict with system Python packages on Raspberry Pi OS.

1. Create a project folder under your home directory.
2. Create and activate a venv.
3. Install the official OpenAI Python package inside the venv.

mkdir -p ~/openllmbuddy && cd ~/openllmbuddy
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install openai

Run source .venv/bin/activate in every new shell before using python3 for OpenLLM Buddy. Your prompt will show (.venv) when active.

Environment variables from step 4 must still be set in that shell (or loaded via systemd in step 9).

Save a small script that reads the same environment variables as curl. Pass a question as a command-line argument, or omit it to use the default test prompt.

1. Save the code below as ~/openllmbuddy/ask.py.
2. Activate the venv from step 7.
3. Run the script with a prompt.

#!/usr/bin/env python3
"""Minimal OpenLLM Buddy client for Raspberry Pi."""
import os
import sys

from openai import OpenAI

def main() -> None:
    base_url = os.environ.get("OPENLLM_BUDDY_BASE_URL")
    api_key = os.environ.get("OPENLLM_BUDDY_API_KEY")
    model = os.environ.get("OPENLLM_BUDDY_MODEL", "gemma4:26b")

    if not base_url or not api_key:
        print("Set OPENLLM_BUDDY_BASE_URL and OPENLLM_BUDDY_API_KEY first.", file=sys.stderr)
        sys.exit(1)

    client = OpenAI(base_url=base_url, api_key=api_key)
    prompt = " ".join(sys.argv[1:]) or "Reply with OK"

    response = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
    )

    print(response.choices[0].message.content)


if __name__ == "__main__":
    main()

cd ~/openllmbuddy
source .venv/bin/activate
chmod +x ask.py
python3 ask.py "What is 2+2?"

Use this pattern in GPIO triggers, sensor pipelines, or cron jobs: load env vars, call client.chat.completions.create, and act on the text response.

To run a script automatically when the Pi boots (health check, notification, or scheduled ask), register a one-shot systemd unit. Secrets live in /etc/openllmbuddy.env from step 4.

[Unit]
Description=OpenLLM Buddy ask-on-boot example
After=network-online.target
Wants=network-online.target

[Service]
Type=oneshot
EnvironmentFile=/etc/openllmbuddy.env
WorkingDirectory=/home/pi/openllmbuddy
ExecStart=/home/pi/openllmbuddy/.venv/bin/python3 /home/pi/openllmbuddy/ask.py "Pi booted successfully"
User=pi
Group=pi

[Install]
WantedBy=multi-user.target

sudo nano /etc/openllmbuddy.env   # paste env file from step 4
sudo chmod 600 /etc/openllmbuddy.env
sudo chown root:root /etc/openllmbuddy.env

sudo nano /etc/systemd/system/openllmbuddy-ask.service
# paste the unit file from above, adjust User/Group/paths if needed

sudo systemctl daemon-reload
sudo systemctl enable --now openllmbuddy-ask.service
sudo systemctl status openllmbuddy-ask.service

Adjust User, Group, and paths if your Pi username is not pi. Use journalctl -u openllmbuddy-ask.service to inspect logs after boot.

For recurring jobs (every hour, on sensor events), prefer cron or a timer unit instead of running at every boot.

Most Pi setup issues fall into a few categories. Check these before changing code.

401 Unauthorized

Wrong or revoked API key. Create a new key in the console and update OPENLLM_BUDDY_API_KEY. Ensure the header is Bearer <key> with no extra quotes in curl.

404 Not Found

Base URL is wrong. Use …/v1, not …/v1/chat/completions in OPENLLM_BUDDY_BASE_URL. See step 3.

400 / model errors

Model handle does not match your deployment. Confirm OPENLLM_BUDDY_MODEL against the models page (step 5).

Connection timed out

Pi has no internet or DNS failure. Run the network checks from step 1. Verify firewall rules allow outbound HTTPS (port 443).

Python: module not found

Virtualenv not activated. Run source ~/openllmbuddy/.venv/bin/activate before python3 ask.py (step 7).

Env vars empty in cron/systemd

Cron and systemd do not load ~/.bashrc. Use EnvironmentFile=/etc/openllmbuddy.env in systemd (step 9) or export vars inside the cron script.

Your Raspberry Pi is now an OpenLLM Buddy client. The same three environment variables power curl, Python, cron, GPIO scripts, and home-automation hooks — the Pi never needs to run inference locally.

• Reuse env vars in new scripts under ~/openllmbuddy or your own project paths.
• Rotate API keys from the console if the Pi is re-imaged or compromised.
• Keep secrets out of git — use /etc/openllmbuddy.env or a secrets manager for production Pis.

Explore other integrations from the integrations hub, or pick another model and revisit this guide with ?model= in the URL.

Quick reference for wiring OpenLLM Buddy into Pi projects. Substitute your real API key from the console — never commit live secrets to git.

OPENLLM_BUDDY_BASE_URL

https://openllmbuddy-proxy.botbuddytech.workers.dev/v1

Full chat-completions URL (for curl only)

https://openllmbuddy-proxy.botbuddytech.workers.dev/v1/chat/completions

OPENLLM_BUDDY_API_KEY

YOUR_API_KEY

OPENLLM_BUDDY_MODEL

gemma4:26b

Auth header

Authorization: Bearer YOUR_API_KEY

Request format

OpenAI Chat Completions JSON at POST /v1/chat/completions with model, messages, and optional temperature / max_tokens.

Any language or library that supports OpenAI-compatible HTTPS calls works on the Pi — Node.js, Go, or plain curl in shell scripts.