← Onidia
Setup guide · 03

Keep her on the Pi.

With an AI HAT+ 2 attached, a language model runs on the Hailo-10H's NPU at about 40 TOPS of INT4 — so she thinks fast, costs nothing per token, and never sends the conversation anywhere.

100% local No API key AI HAT+ 2 required
Why this guide is short

Hailo-Ollama speaks the ordinary Ollama /api/chat dialect, so Onidia needs no special provider — she treats it as ollama. The default address and model in her config already match this server exactly.

Install the server

1

Get the software

HailoRT v5.0 or newer, and the Hailo-Ollama server on top of it. The quickest route is the prebuilt Docker image from Hailo's developer zone.

where to get it
https://hailo.ai/developer-zone/

You need unzip, Docker, and the HAT+ 2 seated on the Pi's PCIe connector. Check the device is visible before going further:

terminal
ls /dev/h1x-0    # the NPU device node
2

Start it

Loaded from the Docker image, the server runs on its own. Note the device flag and the port mapping — /dev/h1x-0 is the accelerator, and 8000 is the port Onidia expects.

terminal
unzip hailort_<ver>_docker.zip
docker load -i hailo_docker_hailort_ub2204_<ver>.tar

docker run -it --rm --device /dev/h1x-0 \
    -p 8000:8000 \
    -v $HOME/.local/share/hailo-ollama/models/blob:/usr/share/hailo-ollama/models/blob \
    hailo_docker_hailort_ub2204:<ver>

Installed natively instead? The server binary is just hailo-ollama, and it listens on the same port:

terminal
hailo-ollama
3

Confirm it answers

Before touching any config, make sure the server is actually up.

terminal
curl --silent http://localhost:8000/api/version
4

Pull a model onto the NPU

Models ship as Hailo's own compiled format, so they are downloaded rather than built. qwen2:1.5b is the sweet spot on a Pi — fast enough to feel conversational and small enough to stay smooth.

terminal
# what is available to download
curl --silent http://localhost:8000/hailo/v1/list

# pull one (stream:true is the default and shows progress)
curl --silent http://localhost:8000/api/pull \
     -H 'Content-Type: application/json' \
     -d '{"model": "qwen2:1.5b", "stream" : true }'

To pull everything on offer in one go:

terminal
curl --silent http://localhost:8000/hailo/v1/list \
  | jq -r '.models[]' \
  | while read model; do
      echo "Pulling $model..."
      curl --no-buffer --silent http://localhost:8000/api/pull \
        -H 'Content-Type: application/json' \
        -d "{\"model\": \"$model\", \"stream\" : true}"
    done
5

Point Onidia at it

No key, because there is nothing to authenticate. These two values are already the defaults, so an empty section works — they are here so you can see the shape.

~/.config/chat-app/chat-app.ini
provider = ollama

[ollama]
ollama-api-url = http://localhost:8000
ollama-model   = qwen2:1.5b
Not 11434

Plain Ollama listens on 11434; Hailo-Ollama listens on 8000, and 8000 is what Onidia assumes when ollama-api-url is unset. If you are running both, set the URL explicitly rather than relying on the default.

6

Check it, then start her

terminal
onidia-chat -preflight warn

# terminal 1 - the body
onidia

# terminal 2 - the brain
onidia-chat

Worth knowing

One conversation at a time

The server handles a single request with one pending behind it. One Onidia, one terminal — perfectly fine, but not built for sharing.

Language only

Text models work. Vision-language models and text-to-image do not, and neither do LoRA adapters — the NPU path is more specialised than full Ollama.

No Ollama CLI

Use curl against the REST API, as above. ollama run will not connect to this server.

Image replies still need the internet

Image lookup uses Pixabay or Wikipedia, and image-source = gemini needs Google. Set image-source = off to keep her entirely offline.