Nextcloud Assistant and LocalAI: How We Optimised Response Speed

We are currently testing Nextcloud with the LocalAI app, connected to a self-hosted LLM via vLLM. The integration gives us text generation, translation, speech-to-text, image generation, and text-to-speech — all without any dependency on external providers like OpenAI.

The Nextcloud Assistant is, according to the vendor, a big step towards smarter workflows. It can create calendar entries, read and send messages in Talk, send emails, and much more. It learns from conversations and makes interactions feel more natural. Users can select text, request an AI operation (summarise, rephrase, generate a heading, etc.) and insert the result directly.

The Problem: Unpredictable Latency

During our testing with LocalAI we hit an unexpected snag: Nextcloud Assistant was taking surprisingly long to forward requests to the LLM, even though the LLM itself was responding quickly. The delays were invisible to users — sometimes the answer came instantly, sometimes it took forever, with no obvious pattern.

The Fix: Background Job Configuration

After digging through the documentation we found a relevant tip in the Nextcloud AI admin manual on how to improve AI task pickup speed. Applying this made a noticeable difference.

However, our test setup uses Nextcloud AIO, which goes offline at night to run backups. This caused the background job command to abort due to set -e, making response speed unpredictable — fast today, sluggish tomorrow until someone restarted the job manually.

The Solution: A Self-Healing Loop

Our goal was simple: on failure, wait five minutes and retry; on success, continue as normal. The result:

while true; do
  docker exec -it nextcloud-aio-nextcloud \
    sudo -E -u www-data php occ background-job:worker \
    -v -t 60 "OC\TaskProcessing\SynchronousBackgroundJob" \
  && : || sleep 300
done

This keeps the background job processing reliable and predictable without exposing the complexity to users.

Where We Are Heading

Our goal is to make the AI integration in Nextcloud as efficient and reliable as possible, while keeping ethics and data privacy at the centre. If testing continues to go well, we may roll out Nextcloud Assistant for production use soon.

We are glad to have found a solution in Nextcloud and LocalAI that puts performance, ethics, and privacy on equal footing.