How to use different LLM backends in Ozeki Voice Keyboard

This page provides guides on configuring different LLM backends for Ozeki Voice Keyboard, including Ollama on Windows, LLama.cpp on Ubuntu, and vLLM on Ubuntu. Each guide covers setup, configuration, and usage of local AI assistants with voice input.

How to use Ollama on Windows as the LLM backend for Ozeki Voice Keyboard

This guide shows how to configure Ollama as the LLM backend for Ozeki Voice Keyboard on Windows. It covers verifying Ollama is running, enabling HTTP logging, configuring LLM settings, using the AI assistant with voice commands, and checking requests in logs. The integration allows voice queries to be processed locally using Ollama's API at localhost:11434.

How to use LLama.cpp as the LLM backend for Ozeki Voice Keyboard on Ubuntu

This tutorial demonstrates building and configuring a LLama.cpp server on Ubuntu as the LLM backend for Ozeki Voice Keyboard. It explains setting up Conda, cloning and building LLama.cpp with CUDA support, downloading the LLM model, starting the server, and connecting it to the voice keyboard for GPU-accelerated inference over the network.

How to use vLLM as the LLM backend for Ozeki Voice Keyboard on Ubuntu

This guide explains how to set up a vLLM server on Ubuntu as the LLM backend for Ozeki Voice Keyboard on Windows. It covers creating a Conda environment, installing vLLM via pip, starting the server with the Qwen model, and connecting it to Ozeki Voice Keyboard. The setup enables GPU-accelerated inference over the network for voice queries.