~ / blog / dictation-on-the-thinkpad-npu-with-voxtype

Teaching My ThinkPad's NPU to Take Dictation

2026-10-07linuxarchvoxtypeomarchy

Every Core Ultra laptop ships with an NPU, a little neural processing unit the marketing people love to talk about. On my ThinkPad L13 it mostly sat there, idle, doing a convincing impression of a chip that isn’t plugged into anything. I wanted to give it an actual job. Dictation turned out to be the perfect one.

Omarchy ships Voxtype, a push-to-talk voice-to-text tool. Out of the box it runs Whisper on the CPU, which works fine. But the NPU was right there, sipping almost no power, built for exactly this kind of model. Voxtype has an OpenVINO engine that can run Whisper on Intel’s NPU. So that is where my words should go.

 speak ──▶ mic ──▶ Voxtype ──▶ Whisper small (int8) ──▶ text typed
                               on the Intel NPU            at the cursor
                               /dev/accel/accel0

Nothing leaves the laptop, the fans stay quiet, and the CPU is free to keep doing everything else.

The pieces

First, the hardware has to actually be there. On the L13 (Meteor Lake) the NPU shows up as a device node, driven by the intel_vpu module:

ls /dev/accel/accel0
lsmod | grep intel_vpu

Omarchy had the driver in place already. What I needed on top was the OpenVINO runtime and, crucially, the GenAI library that Voxtype’s backend links against:

sudo pacman -S --needed openvino openvino-intel-npu-plugin openvino-genai

That last one, openvino-genai, is the important one. It provides libopenvino_genai_c.so, the C API the Rust backend loads at startup. The pip wheel and the -bin package do not ship it, which is a fun afternoon to lose if you go down the wrong path. Take the Arch package and move on.

Pointing Voxtype at the NPU

Voxtype’s config lives in ~/.config/voxtype/config.toml. Two things matter: switch the engine over, and tell the OpenVINO engine which model, device and language to use.

engine = "openvino"

[openvino]
model = "small-int8"
device = "NPU"
language = "de"

Then restart the daemon:

systemctl --user restart voxtype

The first start downloads the small int8 OpenVINO Whisper model, a few hundred megabytes, and loads it onto the NPU. You can check what is installed with voxtype info models, or if you prefer buttons over a text editor, voxtype configure opens a little TUI for all of this.

Two languages, one key

I dictate in German and in English, and I switch between them constantly. Whisper can auto-detect the language, but pinning it is faster and noticeably more accurate, and the NPU engine loads its model exactly once, when the daemon starts. So “switch language” really means “swap the model and restart the daemon”.

That is a job for a three-line script with a nice name. Mine lives at ~/.local/bin/voxtype-language-toggle:

#!/bin/bash
# Switch Voxtype dictation between English and German on the Intel NPU.
# Usage: voxtype-language-toggle [en|de]   (no argument toggles)

set -euo pipefail
config="$HOME/.config/voxtype/config.toml"

current=$(sed -n '/^\[openvino\]/,/^\[/{s/^language = "\(.*\)"$/\1/p}' "$config" | head -n1)

target=${1:-}
[[ -z $target ]] && { [[ $current == "de" ]] && target="en" || target="de"; }

case $target in
  en) model="small.en-int8" label="English" ;;
  de) model="small-int8"    label="German"  ;;
  *)  echo "Usage: voxtype-language-toggle [en|de]" >&2; exit 1 ;;
esac

sed -i '/^\[openvino\]/,/^\[/{
  s/^model = .*/model = "'"$model"'"/
  s/^language = .*/language = "'"$target"'"/
}' "$config"

systemctl --user restart voxtype.service
omarchy-notification-send -g 󰍬 -t 3000 "Dictation: $label" "Model $model on the NPU"

Then one line in the Omarchy keybindings (~/.config/hypr/bindings.lua) puts it on a shortcut:

o.bind("SUPER + CTRL + SHIFT + X", "Switch dictation language", "voxtype-language-toggle")

Now Super + Ctrl + Shift + X flips me between German and English, with a little toast to confirm which model just landed on the NPU. Dictation itself is a separate key, and the words appear wherever my cursor is.

The numbers: NPU vs CPU

So does the NPU actually earn its keep? I timed the exact same small int8 model on all three compute blocks in this chip, transcribing 30 seconds of German speech. To keep it honest I measured two clip lengths and subtracted one from the other, so the figures below are pure transcription time with the one-off model load taken back out:

Device Transcribe 30 s of audio Versus real time
NPU ~0.46 s ~65× faster
CPU ~1.04 s ~29× faster
iGPU ~1.11 s ~27× faster

The NPU is a bit more than twice as fast as the CPU here, and the integrated GPU lands right next to the CPU. But raw speed is not even the main point. All three chew through audio far faster than real time, so for dictation the latency is fine whichever you pick. What the NPU really buys me is that it does the work on a dedicated low-power block: my CPU cores stay free and cool, and the battery barely notices. On the CPU the same job briefly spins the fans and eats most of a core; on the NPU it is a rounding error.

There is one honest catch. The NPU has to compile the model for its own hardware the first time it loads, which takes about seven seconds when the daemon starts, and again each time I flip the language, since that reloads the model. The CPU is ready almost instantly. But I start the daemon once and leave it running, so I pay those seven seconds once a day, not once a sentence. In normal use a finished sentence turns into text in well under a second, usually around half of one.

Why small, and not large

The obvious temptation is to reach for a bigger model. Resist it. The NPU is quick but it is not a datacenter GPU, and Voxtype’s streaming mode types as you talk: it keeps re-running the model over a rolling buffer of your last thirty seconds, every couple of seconds. Each pass has to finish before the next one is due. As the table shows, a full pass of the small model lands around half a second, which fits comfortably inside that window. Medium is several times slower, falls behind, the passes start queuing up, and stopping the recording hangs while it catches up. Small int8 is the sweet spot here: fast, accurate enough for real writing, and it leaves the rest of the machine alone.

The one gotcha: updates

A heads-up from experience. OpenVINO is not a hard dependency of Voxtype, it is an optional extra you bolt on. That means a system update can quietly decide those three packages are orphans and remove them. When that happens the daemon starts crash-looping with Failed to load OpenVINO GenAI library ... libopenvino_genai_c.so, which reads like a disaster but isn’t. Your config and your models are untouched. The fix is to reinstall the exact three packages from above and restart the service. If NPU dictation dies right after an update, that is the first thing to check.

Worth it?

Completely. I tap a key, talk, and the text shows up, in whichever of two languages I am in, without the fans spinning up and without a single word leaving the laptop. It is the first time the “AI” chip in this thing has done something I actually reach for every day, which feels like the NPU finally earning its spot on the spec sheet.

cd ../blog