Parakeet vs. Whisper on an AMD iGPU: a speech-to-text benchmark on Linux
On an integrated Radeon 890M, Parakeet Ultra makes 32-57% fewer errors than Whisper small and transcribes a 10 second phrase in 0.11 s.
Setup
The laptop has an AMD Ryzen AI 9 HX 370 (12 cores, 24 threads) with an integrated Radeon 890M and runs Fedora 44. GPU runs use Vulkan through the RADV driver.
The audio is the test split of FLEURS, 300 sentences per language: 49 minutes of English, 57 of Russian, 65 of German. All files are 16 kHz mono, 16-bit PCM.
Two engines ran the models: whisper.cpp v1.9.5 (libwhisper and libparakeet from the Dabri 2.4.1 RPM) and transcribe.cpp v0.4.2 built with Vulkan and OpenBLAS. Decoding is greedy. Whisper was told the language, Parakeet detects it.
WER is the percentage of words the model got wrong, lower is better. The range in parentheses is a 95% confidence interval. Text is lowercased and stripped of punctuation before scoring and nothing else is normalized, so “20” and “twenty” count as different words and the numbers are higher than on public leaderboards.
Speed is seconds of audio per second of compute, without model loading. The value in parentheses is the time for a 10 second phrase.
Integrated GPU
| Model | Engine | File | English | Russian | German | Speed |
|---|---|---|---|---|---|---|
| Whisper small q5_1 | whisper.cpp | 190 MB | 7.23 (6.3-8.2) | 12.20 (10.9-13.5) | 10.71 (9.7-11.8) | 36x (0.28 s) |
| Parakeet TDT 0.6B v3 q8_0 | whisper.cpp | 669 MB | 5.44 (4.7-6.2) | 6.05 (5.2-6.9) | 5.46 (4.7-6.3) | 79x (0.13 s) |
| Parakeet TDT 0.6B v3 q4_k | whisper.cpp | 416 MB | 5.50 (4.8-6.3) | 6.08 (5.3-6.9) | 5.43 (4.7-6.3) | 68x (0.15 s) |
| Parakeet TDT 0.6B v3 q4_0 | whisper.cpp | 356 MB | 5.86 (5.0-6.7) | 6.69 (5.8-7.6) | 5.84 (5.1-6.7) | 87x (0.11 s) |
| Parakeet TDT 0.6B v3 Q8_0 | transcribe.cpp | 740 MB | 5.37 (4.6-6.1) | not run | not run | 90x (0.11 s) |
| Parakeet Ultra Q8_0 | transcribe.cpp | 740 MB | 5.09 (4.4-5.8) | 5.16 (4.4-5.9) | 4.73 (4.0-5.5) | 82x (0.12 s) |
| Parakeet Ultra Q4_K_M | transcribe.cpp | 486 MB | 4.92 (4.3-5.7) | 5.45 (4.7-6.3) | 4.64 (3.9-5.4) | 87x (0.11 s) |
| GigaAM v3 e2e-rnnt Q8_0 | transcribe.cpp | 274 MB | Russian only | 5.16 (4.3-6.1) | Russian only | 117x (0.09 s) |
Against Whisper small, Parakeet Ultra Q4_K_M makes 32% fewer errors in English, 55% fewer in Russian and 57% fewer in German, and runs 2.4 times faster.
Sentence by sentence
The same 300 sentences, one model against another. The number is the change in WER with its 95% interval. Negative means fewer errors. An interval that crosses zero means no measurable difference.
| Comparison | English | Russian | German |
|---|---|---|---|
| Ultra Q8_0 vs v3 q8_0 | -0.36 [-0.69, -0.03] | -0.89 [-1.31, -0.50] | -0.72 [-1.17, -0.30] |
| Ultra Q4_K_M vs v3 q8_0 | -0.52 [-0.86, -0.18] | -0.60 [-1.03, -0.17] | -0.81 [-1.28, -0.37] |
| v3 q4_k vs v3 q8_0 | +0.06 [-0.21, +0.31] | +0.03 [-0.30, +0.36] | -0.03 [-0.34, +0.28] |
| v3 q4_0 vs v3 q8_0 | +0.42 [0.00, +0.91] | +0.64 [+0.22, +1.04] | +0.38 [+0.02, +0.75] |
Ultra beats v3 in every language. The q4_k file of v3 is as accurate as q8_0 and 250 MB smaller. The q4_0 file is worse.
On Russian, GigaAM v3 and Parakeet Ultra Q8_0 both score 5.16, with a paired difference of 0.00 [-0.73, +0.71]. The same v3 weights on the two engines differ by -0.07 [-0.22, +0.06] in English, so the engines agree.
Large Whisper models
These are slow on this GPU, so they ran on every third sentence, 100 per language. The intervals are wider.
| Model | File | English | Russian | German | Speed |
|---|---|---|---|---|---|
| Whisper large-v3 q5_0 | 1081 MB | 5.37 (4.3-6.5) | 4.19 (3.1-5.4) | 5.31 (3.9-6.9) | 9x (1.1 s) |
| Whisper large-v3-turbo q5_0 | 574 MB | 5.73 (4.5-7.0) | 5.18 (3.7-6.9) | 5.88 (4.6-7.1) | 17x (0.59 s) |
| Parakeet Ultra Q4_K_M | 486 MB | 5.24 (4.1-6.4) | 5.29 (3.9-6.7) | 5.08 (3.9-6.3) | 87x (0.11 s) |
| Parakeet TDT 0.6B v3 q8_0 | 669 MB | 6.26 (5.0-7.5) | 6.11 (4.7-7.4) | 6.35 (4.9-7.8) | 79x (0.13 s) |
| Whisper small q5_1 | 190 MB | 8.12 (6.6-9.7) | 10.46 (8.5-12.4) | 11.61 (9.7-13.7) | 35x (0.29 s) |
Parakeet Ultra is within the margin of error of large-v3-turbo and five times faster.
CPU only
Speed and memory on 40 English sentences (GigaAM on 40 Russian ones).
| Model | Engine | Threads | Speed | Peak memory |
|---|---|---|---|---|
| Whisper small q5_1 | whisper.cpp | 24 | 7.9x (1.27 s) | 393 MB |
| Parakeet v3 q8_0 | whisper.cpp | 4 | 12.5x (0.80 s) | 816 MB |
| Parakeet v3 q8_0 | whisper.cpp | 12 | 25.3x (0.40 s) | 816 MB |
| Parakeet v3 q8_0 | whisper.cpp | 24 | 12.9x (0.78 s) | 817 MB |
| Parakeet v3 q4_k | whisper.cpp | 12 | 39.3x (0.25 s) | 581 MB |
| Parakeet v3 q4_k | whisper.cpp | 24 | 24.9x (0.40 s) | 587 MB |
| Parakeet Ultra Q4_K_M | transcribe.cpp | 12 | 42.1x (0.24 s) | 1008 MB |
| GigaAM v3 e2e-rnnt Q8_0 | transcribe.cpp | 12 | 44.7x (0.22 s) | 389 MB |
libparakeet defaults to 4 threads. With 12, one per physical core, it runs twice as fast. With all 24 logical threads the speed drops back to where it started. For q4_k, 6, 8 and 16 threads gave 27.7x, 33.1x and 36.1x.
transcribe.cpp maps the model file into memory and that mapping is counted in its peak, so memory is not comparable between the engines.
Limits
One laptop, three languages. FLEURS is read speech, cleaner than dictation. Parakeet covers 25 European languages, Whisper covers 99.
Phonon-2.5 and Parakeet Redux are not in the tables. Both are compressed versions of Parakeet v3 that run only through their own Python runtimes.
In Dabri
Dabri gets the q4_k file of Parakeet v3 in its catalog, and Parakeet now runs one thread per physical core. transcribe.cpp joins whisper.cpp as a second engine library and brings Parakeet Ultra Q4_K_M as a new model in the catalog. Whisper small stays the bundled default: it is 190 MB against 486 MB and covers 99 languages. Ultra needs a CPU with AVX2.
Thanks for reading, Asher