The squeeze
Voice is a natural human interface now found in products ranging from appliances and wearables to industrial equipment. Most of them rely on deep neural networks (DNNs) to recognize commands, trigger wake words, and separate speech from noise. These systems often integrate multiple models, audio frontends, classifiers, and post-processing logic, all of which must operate in real time on embedded silicon with tight memory and compute constraints. As a result, model compression is essential for practical deployment.
This post walks through one such effort. We took a voice command detector trained to recognize a single multiword phrase (such as “Open the front storage cabinet”) and asked a simple question: how small can we make it while preserving the user experience? We shrank the model footprint by nearly 90%, from 500 KB to 60 KB, while holding a 95% true positive rate at 0.25 false alerts per day. In plain terms, the compressed model still caught 95 of every 100 real commands, and it falsely triggered only four times across 383 hours (over 16 days) of everyday background audio.
The technique that got us there is weight pruning, and it pairs naturally with femtoAI’s software and hardware.
Why pruning
Of the many ways to shrink a neural network, weight pruning is especially attractive: it reduces model size without requiring architectural redesign or extensive retraining. The femtoAI Sparse Processing Unit, or SPU, is a chip designed to take advantage of sparse weights directly: when a weight is zero, the SPU can skip the work attached to it rather than grinding through a multiplication that adds nothing to a layer’s output. On a conventional processor a pruned model often runs no faster, because the zeros still travel through the same pipeline. On the SPU, sparsity turns into real savings in memory, energy, and latency. Getting the full benefit requires both halves working together, a sparse network on one side and sparsity-aware silicon on the other.
The specific chip used in this study is the SPU-001. It carries 1 MB of on-chip SRAM, but because it stores and runs sparse models efficiently, its effective model capacity is roughly 10 MB. Seen against that budget, the 60 KB detector in this study is small, with ample room left for other models and a shared audio frontend.
We detail how pruning affects quantized-model accuracy across a range of compression levels and operating points. Throughout, we use femtoAI’s SDK for compression and footprint evaluation.
How the model works
The voice command detector has two parts: an audio frontend that turns raw sound into a compact set of features, and a classifier that reads those features and decides whether the command was spoken.
The audio frontend
We operate on 16 kHz audio and begin with a Hann-windowed short-time Fourier transform (STFT), which measures how energy at each frequency evolves over time, using a 32 ms window and a 16 ms hop.
From the STFT we extract 64-dimensional log-mel filterbank features, where the mel scale spaces the frequency bands to match human hearing and the log compresses their dynamic range. We then replace the static bands with their first-order temporal difference, keeping the feature width at 64. We call the result the delta log-mel filterbank, or ΔLMFB.
The frontend carries a footprint of its own. Compiled, the ΔLMFB stage comes to about 46 KB, which you should add on top of the classifier when budgeting for a full deployment. That overhead is easier to absorb than it looks: the frontend, and especially the STFT underneath it, is usually shared across several models in an audio system. In that case, it is computed once and reused by every downstream model, with no duplicated work or memory.
The classifier
The features (64 per frame) feed a classifier built on a long short-term memory (LSTM) network, a recurrent design that carries an internal state across time. We use a hidden dimension of 128 and vary the depth between two and four LSTM layers. The final layer projects to per-class output scores, including a blank label for the decoding scheme described below, producing one output every 16 ms, so a feature sequence of length T yields T sets of scores. Uncompressed, the model contains 281k parameters at two layers and 553k at four.
How we trained it
We train the model in four phases. Three of these phases, pre-training, connectionist fine-tuning, and quantization-aware training, rely on standard techniques that most modern training stacks support. What ties the pipeline to our hardware is how we prune: the sparsity is structured to match what the SPU can skip, so the compression becomes real on-device savings rather than zeros a conventional processor would still push through.
1. Pre-training
We first train the model with a straightforward objective, averaging its frame-by-frame outputs over time and comparing them to the correct label. At this stage each training clip contains at most one command, since the model cannot yet track multiple events in a sequence.
2. Pruning
Next we prune while continuing to train, so the network can adapt as weights are removed rather than taking the hit all at once. This method protects model performance by making the model “pruning-aware.” We use a pattern called 4×1 block sparsity, which zeroes weights in small, aligned blocks rather than scattering the zeros at random, because structured blocks are what the SPU is built to skip efficiently. The pruning threshold, the cutoff below which weights get zeroed, rises every 100 minibatches on a quarter-sine schedule, easing the pressure up gradually.
3. Connectionist fine-tuning
With the model pruned, we switch to a modified Connectionist Temporal Classification loss, usually abbreviated CTC. CTC is a training method built for sequences where you know what was said but not exactly when each piece was said, which is precisely the situation in continuous audio. Our version is tuned for greedy decoding (reading the model’s output in a single fast pass) and for keeping false alarms low. The clips in this phase contain several commands in sequence, so the model learns to spot commands continuously rather than one clip at a time.
4. Quantization-aware training
Finally we quantize the model, replacing its high-precision parameters with much smaller integer representations, and continue training so the network learns to compensate for any errors introduced by rounding. Weights end up as 8-bit integers and the activations and biases as 16-bit integers. We carry the best-performing checkpoints from the previous phase into this one, chosen by their accuracy at fixed false-alarm targets.
The data
A voice detector is only as good as what it hears during training. We used four kinds of data:
- A labeled dataset of the target voice command: a relatively long, multiword phrase of moderate complexity (for example, “Open the front storage cabinet”).
- A large collection of background audio (music, TV, human speech) in multiple languages.
- A small distractor set of phonetically similar phrases (for example, partial fragments and sound-alikes of the target command), generated with text-to-speech to reduce false triggers.
- Room impulse responses (RIRs), both real and synthetic, to simulate reverberation. Convolving clean audio with an RIR makes it sound as though it were recorded in that space.
During training, we mix commands and distractors with background noise at random signal-to-noise ratios (SNRs) from 0 to 20 decibels, and convolve them with RIRs. In the final training phases, we include multiple target phrases in sequence, teaching the model to detect commands continuously over time.
How we measured success
Two numbers matter for a detector like this, and they pull against each other. The true positive rate, or TPR, is the share of real commands the model correctly catches, where higher is better. The false alert rate, which we report as false alerts per day (FA/day), is how often the model fires when no command is spoken, where lower is better. Turn the sensitivity up and you catch more real commands but also raise more false alarms. Turn it down and the reverse happens.
To place the model on this curve, we swept across a range of false-alarm targets (0, 0.125, 0.25, and 0.5 FA/day) and recorded the best TPR it could reach. False alarms were counted over a full 383-hour dataset of background audio. TPR was averaged across a range of conditions: babble, music, and range-hood noise, each at 5, 10, and 15 decibels of SNR, plus a clean, quiet recording. That spread keeps the headline number from reflecting only the easy, quiet cases.
Results
The headline finding is that at realistic false-alarm targets, pruning affects performance very little. At 0.125 FA/day and looser, the curves stay high and flat across a wide range of sizes: a 60 KB model reaches nearly 95% TPR at 0.25 FA/day. Because that whole range holds up, most of the parameters we removed were redundant and contributed little to the model’s output. Moreover, 0.25 FA/day is a demanding operating point: it works out to just four false alerts across the entire 383-hour dataset.
At 0 FA/day, where we tolerate not a single false alarm over 383 hours, the true positive rate for models under 60 KB falls below 50%. The cause is one stubborn, high-confidence false alarm in the dataset. Eliminating it requires setting the threshold so high that a large share of real commands get rejected along with it. This is the tension between perfect rejection and good recall in its rawest form, and it is why a small, sensible false-alarm budget can be a worthwhile tradeoff.
Read together, the curves point to a comfortable operating regime: tolerate a modest false-alarm rate such as 0.25 FA/day, and you can compress the model by 80 to 90% with little loss in accuracy. For a low-memory embedded device, that is an easy choice to make.
Takeaway
Together, structured pruning and quantization place a capable voice command model on the SPU-001 with little accuracy lost. Our compressed models shrink by nearly 90%, from 500 KB to as little as 60 KB, and still catch about 95% of real commands at 0.25 FA/day, which is four false positives in 383 hours of background audio. The result holds up across a range of noise and reverberation conditions, and only breaks down at the strictest zero-false-alarm setting.
The payoff is intrinsically tied to the SPU. Other accelerators can run the pruned and quantized model, but on any processor that treats weights as dense, the zeros still occupy memory and consume power, so the compression never turns into real savings. Those gains show up only on silicon built to skip the zeroed blocks. For teams building compact, always-on voice interfaces on constrained hardware, pruning and quantization are practical, dependable tools, and they show why a sparse network paired with sparsity-aware silicon is such a natural fit for the edge.
