Personal project · shipped to external testers · Aug 2026
Nod
A desktop voice assistant and live meeting HUD
A Windows overlay that listens for a wake word, answers out loud in a locally-synthesised Philippine-English voice, and keeps a rolling summary of the meeting you're in — with microphone and speaker audio kept structurally incapable of reaching the same code path.
- intent accuracy
- 98.4%
- p50 classification
- 606ms
- end to end, CPU only
- 2–4s
- lines of Python
- 11.4k
- test suites
- 16
- realtime local TTS
- 14×
01
Two halves, one overlay
Nod is a frameless, click-through, always-on-top card pinned to the top of the screen. The agent half listens to the microphone for a wake word, transcribes locally with Whisper, classifies intent and answers out loud. The meeting half captures speaker audio through WASAPI loopback, OCRs the screen, and keeps a rolling 120-second summary card of the call you're in.
It is a single Python process: five daemon worker threads plus the Qt main thread, connected only by queues. Workers never touch widgets — the HUD drains its queue on a timer on the main thread.
One nice detail: the overlay publishes its own rectangle so the screen reader can blank that region before OCR. Without it, Nod reads its own last suggestion off the screen and confirms itself in a loop.
Rina asked whether the Q3 migration slips if the vendor contract is not signed by Friday.
Suggested · say
“Contract is with legal — I’ll have an answer by the Thursday standup.”
02
A meeting cannot give Nod orders
Microphone and speaker audio never share a queue. Two capture threads, two transcription workers, two separate Whisper models — and the queue that carries commands is fed only by the microphone.
So a webinar host saying "hey Nod, join the next call" is structurally incapable of issuing a command. Not filtered, not scored, not prompt-guarded: there is no path. That is the one rule in the codebase written down as non-negotiable.
The same instinct closed a command-injection hole at the data layer. Nod's speech is assembled from calendar titles and speech recognition, so the text-to-speech shim takes text as data on stdin and never interpolates it into a shell command. A meeting titled with a shell substitution gets read aloud instead of run.
03
The accent is a phoneme transformation
Piper's voice catalogue has no Philippine English. The options were to mangle English through a Tagalog model, or to clone a speaker. I did neither.
Instead the synthesiser is intercepted between phonemisation and audio generation, and documented sound changes are applied to the IPA directly: TH-stopping, rhoticity, no intervocalic flapping, final /z/ devoicing, no vowel reduction, TRAP lowering. No dataset, no training, no cloning — and it is testable without audio, because you can feed it IPA and assert on IPA.
The two features most associated with caricature are implemented and off by default. Every substitution target is validated against the model's own phoneme table first, because an unknown phoneme id renders as silence, and a word vanishing mid-sentence is worse than a word with the wrong accent.
04
Prompting beat fine-tuning
Local intent classification started at 87.3% zero-shot on a 3B model. Sixteen worked examples took it to 98.4% — 11.1 points for about 25 milliseconds, with zero malformed JSON across 252 calls.
The next 3.8 points came from deleting seven rows of my own dataset, not from tuning. Reading the confusion table showed the model was being punished for an ASR error I had invented rather than observed.
What I did not do was chase the last few points. The remaining errors are roughly 60% Tagalog imperatives that a 3B model will not handle, 20% genuinely ambiguous cases where no correct label exists, and 20% my own bad labels. Deleting the Tagalog rows would have pushed the headline above 97% and meant nothing.
zero-shot
87.3%
16 worked examples
98.4%
05
The optimisation I reverted
Removing an image upscale before OCR made that stage 1.6× faster and looked obviously correct. It also retained 13% of the text — 2,556 characters against 3,548 on the same frame.
I put it back, and left the measurement in a comment so nobody optimises it again.
The wake word got the same treatment. The performance log showed 7 of 18 attempts matching; the other 11 had decoded as "none", "Nun", "Nahn", "Null" and "nerd", scoring between 0.57 and 0.75 against the target. That range proves no threshold value could have fixed it, which is why the fix was a two-tier name table rather than a tuned number.