Add on-device Whisper as a second offline transcription option

sherpa-onnx's native library already reads an offline Whisper config
(verified against sherpa-onnx/jni/offline-recognizer.cc at v1.13.0);
only the Java wrapper was a package-private stub. This adds a public
OfflineWhisperModelConfig builder and wires base.en (int8) in as a
second on-device model next to Moonshine.

- FaceclawVoiceController: setOnboardModelKind() selects the model
  before start("onboard"); defaults to Moonshine. Whisper skips the
  700ms live-partial redecode (each call re-encodes the whole utterance
  plus ~10s of tail padding) and gates near-silent segments before
  decode.
- asr-model.ts: ASR_MODELS registry with per-model download state.
- Settings > Voice: "On-device (Moonshine)", "On-device (Whisper)",
  and "OpenAI (Whisper)" for the existing cloud provider (stored
  value unchanged); one model row per on-device model.
- Microphones live captions pass "moonshine" explicitly (unchanged).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Chris LeeandClaude Sonnet 5 committed 2026-09-14 22:46:22 -04:00
1 parent dcb071621d
commit 2d717a9cdb
9 files changed
+459 -162

No files matched your search

+4 -1
View File
@@ -730,7 +730,10 @@ class MicSession {
},
});
engine.setListener(this.captionListenerProxy);
engine.setAsrModelDir(isAsrModelReady() ? moonshineModelDir() : null);
// Live captions are a separate sherpa-onnx pipeline (FaceclawCaptionEngine)
// that only ever uses Moonshine; unaffected by the two-model on-device
// Whisper addition in native/voice-control.ts's push-to-talk path.
engine.setAsrModelDir(isAsrModelReady("moonshine") ? moonshineModelDir() : null);
engine.setSpeakerModelPath(micModelPath("speaker-embedding"));
engine.start();
this.captionEngine = engine;
+6 -3
View File
@@ -328,17 +328,20 @@ function voiceSpeakersMenu(): MenuLayer {
() => startMicModelDownload("speaker-embedding"),
),
modelItem(
// Live captions always use Moonshine (see the matching note in
// mic-session.ts); the second on-device model (Whisper) is a
// push-to-talk dictation option only, managed in Settings > Voice.
"Caption ASR model",
isAsrModelReady,
() => isAsrModelReady("moonshine"),
() => {
const state = asrModelState();
const state = asrModelState("moonshine");
if (state.status === "ready") return "ready";
if (state.status === "downloading") {
return `${Math.round((state.bytesDownloaded / state.totalBytes) * 100)}%`;
}
return "download (141 MB)";
},
startAsrModelDownload,
() => startAsrModelDownload("moonshine"),
),
modelItem(
"Re-diarization model",