forked from Blendman974/faceclaw
Add on-device Whisper as a second offline transcription option
sherpa-onnx's native library already reads an offline Whisper config
(verified against sherpa-onnx/jni/offline-recognizer.cc at v1.13.0);
only the Java wrapper was a package-private stub. This adds a public
OfflineWhisperModelConfig builder and wires base.en (int8) in as a
second on-device model next to Moonshine.
- FaceclawVoiceController: setOnboardModelKind() selects the model
before start("onboard"); defaults to Moonshine. Whisper skips the
700ms live-partial redecode (each call re-encodes the whole utterance
plus ~10s of tail padding) and gates near-silent segments before
decode.
- asr-model.ts: ASR_MODELS registry with per-model download state.
- Settings > Voice: "On-device (Moonshine)", "On-device (Whisper)",
and "OpenAI (Whisper)" for the existing cloud provider (stored
value unchanged); one model row per on-device model.
- Microphones live captions pass "moonshine" explicitly (unchanged).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
dcb071621d
commit
2d717a9cdb
9 files changed
+459
-162
No files matched your search
@@ -730,7 +730,10 @@ class MicSession {
|
||||
},
|
||||
});
|
||||
engine.setListener(this.captionListenerProxy);
|
||||
engine.setAsrModelDir(isAsrModelReady() ? moonshineModelDir() : null);
|
||||
// Live captions are a separate sherpa-onnx pipeline (FaceclawCaptionEngine)
|
||||
// that only ever uses Moonshine; unaffected by the two-model on-device
|
||||
// Whisper addition in native/voice-control.ts's push-to-talk path.
|
||||
engine.setAsrModelDir(isAsrModelReady("moonshine") ? moonshineModelDir() : null);
|
||||
engine.setSpeakerModelPath(micModelPath("speaker-embedding"));
|
||||
engine.start();
|
||||
this.captionEngine = engine;
|
||||
|
||||
@@ -328,17 +328,20 @@ function voiceSpeakersMenu(): MenuLayer {
|
||||
() => startMicModelDownload("speaker-embedding"),
|
||||
),
|
||||
modelItem(
|
||||
// Live captions always use Moonshine (see the matching note in
|
||||
// mic-session.ts); the second on-device model (Whisper) is a
|
||||
// push-to-talk dictation option only, managed in Settings > Voice.
|
||||
"Caption ASR model",
|
||||
isAsrModelReady,
|
||||
() => isAsrModelReady("moonshine"),
|
||||
() => {
|
||||
const state = asrModelState();
|
||||
const state = asrModelState("moonshine");
|
||||
if (state.status === "ready") return "ready";
|
||||
if (state.status === "downloading") {
|
||||
return `${Math.round((state.bytesDownloaded / state.totalBytes) * 100)}%`;
|
||||
}
|
||||
return "download (141 MB)";
|
||||
},
|
||||
startAsrModelDownload,
|
||||
() => startAsrModelDownload("moonshine"),
|
||||
),
|
||||
modelItem(
|
||||
"Re-diarization model",
|
||||
|
||||
Reference in new issue
Block a user