forked from Blendman974/faceclaw
69 lines
4.2 KiB
Markdown
69 lines
4.2 KiB
Markdown
# Whisper transcription languages
|
|
|
|
Settings > Voice > Transcription language selects French, English, 27 other
|
|
languages, or auto-detection. It applies to ordinary dictation and continuous
|
|
transcription using OpenAI, on-device Whisper, and the other cloud providers.
|
|
Translate retains its own spoken-language preference. Moonshine remains
|
|
English-only; Apple's on-device speech recognizer retains its own locale.
|
|
|
|
The initial **Provider default** preserves existing installations: English for
|
|
on-device Whisper, auto-detection for OpenAI/other cloud providers, and the saved
|
|
Whisper server language for self-hosted Whisper. Selecting another shared
|
|
language overrides the server language without deleting it. Returning to Provider
|
|
default restores it. Settings persist through the existing settings store.
|
|
|
|
For French or auto-detection with **On-device (Whisper)**, download
|
|
**On-device model: Whisper (multilingual, base)** in Settings > Voice (~161 MB).
|
|
English uses the existing English-only base model; neither model is deleted or
|
|
downloaded automatically. A missing model uses the existing “Voice model not
|
|
downloaded” status. Language changes restart active continuous transcription;
|
|
other captures use the new language when started.
|
|
|
|
## Software validation
|
|
|
|
- JavaScript regression tests exercise stored settings/reload, compatible
|
|
defaults, French and auto-detection, Translate overrides in Android/iOS capture
|
|
options, the OpenAI WebSocket configuration, and local bridge restart/sharing.
|
|
- Kotlin tests exercise English/French/other/automatic model selection and file
|
|
names. Android release compilation checks the actual sherpa-onnx bridge.
|
|
- New model downloads use a pinned Hugging Face revision, per-file SHA-256 and
|
|
the existing resumable downloader. Encoder/decoder hashes and sizes come from
|
|
repository LFS metadata; the vocabulary hash was computed from its contents.
|
|
|
|
## Device validation still required
|
|
|
|
With both phone microphone preview and glasses microphone, check French dictation
|
|
(including accents), auto-detection, English rollback, continuous capture while
|
|
changing language, cancellation/resumption of model download, and persistence
|
|
after restarting the app. Compare OpenAI and offline Whisper on the same phrases.
|
|
No paid API call, microphone recording, or firmware update is needed for the
|
|
software checks above; recognition accuracy and latency require device testing.
|
|
|
|
## References
|
|
|
|
- [OpenAI Realtime transcription: language configuration](https://developers.openai.com/api/docs/guides/realtime-transcription)
|
|
- [sherpa-onnx Whisper configuration and automatic language detection](https://github.com/k2-fsa/sherpa-onnx/blob/master/sherpa-onnx/csrc/offline-whisper-model-config.cc)
|
|
- [Multilingual base model, pinned revision](https://huggingface.co/csukuangfj/sherpa-onnx-whisper-base/tree/bb53ee204431c90d314c1cc08d28d23e5b7927cc)
|
|
|
|
## OpenAI transcription model selection
|
|
|
|
Settings > Voice > OpenAI transcription model selects gpt-realtime-whisper,
|
|
gpt-4o-mini-transcribe or gpt-4o-transcribe. For a project whose model list lacks
|
|
gpt-realtime-whisper but includes gpt-4o-mini-transcribe, select the latter.
|
|
The unset default retains gpt-realtime-whisper for existing installations; no
|
|
provider, language or Assistant model preference is overwritten. This setting
|
|
applies to Translate's speech recognition as well as Transcribe and dictation.
|
|
The two transcribe alternatives produce transcript deltas after a committed
|
|
audio segment; existing pause segmentation and push-to-talk release supply the
|
|
commits. Choosing a model retargets active continuous capture.
|
|
|
|
Model access errors name the requested model before the long project error and
|
|
point to the setting. Factory/session-wire tests cover model, language, manual
|
|
commit and final transcript behavior; settings tests cover persistence and the
|
|
Android/iOS option plumbing. No real API inference was performed; actual project
|
|
access, recognition and latency remain to be checked on a device. The model list
|
|
provided by the operator establishes account visibility, not a successful audio
|
|
request.
|
|
|
|
Reference: [OpenAI transcription session API](https://developers.openai.com/api/reference/resources/realtime/subresources/transcription_sessions/methods/create).
|