Commit Graph
17 Commits
Author SHA1 Message Date
James Babcock dfde776016 Voice input tells you if it can't record due to microphone permissions 2026-09-12 00:47:27 -07:00
James Babcock 9b70880a5b Improve the Transcribe app 2026-09-05 19:16:56 -07:00
James Babcock 54005ae95a Preview mode uses phone mic 2026-08-30 20:04:33 -07:00
James Babcock e1c4628566 Merge branch 'main' of https://github.com/jimrandomh/faceclaw into calculator 2026-08-29 20:13:48 -07:00
James Babcock 756ee4e63b Fix mic drop issues
<details>
<summary>Prompts Used</summary>
Investigate and fix the bug in https://github.com/jimrandomh/faceclaw/issues/18

> Fixed. Here's what was going on and what changed.
>
> ## Root cause
>
> The issue's diagnosis was right, and the underlying flaw is a bit broader than the holder set: **`captureHolders` is phone-side bookkeeping, but the mic enable lives in the glasses' EvenHub session.** That enable dies — silently, from the phone's point of view — on a transport drop, the charging case, an EvenHub screen-off suspend, or a refused enable (`startG2AudioCapture` returns false when the display path isn't warm, and the Java worker then exits while the JS side still believes it started).
>
> `acquireCapture()` answered "is the mic running?" from the holder set, so a single stranded holder (a Transcribe window that was open when the link dropped) made every later request — wakeword assistant included — return early onto a stream that no longer existed, for the life of the process.
>
> The reported repro hits it via `resumeEvenHubSession()`, which resets `audioCaptureActive` and replays the prelude without re-enabling the mic; nothing on either side cleaned up.
>
> ## The fix
>
> **Java** — expose the real state of the stream instead of inferring it:
> - `FaceclawBleCommunicator.isAudioCaptureActive()` (`App_Resources/.../FaceclawBleCommunicator.java:363`) — true only while the session is up, not shut down, and the enable has been acked. Every teardown path already clears that flag.
> - `FaceclawVoiceController.isCapturing()` — `started` plus the communicator's view, with a lock-guarded `audioStarted` so the mic-warmup window doesn't read as dead.
>
> **`app/native/voice-control.ts`** — the bridge now asks `micIsLive()` rather than trusting the set, and tears a dead capture down before starting a new one. Added `handleSessionEnded()` (parks live holders, drops the local state) and `resumeCapture()` (restarts them on the next session); `releaseCapture`/`stop` clear the parked set. The raw-PCM tap for EvenHub mic apps had the identical staleness bug (`if (this.rawActive) return true`) and is covered the same way.
>
> **`app/g2/dashboard-controller.ts`** — wired the two session boundaries: any phase leaving `connected` parks the capture, and the first composited frame of the next session resumes it (a rendered frame is exactly the signal that a mic enable will be accepted). The screen-off EvenHub suspend is now deferred while a capture is held, so the screen timeout no longer kills a live transcription; a capture that races the suspend is parked by the same path.
>
> So Transcribe recovers on its own after a reconnect instead of sitting on a dead "Listening…", and the wakeword assistant is no longer blocked by anything left behind.
>
> ## Verification
>
> `./build.sh` (Java + webpack) clean, `tsc --noEmit` clean, `npm test` 61/61. Not committed.
>
> The behavior is all glasses-session state, which the test harness (pure modules only) can't reach, so it needs hardware. I wrote `notes/voice-capture-session-resume.md` with the five checks that matter — most importantly: Transcribe open → walk out of range → return → "Hey Even" should now get audio, and Transcribe should resume itself.
</details>
2026-08-29 10:16:52 -07:00
Kalani Helekunihi fe85846e6f Add calculator and microphone conversation apps
Add symbolic math and graphing tools, dual-microphone audio processing, live captions, translation, speaker diarization, sentiment analysis, and searchable conversation history.

Extend Android voice and BLE support for microphone control, recording, transcoding, and on-device speaker identification, with coverage for calculator and microphone workflows.
2026-08-25 00:22:19 -05:00
James Babcock 2b71eddab5 Optimize APK size
Removes unused keyword-spotting model. Makes on-device voice model a
runtime download rather than bundling it. Removes native-code library
versions for 32-bit and x86_64 processors.
2026-08-22 23:04:01 -07:00
James Babcock b86f395866 Add mic support for EvenHub apps 2026-08-16 21:01:20 -07:00
James Babcock ba81e46624 Add Soniox as an API speech-to-text option 2026-08-04 20:15:39 -07:00
James Babcock 008f2277ca Add OpenAI Whisper Realtime integration 2026-07-20 00:19:31 -07:00
James Babcock 161dbd104e Add wakeword support 2026-07-19 20:45:51 -07:00
James Babcock d0c30f9ca9 Port Transcribe to new UI; remove legacy dashboard 2026-07-07 22:11:28 -07:00
James Babcock 7fcab243a3 Debugged ElevenLabs STT; added debug-recording option 2026-07-05 21:00:40 -07:00
James Babcock 097d40efb2 Work on voice input 2026-07-05 20:59:53 -07:00
James Babcock bcc30545ad Add Transcribe screen 2026-05-01 15:00:53 -07:00
James Babcock 9996b09e9c First-pass G2 microphone streaming support 2026-04-30 20:00:16 -07:00
James Babcock 9fbb985db6 Add initial voice-command support
Adds a single wakeword-style voice command, "screen on", and deals with
all the wiring and permissions that entails. Currently this only
supports using the phone's mic (or a bluetooth headset), not the G2 mic.
2026-04-27 16:45:09 -07:00