Fix mic drop issues

<details>
<summary>Prompts Used</summary>
Investigate and fix the bug in https://github.com/jimrandomh/faceclaw/issues/18

> Fixed. Here's what was going on and what changed.
>
> ## Root cause
>
> The issue's diagnosis was right, and the underlying flaw is a bit broader than the holder set: **`captureHolders` is phone-side bookkeeping, but the mic enable lives in the glasses' EvenHub session.** That enable dies — silently, from the phone's point of view — on a transport drop, the charging case, an EvenHub screen-off suspend, or a refused enable (`startG2AudioCapture` returns false when the display path isn't warm, and the Java worker then exits while the JS side still believes it started).
>
> `acquireCapture()` answered "is the mic running?" from the holder set, so a single stranded holder (a Transcribe window that was open when the link dropped) made every later request — wakeword assistant included — return early onto a stream that no longer existed, for the life of the process.
>
> The reported repro hits it via `resumeEvenHubSession()`, which resets `audioCaptureActive` and replays the prelude without re-enabling the mic; nothing on either side cleaned up.
>
> ## The fix
>
> **Java** — expose the real state of the stream instead of inferring it:
> - `FaceclawBleCommunicator.isAudioCaptureActive()` (`App_Resources/.../FaceclawBleCommunicator.java:363`) — true only while the session is up, not shut down, and the enable has been acked. Every teardown path already clears that flag.
> - `FaceclawVoiceController.isCapturing()` — `started` plus the communicator's view, with a lock-guarded `audioStarted` so the mic-warmup window doesn't read as dead.
>
> **`app/native/voice-control.ts`** — the bridge now asks `micIsLive()` rather than trusting the set, and tears a dead capture down before starting a new one. Added `handleSessionEnded()` (parks live holders, drops the local state) and `resumeCapture()` (restarts them on the next session); `releaseCapture`/`stop` clear the parked set. The raw-PCM tap for EvenHub mic apps had the identical staleness bug (`if (this.rawActive) return true`) and is covered the same way.
>
> **`app/g2/dashboard-controller.ts`** — wired the two session boundaries: any phase leaving `connected` parks the capture, and the first composited frame of the next session resumes it (a rendered frame is exactly the signal that a mic enable will be accepted). The screen-off EvenHub suspend is now deferred while a capture is held, so the screen timeout no longer kills a live transcription; a capture that races the suspend is parked by the same path.
>
> So Transcribe recovers on its own after a reconnect instead of sitting on a dead "Listening…", and the wakeword assistant is no longer blocked by anything left behind.
>
> ## Verification
>
> `./build.sh` (Java + webpack) clean, `tsc --noEmit` clean, `npm test` 61/61. Not committed.
>
> The behavior is all glasses-session state, which the test harness (pure modules only) can't reach, so it needs hardware. I wrote `notes/voice-capture-session-resume.md` with the five checks that matter — most importantly: Transcribe open → walk out of range → return → "Hey Even" should now get audio, and Transcribe should resume itself.
</details>
This commit is contained in:
James Babcock committed 2026-08-29 10:16:52 -07:00
1 parent fe28a6de4b
commit 756ee4e63b
4 files changed
+188 -15

No files matched your search

+45 -10
View File
@@ -702,6 +702,14 @@ class DashboardController {
return;
}
if (voiceControlBridge.isCaptureHeld()) {
// Ending the plugin task ends the mic with it, so a live capture (the
// Transcribe window, typically) outranks the screen-off power save.
// Check before the lease refresh below, which costs a BLE message.
this.scheduleEvenHubSuspend();
return;
}
void (async () => {
if (this.faceclawWakeLeaseSupported) {
// Refresh immediately before teardown so the first suspended wake
@@ -727,6 +735,9 @@ class DashboardController {
.then((suspended) => {
if (suspended === undefined) return;
if (suspended) {
// A capture that raced the suspend lost its mic with the plugin
// task; park it for the resume path.
voiceControlBridge.handleSessionEnded();
this.appendLog("EvenHub session suspended while screen is off");
return;
}
@@ -1036,6 +1047,10 @@ class DashboardController {
// the transport comes back, so do not make lock decisions from a
// stale pre-disconnect value in the meantime.
this.glassesWorn = null;
// The mic enable was session-scoped too: park any live capture so
// the next session restarts it, instead of leaving a holder that
// makes every later request think the mic is already running.
voiceControlBridge.handleSessionEnded();
}
this.setPhase(mappedPhase);
this.setStatus(state.status);
@@ -1098,6 +1113,10 @@ class DashboardController {
this.schedulePreviewUpdate();
if (this.phase === "connected") {
this.setStatus("Connected.");
// A rendered frame also means the display path is warm enough for
// the glasses to accept a mic enable, so this is where a capture
// parked by the previous session (or by an EvenHub suspend) resumes.
this.resumeVoiceCapture();
// A rendered frame means the session is warmed up (fixedLayoutCreated),
// so the buzzer won't be dropped. Play the one-time welcome sound now.
if (this.welcomeSoundArmed) {
@@ -1390,7 +1409,7 @@ class DashboardController {
// Faceclaw's final BLE message before the transport closes.
await mediaControllerBridge.stop().catch(() => {});
await nightscoutBridge.stop().catch(() => {});
voiceControlBridge.stop();
voiceControlBridge.handleSessionEnded();
const cleanupAcked = skipFirmwareCleanup
? false
@@ -1558,6 +1577,30 @@ class DashboardController {
voiceControlBridge.stopContinuousCapture();
}
/** Provider and key settings for a capture on the given connection. */
private voiceCaptureOptions(communicator: FaceclawCommunicatorBridge) {
return {
communicator: communicator.getNativeCommunicator(),
provider: voiceProviderSetting.get(),
elevenLabsApiKey: elevenLabsApiKeySetting.get(),
openAiApiKey: openAiApiKeySetting.get(),
sonioxApiKey: sonioxApiKeySetting.get(),
saveRecording: saveVoiceRecordingsSetting.get(),
};
}
/**
* Restart the mic for a capture that a previous session left holding. Cheap
* to call per frame: it does nothing unless one is actually parked.
*/
private resumeVoiceCapture(): void {
if (!voiceControlBridge.hasSuspendedCapture()) return;
const communicator = this.communicator;
if (!communicator || this.phase !== "connected") return;
this.appendLog("resuming voice capture on the new glasses session");
voiceControlBridge.resumeCapture(this.voiceCaptureOptions(communicator));
}
private beginVoiceCapture(kind: "ptt" | "continuous", endpointing = false): void {
if (this.phase !== "connected" || !this.communicator) {
return;
@@ -1566,15 +1609,7 @@ class DashboardController {
void ensureVoicePermissions()
.then(() => {
if (this.phase !== "connected" || this.communicator !== communicator) return;
const options = {
communicator: communicator.getNativeCommunicator(),
provider: voiceProviderSetting.get(),
elevenLabsApiKey: elevenLabsApiKeySetting.get(),
openAiApiKey: openAiApiKeySetting.get(),
sonioxApiKey: sonioxApiKeySetting.get(),
saveRecording: saveVoiceRecordingsSetting.get(),
endpointing,
};
const options = { ...this.voiceCaptureOptions(communicator), endpointing };
if (kind === "ptt") {
voiceControlBridge.startPushToTalk(options);
} else {