Commit Graph
176 Commits
Author SHA1 Message Date
James Babcock 25c398d60f Wait for final results from Whisper before completing
<details>
<summary>Prompts Used</summary>
User: This checkout corresponds to https://github.com/jimrandomh/faceclaw/pull/27. Review this branch.

GPT-6 Astra:
> Found one issue:
>
> - **[P2] Wait for Whisper’s final result before completing dictation** — [FaceclawVoiceController.java:729](/Users/jbabcock/repositories/g2/faceclaw/App_Resources/Android/src/main/java/com/faceclaw/app/FaceclawVoiceController.java:729). Skipping partial decodes leaves short utterances without text until release, but the dialog still finalizes after its existing timeout. If decoding exceeds roughly 2.7 seconds (native stop’s 1.5-second wait plus the dialog’s 1.2-second timeout), auto-send can abandon a short command or send an incomplete longer one. Continue can append the late editing instruction literally instead of refining the message. I reproduced these outcomes with delayed transcript events. These flows should await recognition completion.
>
> Validation: TypeScript and a clean Android build passed. All 21 test failures also occur on `main`. Actual Whisper latency was not tested on hardware.

<hr>

User: Thanks, let's fix that issue.

GPT-6 Astra:
> Fixed. Dictation now waits for native recognition to finish before fallback timers run. Auto-send, Continue, and AI Chat preserve delayed Whisper results; cancellation ignores late transcripts.
>
> TypeScript and Android builds pass. All 11 new regression tests pass. The full suite still has the same 21 pre-existing failures.
</details>
2026-09-17 22:12:33 -07:00
James Babcock 91890a25ac Merge branch 'pr-ondevice-whisper' of https://github.com/leofingal/faceclaw into pr-ondevice-whisper 2026-09-17 21:59:28 -07:00
James Babcock 5698c584a3 Expand texture cache to 256kb 2026-09-17 18:52:50 -07:00
James Babcock b9b8128756 BLE bandwidth benchmark uses incompressible messages 2026-09-17 18:52:14 -07:00
James Babcock 720f6af8f8 Do message compression at the transport level, rather than in individual messages 2026-09-16 22:47:56 -07:00
Chris LeeandClaude Sonnet 5 2d717a9cdb Add on-device Whisper as a second offline transcription option
sherpa-onnx's native library already reads an offline Whisper config
(verified against sherpa-onnx/jni/offline-recognizer.cc at v1.13.0);
only the Java wrapper was a package-private stub. This adds a public
OfflineWhisperModelConfig builder and wires base.en (int8) in as a
second on-device model next to Moonshine.

- FaceclawVoiceController: setOnboardModelKind() selects the model
  before start("onboard"); defaults to Moonshine. Whisper skips the
  700ms live-partial redecode (each call re-encodes the whole utterance
  plus ~10s of tail padding) and gates near-silent segments before
  decode.
- asr-model.ts: ASR_MODELS registry with per-model download state.
- Settings > Voice: "On-device (Moonshine)", "On-device (Whisper)",
  and "OpenAI (Whisper)" for the existing cloud provider (stored
  value unchanged); one model row per on-device model.
- Microphones live captions pass "moonshine" explicitly (unchanged).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 22:46:22 -04:00
James Babcock 33bfdcf1d1 Send messages through a new SID, rather than image messages 2026-09-13 21:42:29 -07:00
James Babcock 937c73d81b Add battery indicator for Android Wear watch 2026-09-12 19:35:38 -07:00
James Babcock d63058765a Check firmware version step pairs with both lenses (hopefully) 2026-09-12 00:17:08 -07:00
James Babcock b0188d8af9 Fix Android app and background service notification title capitalization; don't show a home screen badge on some devices 2026-09-11 22:38:42 -07:00
James Babcock d41096f336 Music app rechecks for permission if it didn't have notifications access 2026-09-11 22:30:26 -07:00
James Babcock 5c17498a46 Do pairing-auth at "Checking Firmware" stage 2026-09-11 22:05:23 -07:00
James Babcock 049a348627 Merge branch 'glanceboard' 2026-09-11 22:01:12 -07:00
James Babcock 6d85c0ac3e Onboarding "Disconnect Other Apps" page handles case where Even app isn't installed 2026-09-11 21:58:31 -07:00
James Babcock 70d782a4f1 Build out the glanceboard settings app and widgets
<details>
<summary>Prompts Used</summary>
User: Thanks, looks pretty good. I notice that during the transition from the glanceboard back to off, the glasses display a flash of the main UI. Let's address that.

AI:
> Fixed and built. The cause was the hide order in the glance host: it hid the glance surface first, which made the compositor recomposite and transmit the sleeping UI's retained window frames, and only afterwards did the screen-off path blank the display. That one transmitted frame was the flash.
>
> The hide path in `app/g2/glance-host.ts` now blanks the compositor first and hides the surface second. Blanked, the surface change composites to the same black frame, so nothing extra goes to the glasses. The controller's screen-off path still runs afterwards to drop the wakelock and re-arm the 5-second suspend. Its own blank call is now redundant but harmless.
>
> The double-tap path is unchanged on purpose: there the surface hides while the shell is waking, so the retained UI appearing is the intended result.
>
> Typecheck and the full Android build both pass. I recorded the finding in `notes/glanceboard.md` with a new checklist item for the timeout and release transitions. Worth re-checking on hardware: both the 3-second timeout and the long-press release should now go straight to dark.

<hr>

User: Thanks. Next up, let's make a firmware mod so that we can capture tap and tab-and-hold from deep sleep.

AI:
> Done. Firmware revision 2 forwards tap, long-press, and long-press release from deep sleep, and the phone side consumes them. Everything builds and the tests pass, but nothing is flashed or hardware-tested yet.
>
> **How the firmware does it**
>
> Decompiling the display thread showed that with no app on screen, its touch branch launches the dashboard on double-tap or head-up, launches the stock Menu on tap-then-hold, and frees every other gesture unhandled. The call right after that idle gate at `0x45f01a` has the input record in r4 (raw source at +2, gesture subtype at +4). The new hook in `g2flash/patches/settings_ext.c` wraps that call. When Faceclaw holds the wake lease and the firmware is in the mode where a double-tap would have woken the dashboard, it reports subtype 0 (tap), 3 (long press) and 0xe (release) as settings-sid field-102 events 2, 3 and 4, carrying the raw source. The stock result is returned unchanged, so stock behavior is untouched. Unlike the double-tap wake, nothing stock is deferred, so there is no claim or fallback handshake. I verified in the patched image that the site is a branch into the blob, that the shim hands r4 to the C function, and that the function preserves r4.
>
> **Phone side**
>
> - Java parses the new events into ordinary sys-events with the protobuf source while the page is suspended, so TypeScript sees exactly the soft-sleep gestures and the Glanceboard logic needs no change.
> - The firmware string is now `Faceclaw/2`, the app requires revision 2, and the embedded patch set was regenerated. Onboarding will offer the reflash.
> - A Java test covers the event and source mapping, the wake event staying distinct, and malformed payloads.
>
> **Verification**
>
> | Check | Result |
> |---|---|
> | g2flash `build_cfw.sh` | verified, new output hash pinned |
> | Faceclaw `build.sh` | success |
> | `npm test` | new tests pass; only the pre-existing pinball and compass-style failures remain |
>
> **Two caveats**
>
> - I read "tap-and-hold" as press-and-hold, which is the long-press the Glanceboard uses. The firmware's distinct tap-then-hold gesture still opens the stock Menu app from sleep and is not forwarded. Say so if you wanted that one instead.
> - Only the right lens notifies the phone. A double-tap on either temple wakes both lenses, so left-temple taps very likely reach the right lens's display thread while idle, but that is unverified until it runs on hardware. The checklist in `notes/glanceboard.md` has the new deep-sleep steps and the logcat line to look for.
>
> Nothing is committed in either repository.

<hr>

User: Thanks, that worked. Next up, there's also a screen-on trigger from the accelerometer detecting that you've tilted your head up 45deg. Let's make that trigger go to the glanceboard rather than the main app. (If this requires another firmware extension, go ahead and make it; otherwise use what's already there.)

AI:
> Done. It did need a firmware extension, so this is firmware revision 3 plus matching phone changes. Everything builds and tests pass; nothing is flashed or hardware-tested yet.
>
> **Why firmware was needed.** In deep sleep the head-up already reached the phone, but through the same deferred-wake event as a double-tap, so the phone could not tell them apart. In soft sleep, the first five seconds after the screen goes dark, the head-up never reached the phone at all: the display thread's idle gate drops it while an EvenHub page is on screen.
>
> **Firmware, revision 3**
>
> - **Deep sleep.** The idle policy's head-up display-start site now goes through its own wrapper. It runs the identical deferred-wake claim-and-ready handshake but reports field-102 event code 5 instead of 1. One deliberate difference: a head-up while a wake is already pending is ignored, instead of the double-tap's emergency stock-dashboard launch, so lifting your head right after a tap cannot flash the stock dashboard.
> - **Soft sleep.** The idle-gate call is wrapped, ported from the old head-tilt branch with two extra guards mirroring the stock idle path: the glasses' head-up switch must be on and the stock busy check must be clear. It forwards the head-up as EvenHub sys-event 12 under the framebuffer lease.
> - Both hooks and the two wrappers were verified by disassembling the patched image. The output hash is pinned and the embedded patch set in Faceclaw is regenerated.
>
> **Phone side**
>
> - Java decodes wake code 5 as a display-wake carrying the head-up event type, and rewrites the soft-sleep sys-event 12 into the same thing. The Glanceboard logic from earlier already treats that as a press, so a head-tilt now shows the board for the timeout instead of waking the main UI. With the Glanceboard toggle off, a head-tilt still wakes the regular UI as before.
> - The app now requires firmware revision 3, so onboarding will offer the reflash.
> - The Java test covers the new wake code alongside the tap, long-press, and release events.
>
> **One thing left as is.** Faceclaw does not push the head-up switch or angle to the glasses. The trigger uses whatever the Even app configured, which is what you have today. The old branch's setting for that is not merged.
>
> The hardware checklist in `notes/glanceboard.md` has the four head-tilt cases: deep sleep, soft sleep, screen on, and head-tilt right after a double-tap.

<hr>

User: Thanks. Next up, let's make a glanceboard widget for the Calendar.

AI:
> Waiting on the Android build; the widget, its registration, and the notes are done.

<hr>

User: Thanks. Next up, let's make a glanceboard widget for the Terminal app. This will show the same terminals (including, especially, activity indicators) as the terminal list window and reuse some code from it. If the window title is too long to fit, we truncate rather than wrap.

Also, if a terminal has a window open, that window's number will be shown in a left column in the terminals list (both the terminal-list window version and the glanceboard version). This will replace the "[open]" tag currently at the end of the line.

<hr>

User: Continue

AI:
> The Terminal widget is in, the hub list has the new number column, and the build passes. Not yet tried on hardware.
>
> **Getting the list to the main thread.** The Terminal app lives in a worker, so the widget cannot read its state directly. I added a small generic channel: a worker can post a `publish-state` reply with a key and a JSON value, and the host stores it in a main-thread registry with change notifications. The Terminal worker publishes a session snapshot under `terminal:sessions` whenever the hub would repaint, deduplicated by content. Each row carries its label, the open view window's sidebar glyph, and an activity deadline in the phone's clock, so the widget can blink and expire indicators on its own 800 ms timer without further messages.
>
> **Shared row painter.** `app/apps/terminal/session-list.ts` now holds one `drawSessionRow` used by both the hub and the widget: activity square in the gutter, the window number column, then the label truncated to fit, never wrapped. The hub used to draw labels untruncated, so it gains that too.
>
> **The number column.** A session with an open view window now shows that window's sidebar glyph, "1", "2" and so on, in a column left of the label, in both the hub and the widget. The "[open]" suffix is gone from the list. The assistant's list-sessions tool text says "[open in window 2]" instead.
>
> **Widget layout** at 288x144: "Terminal" label, then host headings only when more than one host is connected, then session rows, as many as fit. Empty states cover no connections, connecting, and no live sessions.
>
> **One limitation to know about.** The snapshot exists only once the Terminal app has been opened in this app session, because that is when the worker and its g2mirror control clients start. Before that the widget says to open the Terminal app. Since the shell restores open apps at boot, a hub left open carries across restarts. Making the worker start its control clients at boot without a window is a possible follow-up if you want the widget live from launch.
>
> Checklist entry 7c in the design note covers the blink sync, the number column in both views, and truncation. Nothing is committed.

<hr>

User: Thanks. Next up, let's make it so that selecting a widget for a slot that's already used in another slot clears that slot, _except_ that certain widgets that support it can occupy both left slots or both right slots, in which case they render at double height and the line between the slots is removed. We'll implement double-height for the Terminal and Calendar widgets.

AI:
> Done. Slot uniqueness and double-height regions are in, with a node test for the pure part, and the build passes.
>
> **The rule** lives in two places that share one adjacency helper in the layout module:
>
> - **When a picker changes**, the config app clears every other slot holding the same widget, unless the widget is marked tall and that slot is vertically adjacent to the one just chosen. So picking Terminal for Bottom left while it is in Top left keeps both, and then picking it for Top right clears both left slots. This runs from the picker's change hook and goes through the settings broadcast, so the board and the other rows follow on their own.
> - **When the board paints**, it resolves the slot choices into regions. A tall widget in two vertically adjacent slots becomes one 288x288 region and its widget gets the taller canvas. Dividers are drawn per region, so the line between the merged slots disappears.
>
> **Tall widgets.** Terminal and Calendar are marked tall. Both already lay out rows until they run out of height, so at double height they simply show more sessions or more events with no widget code changes. The other widgets stay single-slot.
>
> **Region diffing.** The board keys live regions by widget plus slots and restarts only what changed on a settings change. Switching a widget from single to tall restarts just that widget; unrelated widgets keep their subscriptions and hardware holds.
>
> **Verification.** `tests/glanceboard-layout.test.cjs` covers adjacency, left and right merges, non-spanning and diagonal duplicates staying separate, and empty boards. Typecheck and the Android build pass. The design note has a checklist step for the three picker cases. Nothing is committed.

<hr>

User: Thanks. Let's organize the Glanceboard configuration app into several screens. The entry page will have a title, a paragraph of text describing how the glanceboard works, and a menu with an "Enable Glanceboard" toggle, Preview, Select Contents, and Settings. Settings will contain:
 * "Show on tap", which instead of a toggle will be a duration (from a list of options 3s, 5s, 7s, 10s, default 5s)
 * "Show on long press", which will be a toggle
 * "Show on head tilt", which will be a toggle
 * "Show lines", which will toggle visibility of the quadrant-separator lines

Fable 5.1:
> The Glanceboard app is now three screens, the four settings are wired through, and the build and tests pass.
>
> **Entry page.** Title, a short paragraph on how the board works, and a menu drawn on the same page below it: Enable Glanceboard (toggle, the existing setting relabeled), Preview, Select Contents, Settings. Double-tap on the entry page yields to the sidebar as before; on the sub-screens it goes back.
>
> **Select Contents** is the slot pickers page from before, including the one-widget-per-board rule and double-height merging.
>
> **Settings** holds:
>
> - **Show on tap**: a duration picker, 3, 5, 7 or 10 seconds, default 5. The head-tilt uses the same duration since it behaves as a tap.
> - **Show on long press**: toggle. Off means a hold while asleep does nothing. A release still passes through so a hold already in progress can end.
> - **Show on head tilt**: toggle. Off sends the head-tilt wake to the regular UI, the pre-Glanceboard behavior.
> - **Show lines**: toggle for the slot dividers.
>
> **Wiring.** The gesture policy moved out of the controller into the glance host, which asks the board provider for the four values. The timeout is handed to the state reducer per event, so the pure module stays testable; a new test covers the supplied timeout. The master toggle off still makes every sleep gesture a no-op.
>
> The design note's checklist has a step per setting. Nothing is committed.

<hr>

User: Thanks. Let's maket the menu half width, and use the space on the right for a preview with the quadrant-separator lines and the title of what's in each slot. Instead of "Select Contents" opening a separate page, it will move the focus to that view, so you can pick one of the quadrants there and tap to open a menu of what goes in that quadrant. The four-quadrants preview will show the lines between them; if the show-lines setting is disabled, they'll be reduced to min brightness and dotted (every other pixel). The preview will omit the line between quadrants as appropriate for double-height views.

Fable 5.1:
> The entry page now has the half-width menu on the left and the slot preview on the right, with Select Contents working inside the preview. Build and tests pass.
>
> **Preview.** The right half shows the board scaled down with its slot dividers and the name of the widget in each region, or "Empty". A double-height region shows one name and no line across it. With "Show lines" off, the dividers drop to the dimmest visible shade and light only every other pixel, so the layout still reads but signals off.
>
> **Select Contents.** Choosing it moves focus into the preview instead of opening a page. Scroll moves a highlight between the four quadrants, a tap opens a modal picker for that slot with the current choice marked, and picking applies the same one-widget-per-board rule as before. The preview updates as soon as a choice lands. Double-tap returns focus to the menu. While the preview has focus the menu's selection shows as outline only, following the shell's convention that a filled selection means it has input.
>
> **Shared geometry.** The divider layout moved out of the board into a pure helper in the layout module, so the live board and the preview draw the same lines, including the merged-region rule. A new test pins the four-slot and tall-Terminal cases.
>
> The design note's checklist has a step for the preview interactions. Nothing is committed.

<hr>

User: Thanks. One fix: while the quadrants-preview is selecte, double-tap back currently goes to the app-switcher, but should go to the glanceboard menu. We can also remove the gesture-help below the preview.

Fable 5.1:
> Fixed and built. The cause was the root wrapper that turns any double-tap at the app's root into a yield to the sidebar, which ran before the page could see the gesture.
>
> The home page is now the window's root without that wrapper and routes double-tap itself: with the preview focused it returns focus to the menu, with the menu focused it yields to the sidebar as before. The gesture hint under the preview is gone.
>
> Typecheck and the Android build pass. Nothing is committed.

<hr>

User: Thanks, looks good.
Let's make the lines in the glanceboard (on the real glanceboard, not the configuration widget) dimmer.
In the Terminal glanceboard widget, with my configured font size (Roboto Light 12), there is room for one more row. It does seem to be responsive (when I increased the font size to 18 the row count dropped accordingly) but the bottom boundary might be above where it could be.
Let's add IOB to the Nightscout glanceboard widget. Also include any of cannula age, cartridge volume, battery voltage, and time since last loop that breach the limits configured in Nightscout settings (if they aren't already included), and double check that the glucose value is crossed out if it's too old.

Fable 5.1:
> All three done, typechecked and built.
>
> **Dimmer lines.** The live board's dividers dropped from shade 40 to 20, the dimmest level that survives the 4-bit quantization. The configuration preview keeps its own brighter lines.
>
> **Terminal rows.** The bottom bound was stricter than it needed to be: it required a full line step, which includes inter-line spacing, to fit above an 8 px bottom pad. Rows are still placed a step apart, but the last one now only needs its glyph line height to clear a 2 px margin, and the gap under the "Terminal" label lost two pixels. With a 12 px font that admits one more row. The Calendar widget got the same rule for consistency.
>
> **Nightscout widget.** IOB now follows the age line. Below that, only fields breaching the limits from Nightscout settings appear, inverted the way the app marks them: cannula age, reservoir volume, pump battery voltage, and loop age, one per line while room remains. Nothing shown means nothing is out of range. The strikethrough was already correct: the value is crossed at mid-height once the reading is older than the app's 15-minute limit, using the app's own helper. I added a comment saying so.
>
> Nothing is committed.
</details>
2026-09-11 16:56:55 -07:00
James Babcock b0cf1e4f67 Switch from capability strings to numeric versioning for firmwares 2026-09-11 00:22:36 -07:00
James Babcock 5a97b218e6 Add bluetooth settings options to bandwidth benchmark 2026-09-10 12:21:01 -07:00
James Babcock 50cbc3a756 Merge branch 'main' into ble-bandwidth-test 2026-09-09 21:59:38 -07:00
James Babcock 58ebe95e0c Merge branch 'main' of https://github.com/jimrandomh/faceclaw 2026-09-09 11:45:58 -07:00
James Babcock adf9fa89c4 Add R1 ring battery indicator 2026-09-09 11:31:44 -07:00
James Babcock 57ace70649 Merge pull request #24 from Leofingal/upstream-pr-wake-latency
Fix EvenHub resume polling for up to 100ms instead of being notified (relates to #9)
2026-09-09 00:53:07 -07:00
Chris Lee 6f304fc076 Wake up notifyAll() waiters when a BLE ack lands (fixes up-to-100ms dead time on every EvenHub resume)
awaitEvenHubSessionReady() polls fixedLayoutCreated/displayedFingerprint at up to 100ms
granularity because resolveAckLocked() never notified waiters when the relevant ack
actually arrived. Confirmed live on real hardware (2026-09-08): full resume (prelude +
create-layout + content frame) now completes in 449ms with a clean, tight ack sequence
and no regressions. Safe by construction: the one caller already holds the lock, and
every lock.wait() consumer in this file rechecks its own condition in a loop.
2026-09-08 22:54:27 -04:00
James Babcock 78c56ffc4e Add source-filtering to Music app, to exclude "media sessions" from apps that aren't really media players 2026-09-08 18:14:25 -07:00
James Babcock 083628f37a Display compass diagnostic information 2026-09-08 11:20:43 -07:00
James Babcock 3d39d27456 Add notification filters 2026-09-08 02:21:50 -07:00
James Babcock 5c3b71aeea Increase compass sample rate; reduce compass bandwidth per frame 2026-09-07 20:52:34 -07:00
James Babcock b43a518f89 Compass updates in 1-degree rather than 2-degree increments 2026-09-07 17:52:35 -07:00
James Babcock 6051be0456 Fix bug where Navigate app could decide you'd arrived based on a stale fix 2026-09-04 14:39:31 -07:00
James Babcock 74199390d2 Implement a bunch of features for alarm reliability 2026-09-04 02:24:02 -07:00
James Babcock f601a62c75 Rewrite the Timer app 2026-09-04 02:23:58 -07:00
James Babcock c37a9eec45 Add a light-sensor demo to the Developer app 2026-09-03 22:23:54 -07:00
James Babcock aef6194687 Add keyboard-input button to the phone home screen 2026-09-03 16:06:46 -07:00
James Babcock 00dd546b1c Navigate app: Location indicator rotates based on the compass; darken the map background color 2026-09-03 12:47:30 -07:00
James Babcock 9234458dd6 Dim the screen underneath when a context menu is open 2026-09-03 01:25:10 -07:00
James Babcock 76825f68a7 Flashing uses 2.2.9 as base version 2026-09-02 00:30:50 -07:00
James Babcock c364099a8c Files app: Gamma correct and dither images when viewing 2026-09-01 21:17:42 -07:00
James Babcock a72e29a04f Compass has higher sample rate, and distinguishes map north from true north 2026-09-01 14:37:09 -07:00
James Babcock 85508b87ca Add "Show BLE bandwidth usage" option to developer settings 2026-09-01 00:33:07 -07:00
James Babcock 6483d00647 Add support for the new short-then-long-press gesture 2026-08-31 20:31:08 -07:00
James Babcock 0114c521ad Merge branch 'main' into ble-bandwidth-test 2026-08-31 01:24:53 -07:00
James Babcock 54005ae95a Preview mode uses phone mic 2026-08-30 20:04:33 -07:00
James Babcock 602f66f15b Make preview-only mode work 2026-08-30 19:17:30 -07:00
James Babcock 3e8be93833 Add a settings import/export receiver
If you install a signed release build and want to switch to an unsigned
debug build, that involves an uninstall and reinstall, which by default
will lose your configuration. Modify scripts/pull_config.sh and
scripts/push_config.sh so it works on release builds, not just debug
builds, and update README.md to tell you to export your config before
doing this kind of upgrade.
2026-08-30 12:25:06 -07:00
James Babcock e1c4628566 Merge branch 'main' of https://github.com/jimrandomh/faceclaw into calculator 2026-08-29 20:13:48 -07:00
James Babcock 756ee4e63b Fix mic drop issues
<details>
<summary>Prompts Used</summary>
Investigate and fix the bug in https://github.com/jimrandomh/faceclaw/issues/18

> Fixed. Here's what was going on and what changed.
>
> ## Root cause
>
> The issue's diagnosis was right, and the underlying flaw is a bit broader than the holder set: **`captureHolders` is phone-side bookkeeping, but the mic enable lives in the glasses' EvenHub session.** That enable dies — silently, from the phone's point of view — on a transport drop, the charging case, an EvenHub screen-off suspend, or a refused enable (`startG2AudioCapture` returns false when the display path isn't warm, and the Java worker then exits while the JS side still believes it started).
>
> `acquireCapture()` answered "is the mic running?" from the holder set, so a single stranded holder (a Transcribe window that was open when the link dropped) made every later request — wakeword assistant included — return early onto a stream that no longer existed, for the life of the process.
>
> The reported repro hits it via `resumeEvenHubSession()`, which resets `audioCaptureActive` and replays the prelude without re-enabling the mic; nothing on either side cleaned up.
>
> ## The fix
>
> **Java** — expose the real state of the stream instead of inferring it:
> - `FaceclawBleCommunicator.isAudioCaptureActive()` (`App_Resources/.../FaceclawBleCommunicator.java:363`) — true only while the session is up, not shut down, and the enable has been acked. Every teardown path already clears that flag.
> - `FaceclawVoiceController.isCapturing()` — `started` plus the communicator's view, with a lock-guarded `audioStarted` so the mic-warmup window doesn't read as dead.
>
> **`app/native/voice-control.ts`** — the bridge now asks `micIsLive()` rather than trusting the set, and tears a dead capture down before starting a new one. Added `handleSessionEnded()` (parks live holders, drops the local state) and `resumeCapture()` (restarts them on the next session); `releaseCapture`/`stop` clear the parked set. The raw-PCM tap for EvenHub mic apps had the identical staleness bug (`if (this.rawActive) return true`) and is covered the same way.
>
> **`app/g2/dashboard-controller.ts`** — wired the two session boundaries: any phase leaving `connected` parks the capture, and the first composited frame of the next session resumes it (a rendered frame is exactly the signal that a mic enable will be accepted). The screen-off EvenHub suspend is now deferred while a capture is held, so the screen timeout no longer kills a live transcription; a capture that races the suspend is parked by the same path.
>
> So Transcribe recovers on its own after a reconnect instead of sitting on a dead "Listening…", and the wakeword assistant is no longer blocked by anything left behind.
>
> ## Verification
>
> `./build.sh` (Java + webpack) clean, `tsc --noEmit` clean, `npm test` 61/61. Not committed.
>
> The behavior is all glasses-session state, which the test harness (pure modules only) can't reach, so it needs hardware. I wrote `notes/voice-capture-session-resume.md` with the five checks that matter — most importantly: Transcribe open → walk out of range → return → "Hey Even" should now get audio, and Transcribe should resume itself.
</details>
2026-08-29 10:16:52 -07:00
James Babcock f3ad55ca3d Merge branch 'calculator' of https://github.com/kalanihelekunihi/faceclaw into calculator 2026-08-26 00:49:20 -07:00
James Babcock b9e5e56ac8 Refine the UI of the main screen bottom area
<details>
<summary>Prompts Used</summary>
Thanks. Next up, we're going to work on the main screen bottom input area.

We will have a distinct state for keyboard-input mode (ie when you select a setting with a text field). While in that state, we'll hide other inputs (ie the simulated watch and ring input buttons) and put the text input (which is currently a modal on top of the screen preview) in the space below the screen preview. If it doesn't fit, given the reduced screen height while the keyboard is open, it will overlap the screen preview. When leaving a text-input screen, we will close the keyboard.

When the keyboard isn't open, we will have a tab bar-style row of buttons underneath the screen preview, with three tabs: Settings, Watch, and Ring. The Settings tab will have brightness and screen size options (we will drop the options in the Phone Display section of the Settings app which would hide these settings). The Ring tab will have a set of buttons corresponding to simulated R1 ring inputs (the main page has recently had this, but currently doesn't seem to). The Watch tab will have a simulated watch face for simulated Android Wear watch inputs (this is currently on the main page).

> Done — builds and typechecks cleanly. Here's the shape of it:
>
> **Keyboard-input mode**
> - When a text setting becomes active, the whole tabbed controls area collapses (`controlsVisibility` on the `rc:remote-controls` instances in both orientations), leaving the space below the preview for the editor.
> - The text editor overlay is now anchored to the **bottom** of the page instead of the top. Combined with the keyboard resizing the page, it sits in the space below the preview and only rides up over the preview when the keyboard leaves too little room. To make that resize behavior deterministic, I added `android:windowSoftInputMode="adjustResize"` to the activity in the manifest (previously unset, so Android picked pan-vs-resize itself; note this applies app-wide, which is generally the better behavior for the other pages with text fields too).
> - When the glasses navigate away from the text-input screen (`activeTextSettingId` goes null), the page now dismisses the soft keyboard, and the tab area comes back.
>
> **Tab bar** (`remote-controls.xml` rewritten)
> - Three tabs — Settings, Watch, Ring — as a segmented row of buttons; the selected tab is remembered across page navigations (module-level, in-memory), defaulting to Watch.
> - **Settings**: screen-size picker (labeled row) and the brightness slider + Auto switch, now always shown.
> - **Watch**: the simulated watch face unchanged (touchpad with d-pad overlay and legend) plus the Back/Menu row.
> - **Ring**: a 3×2 grid of larger labeled buttons for the R1 inputs — ▲ Up, ▼ Down, ● Click, ●● Double, ‒ Hold, and Hey Even (the wakeword, which moved here from the old shared action row since it's a ring-scheme input).
>
> **Settings app**: `showBrightnessControlSetting` and `showScreenSizeSetting` are deleted outright (definitions, menu items, and the view-model visibility getters). The Phone display section keeps Preview color and Touch mirror. Users' old stored values for the deleted keys are simply orphaned, which is harmless.
>
> Things worth checking on hardware: the editor's position with the keyboard open (especially on a short screen where it should overlap the preview), keyboard dismissal when backing out of a text setting on the glasses, and dark-mode contrast of the tab buttons (I inverted the selected/unselected colors for `.ns-dark`).
<hr>
Keyboard-open view: Looks good. One minor (preexisting) issue: The input title (eg "Agent bridge port") appears twice.

Keyboard-closed view: It looks like the tabs are vertically centered when they should be closer to the screen preview; and, the tab contents don't seem to be visible at all.

> Both issues fixed, verified on the tablet, and the fixed build is installed on both devices.
>
> **Tab bar centered / contents invisible** — I reproduced it on the tablet and traced it with a native view-hierarchy dump. The visibility bindings and tab switching were actually working fine; the bug was a NativeScript layout quirk: a `GridLayout` with no `rows` attribute gets a single implicit row that behaves like `*` (star). The old page measured the controls inside an `auto` grid row (unbounded, so the star row collapsed to content), but the restructured page puts them in the outer grid's `*` row (bounded measure) — so the rows-less tab-bar grid ballooned to fill all remaining height, centered its buttons in the stretch, and left zero height for the tab contents below it. The fix is explicit `rows="auto"` on the tab bar, both Settings rows, and the Back/Menu row in `remote-controls.xml`. Verified all three tabs on-device: Watch shows the touchpad + Back/Menu, Settings shows the screen-size picker and brightness row, Ring shows the six input buttons, all sitting directly under the preview.
>
> **Duplicate input title** — for a single-field edit, the editor title and the field's own label are the same string (`editorTitle` defaults to the field label), and the page rendered both. The per-field label is now only shown when a second field exists (where the labels actually distinguish the fields); single-field edits show just the one heading.
>
> One thing I noticed but couldn't test while disconnected: the keyboard-mode change is untouched except for that label, so it should still look as you saw it. There's also a pre-existing console error on page load — `keyboardType="text"` is not a valid NativeScript keyboard type (only email/number/etc. are), so the binding logs an error and falls back to the default keyboard. Harmless, but the clean fix would be to leave `keyboardType` unset for plain text; happy to do that if you want.
<hr>
Thanks, that is now working. Next up let's add a microphone button to the tab bar, as the rightmost button. Use an icon rather than text, and make it narrower than the others. The button will leave the tab selection unchanged, and trigger as if the wakeword ("Hey Even") was spoken. This will replace the "Hey Even" button in the Ring section.

In the Watch section, we will vertically center the emulated-watch UI within the remaining space beneath the tab bar, and set a maximum width corresponding to an aspect ratio for the simulated watch face of 1.5:1.

> Done and verified on the tablet; both devices have the new build.
>
> **Mic button** — added as a fourth, narrower (56dp) column at the right end of the tab bar, showing the 🎤 emoji. It uses the plain tab style, never shows the selected state, and taps route to the existing `onSyntheticMicTap` (ring "wakeword"), leaving the tab selection untouched. The "Hey Even" button is gone from the Ring tab, which now has just the five ring inputs (Up / Down / Click on the top row, Double / Hold below).
>
> **Watch tab centering + aspect cap** — the component root is now a `rows="auto,*"` grid (tab bar on top, the three sections sharing the star cell), and the portrait page gives the controls the full star row (with Stop recording moved to an `auto` row beneath, so it still appears when recording). The watch section — face plus the Back/Menu row — is `verticalAlignment="center"` in the leftover height and width-capped at `min(available width, 1.5 × 230dp face height ≈ 345dp)` via a new `watchFaceWidth` getter that recomputes on orientation change (in landscape the side panel's 328dp effectively governs). Verified on the tablet: the face sits centered between the tab bar and screen bottom at the capped width, and Settings/Ring stay top-aligned.
>
> One aesthetic note: on the Samsung tablet the 🎤 emoji renders as a gray studio mic rather than a colorful one — legible, but if you'd rather have a crisper icon I could switch to a font-icon or a tinted drawable.
<hr>
Thanks. For the microphone button, let's use `~/Downloads/noun-microphone-1368757.svg`.

> Done — the mic button now uses your Noun Project SVG instead of the emoji, verified on the tablet and installed on both devices.
>
> Since NativeScript Buttons are text-only, I converted the SVG's two paths into an Android VectorDrawable (`App_Resources/Android/src/main/res/drawable/ic_tab_mic.xml`, with a comment crediting the source file) and made the mic control a button-styled GridLayout tile holding a tinted `Image`. It keeps the 56dp width and the `.tab-button` look, plus a small `android-elevation` so it casts the same shadow as the real MaterialButtons beside it. The icon tints `#333333` in light mode and `#cccccc` in dark mode via `tint-color`, matching the tab label colors, and the tap still fires the wakeword without touching tab selection.
>
> One licensing note: Noun Project icons are CC BY unless you have a Pro license — if you don't, the icon's creator should be credited somewhere user-visible (e.g. an about/credits page) before a public release.
<hr>
(I do have a Nounproject pro license)
Thanks, the icon renders correctly, but it's off-center (top and left rather than centered on both axes), and the height and drop shadow of the mic button now doesn't match the other buttons in the tab strip.

> Fixed and verified on the tablet; both devices have the build.
>
> Two things were wrong, both variations of the same NativeScript grid quirk from before:
>
> - **Off-center icon**: the tile's row was effectively auto-sized — row heights are locked in at measure time, and inside the tab bar's `auto` row the tile was measured unbounded, so its cell hugged the 20dp icon and the layout-time stretch dumped the leftover space below/right of it. Giving the tile an explicit `height: 48` bounds its measure, so its `rows="*"` cell genuinely fills and the icon's center alignments work — confirmed by native bounds (icon at exactly (72−30)/2 within the tile on both axes).
> - **Height/shadow mismatch**: MaterialButtons render 48dp tall inside a 60dp layout box (6dp built-in top/bottom insets), while the tile was stretching to the full 60dp row. It now gets `margin: 6 2` plus the 48dp height, so its edges and its elevation shadow align with the neighboring buttons.
>
> The 48dp is derived from the tab buttons' current font/padding metrics, so if the tab-button styling ever changes materially, that value would need to follow — I left a comment in `app.css` tying them together.
</details>
2026-08-26 00:44:22 -07:00
James Babcock b99aa85964 Move phone screen connection status and warnings to the top bar
<details>
<summary>Prompts Used</summary>
On the main page, there's a triple-dot menu. Its icon is currently black-on-dark (this is true on both a light-mode device and a dark-mode device; the top bark has a dark background in both cases). Make that icon white instead.

We have a connection-status indicator on the main page, that says "Disconnected" or "Connected" or "Reconnecting". Let's move this onto the top-bar, as white text, right-aligned next to the triple-dot menu.

We have several problem-status warnings, eg for not having battery-optimization permission, that appear on the main page in the form of a large button below the screen preview. Let's convert all of thes into a yellow triangle ! icon that appears on the top bar, in between the connection-status indicator and the triple-dot menu. Tapping it will open a modal that shows the message that was previously shown, with a button to do the suggested action.

> All three changes are in, and the project builds and typechecks cleanly. Here's what I did:
>
> **White overflow icon** — The root cause is that NativeScript builds its toolbar programmatically, so the `theme` overlay in `NativeScriptToolbarStyle` (an XML-inflation-time attribute) never applied, and the overflow icon fell back to the light activity theme's dark `textColorSecondary`. I added an `actionOverflowButtonStyle` with a white `android:tint` to `AppThemeBase` in `App_Resources/Android/src/main/res/values/styles.xml`, so the icon is white on every page's bar (they all share the dark #424242 background), in both light and dark device themes.
>
> **Connection status in the top bar** — The ActionBar now has a custom titleView (`main-page.xml:5`): "Faceclaw" on the left, and a right-aligned white status label next to the menu. Since the old status string could be a whole sentence ("Reconnecting after ...", "Failed: ..."), the bar shows a short label derived from the connection phase — Connected / Connecting / Reconnecting / Disconnected / Charging / Disconnecting / Failed — and tapping it opens an alert with the full status message, so failure details aren't lost. The old status label below the controls is removed.
>
> **Warnings → ⚠️ icon + modal** — The three inline warning blocks (Even-app conflict, firmware, battery optimization) are gone from the page body in both portrait and landscape. A yellow warning-triangle emoji now sits between the status label and the overflow menu whenever any warning is active. Tapping it opens an in-page modal (same overlay pattern as the existing text-setting editor) titled "Attention needed", listing each active warning with its original message and action button, plus a Close button. The modal closes when you take an action, and auto-closes if the last warning clears while it's open. One side effect: the landscape layout previously showed the firmware warning without its install button — in the modal it now has the button like everything else.
>
> Worth an on-device check: the overflow-icon tint (theme change, so a full reinstall rather than hot reload), and that the titleView labels sit vertically centered against the toolbar height.
<hr>
Thanks. That all looks good, except there are some stylign issues on the warning modal. On a wide tablet, the modal is full-width, which is too wide. When the device is dark-mode, the "Attention needed" title is white-on-light-gray. The title and the warning text need some left/right padding between them and the edges of the modal.

> All three styling issues are fixed and the build is clean:
>
> - **Too wide on tablets**: the modal's width now comes from a `warningsModalWidth` getter in the view model — screen width minus the margins on phones, capped at 480 DIPs on tablets — and it's centered horizontally. It re-computes on orientation change along with the other layout metrics.
> - **White title in dark mode**: the theme's `.ns-dark .h2` rule was beating my `.warnings-modal Label` rule on specificity. I added a `.ns-dark .warnings-modal Label` selector so all text in the modal stays dark-on-light in both modes (the modal deliberately keeps its light look, matching the text-setting modal).
> - **Missing edge padding**: NativeScript ScrollViews don't apply their own padding to their content, so the `padding: 16` on the modal was silently doing nothing. It now lives on an inner `warnings-modal-content` StackLayout; the border/background/margin stay on the ScrollView.
>
> One note: the text-setting editor modal (`system-name-modal`) uses the same `h2`-on-light-background pattern, so its title likely has the same white-on-light problem in dark mode — I left it alone since you only flagged the warnings modal, but it'd be the same one-line fix if you want it.
</details>
2026-08-25 23:09:21 -07:00
James Babcock b33d858fbc Merge branch 'main' of https://github.com/jimrandomh/faceclaw into android-wear-integration 2026-08-25 22:29:42 -07:00
James Babcock b74868bfb3 Flasher only uses right lens for confirmation, rather than confusedly trying to check via each lens separately 2026-08-25 21:14:50 -07:00