Commit Graph
37 Commits
Author SHA1 Message Date
James Babcock 38da8e6168 Fix several bugs that could lead to reconnect-looping
Fix misinterpretation of EvenHub event type 5.
Notification modals stacked, in a way that's O(n) in device resource memory; replace with a queue.
2026-10-05 01:36:40 -07:00
James Babcock 392554dd37 Depth effects 2026-09-22 10:59:51 -07:00
James Babcock 4320021222 Display lists 2026-09-22 00:25:40 -07:00
James Babcock 7e58c7c05f EvenHub compatibility for iOS 2026-09-19 00:48:26 -07:00
James Babcock 25c398d60f Wait for final results from Whisper before completing
<details>
<summary>Prompts Used</summary>
User: This checkout corresponds to https://github.com/jimrandomh/faceclaw/pull/27. Review this branch.

GPT-6 Astra:
> Found one issue:
>
> - **[P2] Wait for Whisper’s final result before completing dictation** — [FaceclawVoiceController.java:729](/Users/jbabcock/repositories/g2/faceclaw/App_Resources/Android/src/main/java/com/faceclaw/app/FaceclawVoiceController.java:729). Skipping partial decodes leaves short utterances without text until release, but the dialog still finalizes after its existing timeout. If decoding exceeds roughly 2.7 seconds (native stop’s 1.5-second wait plus the dialog’s 1.2-second timeout), auto-send can abandon a short command or send an incomplete longer one. Continue can append the late editing instruction literally instead of refining the message. I reproduced these outcomes with delayed transcript events. These flows should await recognition completion.
>
> Validation: TypeScript and a clean Android build passed. All 21 test failures also occur on `main`. Actual Whisper latency was not tested on hardware.

<hr>

User: Thanks, let's fix that issue.

GPT-6 Astra:
> Fixed. Dictation now waits for native recognition to finish before fallback timers run. Auto-send, Continue, and AI Chat preserve delayed Whisper results; cancellation ignores late transcripts.
>
> TypeScript and Android builds pass. All 11 new regression tests pass. The full suite still has the same 21 pre-existing failures.
</details>
2026-09-17 22:12:33 -07:00
James Babcock 9234458dd6 Dim the screen underneath when a context menu is open 2026-09-03 01:25:10 -07:00
James Babcock 80d688dd4e Move and rename DashboardInputEvent->InputEvent 2026-08-30 13:02:44 -07:00
James Babcock d542acb891 Merge branch 'main' of https://github.com/jimrandomh/faceclaw 2026-08-30 00:42:27 -07:00
James Babcock 6c46f2f7cc EvenHub app stores login token instead of password 2026-08-30 00:35:55 -07:00
James Babcock eef7aa2679 <details>
<summary>Prompts Used</summary>
Act as a code reviewer for https://github.com/jimrandomh/faceclaw/pull/14 (checked out in this working directory).

> * The review finished — 42 raw candidates were verified down to 10 confirmed findings. Reporting them now.
>
> * Code review(review · 10 findings)
>   ⎿  app/phone-ui/main-view-model.ts
>        ●  766 [correctness]   mirrorFraction subtracts view.getLocationOnScreen() from tap coordinates that NativeScript already delivers
>                               view-local, so every phone-mirror tap is shifted up/left and misroutes
>        ●   84 [resource-leak] MainViewModel's constructor registers onAnySettingChanged and discards the unsubscriber; a new view model is
>                               built on every page navigation, leaking listeners and retained preview bitmaps
>      wear/app/src/main/kotlin/com/faceclaw/wear/MainActivity.kt
>        ●   52 [correctness]   onStop abandons an in-progress stem-button hold without sending LONG_PRESS_RELEASE, so the glasses shell
>                               escape menu opens by itself
>      wear/app/src/main/kotlin/com/faceclaw/wear/ui/TouchpadGestures.kt
>        ●   62 [correctness]   Breaking out of the touchpad gesture loop on a consumed pointer event (or coroutine cancellation mid-hold)
>                               skips onLongPressEnd, leaving a stuck long-press
>      wear/app/src/main/kotlin/com/faceclaw/wear/PhoneLink.kt
>        ●  270 [correctness]   loadStateItem queries the Data Layer with a hostless Uri (`wear:/faceclaw/state`), which matches no stored
>                               items, so the cached phone state is never read
>      app/ui/shell/modal-layer.ts
>        ●   14 [correctness]   MODAL_INTERIOR is a module-level constant frozen at import time from the (now runtime-dynamic) display mode,
>                               while modalRect() is computed live, so modals render corrupted after a display-mode switch
>      app/g2/dashboard-controller.ts
>        ● 1404 [correctness]   handleMirrorTouch wakes the glasses display unconditionally, bypassing the watchWakesDisplaySetting gate
>                               that the parallel injectSyntheticRingInput path honors
>        ● 1440 [correctness]   A mirror tap while any second layer is stacked (e.g. an in-window long-press menu) degrades to a blind
>                               synthetic click that activates whatever is highlighted, not what was tapped
>      App_Resources/Android/src/main/java/com/faceclaw/app/FaceclawWearBridge.java
>        ●  147 [correctness]   Watch gestures/commands queued while the JS listener is down are replayed up to 30 seconds later on listener
>                               registration, executing stale non-idempotent input the watch already reported as failed
>      wear/app/src/main/kotlin/com/faceclaw/wear/ui/RemoteScreen.kt
>        ●  232 [efficiency]    WristTwistDetector (gyroscope, default-on) samples at 100 Hz via DisposableEffect and is not stopped when
>                               the activity stops, draining the watch battery while the process is cached
>
> * Review of PR #14 (Wear OS companion app + remote controls, 85 files, +5813/−102) is done — 10 confirmed findings reported above, ranked by
>   severity. The headline items are the phone-mirror tap coordinate double-correction (which misroutes every mirror tap in the default
>   configuration) and two independent paths on the watch that can drop LONG_PRESS_RELEASE, each of which makes the shell escape menu open on the
>   glasses by itself.
>
> A few things beyond the reported list, for context:
>
> Refuted during verification (looked like bugs, aren't): the watch ack-timeout-during-render theory (input injection never blocks on BLE
> transmit), mirror swipes stealing page scroll (no enclosing ScrollView), and a suspected raw-settings-key notification gap (the store-level
> broadcast covers raw keys).
>
> Confirmed but below the 10-finding cap — worth folding in if you're touching those areas anyway:
> - shell.closeAssistant() no-ops via popIfTop when another layer sits above the assistant, yet still acks ok to the watch.
> - Landscape preview sizing still assumes the old 260-wide panel after the XML moved to 360 (overlap under ~568 dip).
> - The shell escape menu and tool-debug dialogs hardcode SIDEBAR_WIDTH + 8, wrong in the new full-panel mode.
> - A CapabilityClient IPC fires every 250 ms during streamed assistant replies even on watchless phones; backgrounded preview updates bypass
>   their 150 ms floor.
> - Cleanups: the sidebarWidth() > 0 || focus === "sidebar" predicate appears three times, sidebar slot math is duplicated in chrome-layer.ts,
>   dead protocol-version plumbing, dead padPointers field, and display-mode is missing from wear/PROTOCOL.md.
>
> If you'd like, I can apply fixes for any subset of these, or post the findings as inline PR comments (/code-review 14 --comment).

<hr>

Thanks, let's go ahead and write fixes for all of the issues you described.

> Everything builds and tests pass. All 16 issues from the review are fixed — the 10 reported findings plus the six below-cap items. Here's what changed, by fix:
>
> **Watch app (Kotlin)**
> - **Stuck long-press via onStop** — `MainActivity.onStop` now sends `LONG_PRESS_RELEASE` when a stem hold is in progress before dropping the link, so the phone's escape-menu countdown can't run to completion on its own.
> - **Stuck long-press via the gesture loop** — `detectTouchpadGestures` wraps the loop in try/finally that pairs every `onLongPressStart` with an `onLongPressEnd`, covering both the consumed-event `break` and coroutine cancellation mid-hold.
> - **Cached state never loading** — `PhoneLink.loadStateItem` now queries `wear://*/faceclaw/state` (wildcard host) instead of a hostless Uri that matched nothing, so the watch shows the cached dashboard state when the phone is unreachable.
> - **Sensors draining battery while stopped** — the gyroscope/accelerometer/off-body detectors are now driven by a `WhileStartedEffect` helper tied to the activity lifecycle: they stop on `ON_STOP` (palmed screen, back to watch face) and restart on `ON_START`, instead of running at 100 Hz for as long as the process is cached.
> - **Dead protocol plumbing** — removed the unused `Protocol.PROTOCOL_VERSION` const and the never-read `PhoneState.protocol` field; the wire field stays and PROTOCOL.md now documents it as reserved.
>
> **Phone Java**
> - **Stale input replay** — `FaceclawWearBridge` now queues only idempotent `/faceclaw/state/request` messages for replay after the JS listener registers; gestures and commands that arrived while the dashboard was down are acked as failed and dropped, never executed up to 30 seconds later.
>
> **Phone TypeScript**
> - **Mirror taps misrouting (the top finding)** — `mirrorFraction` no longer subtracts `getLocationOnScreen()`; NativeScript's `getX`/`getY` are view-local DIPs (I verified this in the installed `@nativescript/core` gesture source), so every mirror tap was being shifted up-left by the status bar + action bar. Taps now land where you touch.
> - **Mirror touch ignoring the wake setting** — `handleMirrorTouch` now honours `watchWakesDisplaySetting`: with it off, only a double-tap reaches a dark display, matching the rest of the watch scheme.
> - **Blind click through open menus** — `LayerStack.hitTest` now consumes a mirror tap when any layer is stacked (menu, dialog) instead of reporting a miss, so a tap on visible app content can no longer activate whatever menu row happened to be highlighted (e.g. "Close window").
> - **Modal corruption after display-mode switch** — `MODAL_INTERIOR` is now a live `modalInterior()` function, and `ShellModalLayer.paint` re-syncs the inner stack's size each paint, so notification modals render correctly after switching Band ↔ Full panel.
> - **View-model leak** — `MainViewModel` keeps its controller/settings unsubscribers and exposes `attach()`/`dispose()`; `main-page.ts` disposes on unload and re-attaches on load (re-attach is needed because suspend/resume fires unloaded/loaded without a navigation, and `subscribe` delivers a fresh snapshot so nothing is missed).
> - **`closeAssistant` no-op** — new `LayerStack.popThrough` closes the assistant overlay plus anything stacked above it (follow-up voice dialog, alert); the watch ack now says "The assistant was not open." when there was nothing to close.
> - **Watchless-phone IPC churn** — assistant stream events and alerts are skipped when no watch is reachable, so a streaming reply no longer triggers a CapabilityClient lookup every 250 ms.
> - **Backgrounded preview floor bypass** — the 150 ms floor check (and timestamp) in `updateCompositePreview` now runs before the foreground-activity check, so backgrounded per-frame refreshes actually throttle.
> - **Full-panel hardcodes** — the shell escape menu and tool-debug dialogs position at `sidebarWidth() + 8` (0 in full-panel mode) instead of the fixed 64 px strip.
> - **Cleanups** — the strip-visible predicate is now one `sidebarStripVisible()` helper in geometry.ts (used in three places); sidebar slot math is a single `slotPosition()` shared by drawing and hit-testing; dead `padPointers` removed; landscape preview sizing updated to the XML's 360 dp panel; `wear/PROTOCOL.md` documents the `display-mode` command and `displayMode` state field.
>
> Verification: `./build.sh` (webpack + Gradle) succeeds, `npx tsc --noEmit` is clean, the wear `:app:assembleDebug` builds, and all 59 tests pass. The behavioral fixes — especially mirror-tap coordinates, the two long-press-release paths, and the watch state cache — still need a hardware pass to confirm on device.
</details>
2026-08-25 00:09:47 -07:00
Kalani Helekunihi 8fa04d1b46 Add Wear OS companion app and remote controls
Add a Wearable Data Layer bridge for syncing dashboard state and commands.

Introduce a standalone watch app with assistant, app launcher, touchpad remote, settings, and connectivity UI.
2026-08-23 15:02:35 -05:00
James Babcock 92cd05b6ca Add options to load an app from a URL or a QR code
(The QR code reading does not work yet)
2026-08-21 12:03:24 -07:00
James Babcock 96842ceb23 Use TTF fonts throughout the UI; also add some more functions to app extension API 2026-08-19 19:35:24 -07:00
James Babcock deac81f98f EvenHub app uses logo; move login from the Settings app into the EvenHub app 2026-08-19 00:49:18 -07:00
James Babcock fdf683761e Apps submit mutli-plane images with glyphs, rather than mere images 2026-08-16 21:01:20 -07:00
James Babcock e5649e9782 Add experimental option to end EvenHub session when the screen is off for power saving 2026-07-23 18:40:48 -07:00
James Babcock afc05d6a19 Redesign handling of long-press gesture and context menus 2026-07-22 22:45:14 -07:00
James Babcock 5c73ba2a4d Add Timer, Weather apps; build out UI of music app; add wakeword action setting 2026-07-20 02:04:21 -07:00
James Babcock 161dbd104e Add wakeword support 2026-07-19 20:45:51 -07:00
James Babcock 22c779553c Sidebar uses icons for apps 2026-07-08 10:45:56 -07:00
James Babcock d0c30f9ca9 Port Transcribe to new UI; remove legacy dashboard 2026-07-07 22:11:28 -07:00
James Babcock 56d0f8c351 Sound effects demo, and music controller 2026-07-07 21:21:02 -07:00
James Babcock 68ab599856 Migrate things into new UI design 2026-07-07 00:52:47 -07:00
James Babcock 1133f952e9 App-switchign UI: Sidebar, top bar, launcher 2026-07-06 20:06:29 -07:00
James Babcock 097d40efb2 Work on voice input 2026-07-05 20:59:53 -07:00
James Babcock 7b8fc8f37f Add long-press and long-press-release events 2026-07-05 01:19:20 -07:00
James Babcock fab6f6689e Terminal feature; raw screenshots. 2026-07-04 17:38:53 -07:00
James Babcock d2a942f5a3 UI improvements
Menus can paginate and be taller than quarter-screen. Text config
setting editor selects all on enter, exits on confirm. Notification
icons are displayed on the notifications-list and single-notification
pages.
2026-07-04 15:02:09 -07:00
James Babcock 630b511d32 Add a performance tracing system (FrameTimings) and fix a big that would cause 6-second stalls 2026-07-04 12:20:13 -07:00
James Babcock acda9cdd7e Add sound demo to Faceclaw; use CFW mode 6 (headerless 4bpp) for images 2026-07-03 23:32:46 -07:00
James Babcock 5da3b48c7d Refactor settings 2026-06-01 15:35:35 -07:00
James Babcock 964cd11785 Refactor font handling; add image-compression estimation debug option 2026-06-01 11:36:46 -07:00
James Babcock bcc30545ad Add Transcribe screen 2026-05-01 15:00:53 -07:00
James Babcock aa0b1d6874 Connection reliability and thread safety fixes 2026-04-28 02:22:11 -07:00
James Babcock 0d0b55c2da Many bug fixes 2026-04-27 22:46:53 -07:00
James Babcock 9fbb985db6 Add initial voice-command support
Adds a single wakeword-style voice command, "screen on", and deals with
all the wiring and permissions that entails. Currently this only
supports using the phone's mic (or a bluetooth headset), not the G2 mic.
2026-04-27 16:45:09 -07:00
James Babcock 6e442e7faa Initial publishable release 2026-04-26 23:07:54 -07:00