forked from Blendman974/faceclaw
<summary>Prompts Used</summary> Act as a code reviewer for https://github.com/jimrandomh/faceclaw/pull/14 (checked out in this working directory). > * The review finished — 42 raw candidates were verified down to 10 confirmed findings. Reporting them now. > > * Code review(review · 10 findings) > ⎿ app/phone-ui/main-view-model.ts > ● 766 [correctness] mirrorFraction subtracts view.getLocationOnScreen() from tap coordinates that NativeScript already delivers > view-local, so every phone-mirror tap is shifted up/left and misroutes > ● 84 [resource-leak] MainViewModel's constructor registers onAnySettingChanged and discards the unsubscriber; a new view model is > built on every page navigation, leaking listeners and retained preview bitmaps > wear/app/src/main/kotlin/com/faceclaw/wear/MainActivity.kt > ● 52 [correctness] onStop abandons an in-progress stem-button hold without sending LONG_PRESS_RELEASE, so the glasses shell > escape menu opens by itself > wear/app/src/main/kotlin/com/faceclaw/wear/ui/TouchpadGestures.kt > ● 62 [correctness] Breaking out of the touchpad gesture loop on a consumed pointer event (or coroutine cancellation mid-hold) > skips onLongPressEnd, leaving a stuck long-press > wear/app/src/main/kotlin/com/faceclaw/wear/PhoneLink.kt > ● 270 [correctness] loadStateItem queries the Data Layer with a hostless Uri (`wear:/faceclaw/state`), which matches no stored > items, so the cached phone state is never read > app/ui/shell/modal-layer.ts > ● 14 [correctness] MODAL_INTERIOR is a module-level constant frozen at import time from the (now runtime-dynamic) display mode, > while modalRect() is computed live, so modals render corrupted after a display-mode switch > app/g2/dashboard-controller.ts > ● 1404 [correctness] handleMirrorTouch wakes the glasses display unconditionally, bypassing the watchWakesDisplaySetting gate > that the parallel injectSyntheticRingInput path honors > ● 1440 [correctness] A mirror tap while any second layer is stacked (e.g. an in-window long-press menu) degrades to a blind > synthetic click that activates whatever is highlighted, not what was tapped > App_Resources/Android/src/main/java/com/faceclaw/app/FaceclawWearBridge.java > ● 147 [correctness] Watch gestures/commands queued while the JS listener is down are replayed up to 30 seconds later on listener > registration, executing stale non-idempotent input the watch already reported as failed > wear/app/src/main/kotlin/com/faceclaw/wear/ui/RemoteScreen.kt > ● 232 [efficiency] WristTwistDetector (gyroscope, default-on) samples at 100 Hz via DisposableEffect and is not stopped when > the activity stops, draining the watch battery while the process is cached > > * Review of PR #14 (Wear OS companion app + remote controls, 85 files, +5813/−102) is done — 10 confirmed findings reported above, ranked by > severity. The headline items are the phone-mirror tap coordinate double-correction (which misroutes every mirror tap in the default > configuration) and two independent paths on the watch that can drop LONG_PRESS_RELEASE, each of which makes the shell escape menu open on the > glasses by itself. > > A few things beyond the reported list, for context: > > Refuted during verification (looked like bugs, aren't): the watch ack-timeout-during-render theory (input injection never blocks on BLE > transmit), mirror swipes stealing page scroll (no enclosing ScrollView), and a suspected raw-settings-key notification gap (the store-level > broadcast covers raw keys). > > Confirmed but below the 10-finding cap — worth folding in if you're touching those areas anyway: > - shell.closeAssistant() no-ops via popIfTop when another layer sits above the assistant, yet still acks ok to the watch. > - Landscape preview sizing still assumes the old 260-wide panel after the XML moved to 360 (overlap under ~568 dip). > - The shell escape menu and tool-debug dialogs hardcode SIDEBAR_WIDTH + 8, wrong in the new full-panel mode. > - A CapabilityClient IPC fires every 250 ms during streamed assistant replies even on watchless phones; backgrounded preview updates bypass > their 150 ms floor. > - Cleanups: the sidebarWidth() > 0 || focus === "sidebar" predicate appears three times, sidebar slot math is duplicated in chrome-layer.ts, > dead protocol-version plumbing, dead padPointers field, and display-mode is missing from wear/PROTOCOL.md. > > If you'd like, I can apply fixes for any subset of these, or post the findings as inline PR comments (/code-review 14 --comment). <hr> Thanks, let's go ahead and write fixes for all of the issues you described. > Everything builds and tests pass. All 16 issues from the review are fixed — the 10 reported findings plus the six below-cap items. Here's what changed, by fix: > > **Watch app (Kotlin)** > - **Stuck long-press via onStop** — `MainActivity.onStop` now sends `LONG_PRESS_RELEASE` when a stem hold is in progress before dropping the link, so the phone's escape-menu countdown can't run to completion on its own. > - **Stuck long-press via the gesture loop** — `detectTouchpadGestures` wraps the loop in try/finally that pairs every `onLongPressStart` with an `onLongPressEnd`, covering both the consumed-event `break` and coroutine cancellation mid-hold. > - **Cached state never loading** — `PhoneLink.loadStateItem` now queries `wear://*/faceclaw/state` (wildcard host) instead of a hostless Uri that matched nothing, so the watch shows the cached dashboard state when the phone is unreachable. > - **Sensors draining battery while stopped** — the gyroscope/accelerometer/off-body detectors are now driven by a `WhileStartedEffect` helper tied to the activity lifecycle: they stop on `ON_STOP` (palmed screen, back to watch face) and restart on `ON_START`, instead of running at 100 Hz for as long as the process is cached. > - **Dead protocol plumbing** — removed the unused `Protocol.PROTOCOL_VERSION` const and the never-read `PhoneState.protocol` field; the wire field stays and PROTOCOL.md now documents it as reserved. > > **Phone Java** > - **Stale input replay** — `FaceclawWearBridge` now queues only idempotent `/faceclaw/state/request` messages for replay after the JS listener registers; gestures and commands that arrived while the dashboard was down are acked as failed and dropped, never executed up to 30 seconds later. > > **Phone TypeScript** > - **Mirror taps misrouting (the top finding)** — `mirrorFraction` no longer subtracts `getLocationOnScreen()`; NativeScript's `getX`/`getY` are view-local DIPs (I verified this in the installed `@nativescript/core` gesture source), so every mirror tap was being shifted up-left by the status bar + action bar. Taps now land where you touch. > - **Mirror touch ignoring the wake setting** — `handleMirrorTouch` now honours `watchWakesDisplaySetting`: with it off, only a double-tap reaches a dark display, matching the rest of the watch scheme. > - **Blind click through open menus** — `LayerStack.hitTest` now consumes a mirror tap when any layer is stacked (menu, dialog) instead of reporting a miss, so a tap on visible app content can no longer activate whatever menu row happened to be highlighted (e.g. "Close window"). > - **Modal corruption after display-mode switch** — `MODAL_INTERIOR` is now a live `modalInterior()` function, and `ShellModalLayer.paint` re-syncs the inner stack's size each paint, so notification modals render correctly after switching Band ↔ Full panel. > - **View-model leak** — `MainViewModel` keeps its controller/settings unsubscribers and exposes `attach()`/`dispose()`; `main-page.ts` disposes on unload and re-attaches on load (re-attach is needed because suspend/resume fires unloaded/loaded without a navigation, and `subscribe` delivers a fresh snapshot so nothing is missed). > - **`closeAssistant` no-op** — new `LayerStack.popThrough` closes the assistant overlay plus anything stacked above it (follow-up voice dialog, alert); the watch ack now says "The assistant was not open." when there was nothing to close. > - **Watchless-phone IPC churn** — assistant stream events and alerts are skipped when no watch is reachable, so a streaming reply no longer triggers a CapabilityClient lookup every 250 ms. > - **Backgrounded preview floor bypass** — the 150 ms floor check (and timestamp) in `updateCompositePreview` now runs before the foreground-activity check, so backgrounded per-frame refreshes actually throttle. > - **Full-panel hardcodes** — the shell escape menu and tool-debug dialogs position at `sidebarWidth() + 8` (0 in full-panel mode) instead of the fixed 64 px strip. > - **Cleanups** — the strip-visible predicate is now one `sidebarStripVisible()` helper in geometry.ts (used in three places); sidebar slot math is a single `slotPosition()` shared by drawing and hit-testing; dead `padPointers` removed; landscape preview sizing updated to the XML's 360 dp panel; `wear/PROTOCOL.md` documents the `display-mode` command and `displayMode` state field. > > Verification: `./build.sh` (webpack + Gradle) succeeds, `npx tsc --noEmit` is clean, the wear `:app:assembleDebug` builds, and all 59 tests pass. The behavioral fixes — especially mirror-tap coordinates, the two long-press-release paths, and the watch state cache — still need a hardware pass to confirm on device. </details>
68 lines
4.3 KiB
Markdown
68 lines
4.3 KiB
Markdown
# Watch ⇄ phone protocol
|
||
|
||
Transport: the Wearable Data Layer (Google Play services). Both apps share the
|
||
`com.faceclaw.app` application id and, for release builds, the signing key —
|
||
the Data Layer routes only between matching apps. Payloads are UTF-8 JSON
|
||
objects. Phone side: `App_Resources/.../FaceclawWearBridge.java` (transport),
|
||
`app/native/wear-bridge.ts` (JS wrapper), `app/g2/wear-remote.ts` (meaning).
|
||
Watch side: `app/src/main/kotlin/com/faceclaw/wear/Protocol.kt`, `PhoneLink.kt`.
|
||
|
||
Capabilities (`res/values/wear.xml` on each side): the phone advertises
|
||
`faceclaw_phone`, the watch `faceclaw_watch`. Each side finds the other with
|
||
`CapabilityClient.getCapability(…, FILTER_REACHABLE)`.
|
||
|
||
Every watch → phone message carries `seq` (monotonic per watch process) and is
|
||
answered on `/faceclaw/ack`.
|
||
|
||
## Watch → phone (MessageClient)
|
||
|
||
| Path | Payload | Effect |
|
||
| --- | --- | --- |
|
||
| `/faceclaw/input` | `{gesture, steps?}` | A ring gesture through the same path as the phone UI's test buttons. `gesture` ∈ `click`, `double-click`, `scroll-up`, `scroll-down`, `long-press` (complete short hold), `long-press-start`, `long-press-release`, `wakeword` ("Hey Even"), `swipe-up`, `swipe-down`, `swipe-left`, `swipe-right` (spatial input only a watch can produce: delivered to the glasses UI as `swipe-*` events for components that navigate spatially — launcher grid, settings columns, sidebar, Music — with a scroll / click / double-click fallback elsewhere). `steps` (1–12) repeats a scroll. Refused while the glasses are disconnected. The lock screen is honoured exactly as for the ring (only a double-click reaches it, toggling the locked display). |
|
||
| `/faceclaw/command` | `{command, …}` | `launch-app {appId}`, `focus-window {windowId}`, `close-window {windowId}`, `sidebar`, `wake`, `sleep`, `lock`, `unlock`, `connect`, `disconnect`, `close-assistant`, `display-mode {value}` (`value` ∈ `576x288`, `576x480`, `640x480`; sets the phone's Display > Display mode). Everything but connect/disconnect needs a connection; everything but lock/unlock is refused while locked; `unlock` also needs the phone's "Watch can unlock glasses" setting. |
|
||
| `/faceclaw/assistant` | `{text}` | Send a query to the assistant (`shell.sendToAssistant`) — the reply shows on the glasses and streams back to the watch as events. |
|
||
| `/faceclaw/text` | `{text}` | Type text into the foreground window (`receiveTextInput`, e.g. the terminal). Refused when the window doesn't take text. |
|
||
| `/faceclaw/state/request` | `{}` | Re-publish the state item even if unchanged. |
|
||
|
||
All of the above except `state/request` are refused with `ok:false` when the
|
||
phone's "Watch remote control" setting is off.
|
||
|
||
## Phone → watch
|
||
|
||
`/faceclaw/ack` (MessageClient, to the sender):
|
||
`{seq, ok, jsReady, message}`. `jsReady:false` means Play services started the
|
||
app process for the message but the JS dashboard isn't up (the user has not
|
||
opened Faceclaw since boot); the watch shows "Open Faceclaw on the phone".
|
||
|
||
`/faceclaw/event` (MessageClient, to every reachable watch):
|
||
- `{type:"assistant", phase, text}` — `phase` ∈ `thinking`, `streaming` (text so
|
||
far, replace semantics, ≤ 4/s), `done` (final text), `error` (message),
|
||
`closed` (overlay dismissed).
|
||
- `{type:"alert", text}` — a `glasses.show_alert` popup.
|
||
Both are gated by the phone's "Mirror assistant to watch" setting.
|
||
|
||
`/faceclaw/state` (DataClient item, keys `json` and `updatedAt`): the mirrored
|
||
dashboard state, re-sent whenever it changes (and at least every 30 s):
|
||
|
||
```json
|
||
{
|
||
"protocol": 1, "version": "…",
|
||
"phase": "connected", "status": "…", "connected": true,
|
||
"screenOn": true, "locked": false, "worn": true, "listening": false,
|
||
"battery": 78, "charging": false,
|
||
"foreground": {"appId": "music", "title": "Music"},
|
||
"windows": [{"windowId": "…", "title": "…", "appId": "…", "focused": true, "closeable": true, "acceptsText": false}],
|
||
"apps": [{"appId": "timer", "title": "Timer"}],
|
||
"displayMode": "576x288",
|
||
"remoteEnabled": true, "canUnlock": true, "mirrorAssistant": true,
|
||
"assistantAvailable": true
|
||
}
|
||
```
|
||
|
||
`protocol` is reserved for incompatible revisions of this format (nothing
|
||
checks it yet); `displayMode` mirrors the phone's Display > Display mode and
|
||
is what the `display-mode` command changes.
|
||
|
||
`apps` is the launcher grid plus installed EvenHub packages, i.e. exactly what
|
||
the assistant's `apps.launch` tool accepts.
|