forked from Blendman974/faceclaw
<summary>Prompts Used</summary> Act as a code reviewer for https://github.com/jimrandomh/faceclaw/pull/14 (checked out in this working directory). > * The review finished — 42 raw candidates were verified down to 10 confirmed findings. Reporting them now. > > * Code review(review · 10 findings) > ⎿ app/phone-ui/main-view-model.ts > ● 766 [correctness] mirrorFraction subtracts view.getLocationOnScreen() from tap coordinates that NativeScript already delivers > view-local, so every phone-mirror tap is shifted up/left and misroutes > ● 84 [resource-leak] MainViewModel's constructor registers onAnySettingChanged and discards the unsubscriber; a new view model is > built on every page navigation, leaking listeners and retained preview bitmaps > wear/app/src/main/kotlin/com/faceclaw/wear/MainActivity.kt > ● 52 [correctness] onStop abandons an in-progress stem-button hold without sending LONG_PRESS_RELEASE, so the glasses shell > escape menu opens by itself > wear/app/src/main/kotlin/com/faceclaw/wear/ui/TouchpadGestures.kt > ● 62 [correctness] Breaking out of the touchpad gesture loop on a consumed pointer event (or coroutine cancellation mid-hold) > skips onLongPressEnd, leaving a stuck long-press > wear/app/src/main/kotlin/com/faceclaw/wear/PhoneLink.kt > ● 270 [correctness] loadStateItem queries the Data Layer with a hostless Uri (`wear:/faceclaw/state`), which matches no stored > items, so the cached phone state is never read > app/ui/shell/modal-layer.ts > ● 14 [correctness] MODAL_INTERIOR is a module-level constant frozen at import time from the (now runtime-dynamic) display mode, > while modalRect() is computed live, so modals render corrupted after a display-mode switch > app/g2/dashboard-controller.ts > ● 1404 [correctness] handleMirrorTouch wakes the glasses display unconditionally, bypassing the watchWakesDisplaySetting gate > that the parallel injectSyntheticRingInput path honors > ● 1440 [correctness] A mirror tap while any second layer is stacked (e.g. an in-window long-press menu) degrades to a blind > synthetic click that activates whatever is highlighted, not what was tapped > App_Resources/Android/src/main/java/com/faceclaw/app/FaceclawWearBridge.java > ● 147 [correctness] Watch gestures/commands queued while the JS listener is down are replayed up to 30 seconds later on listener > registration, executing stale non-idempotent input the watch already reported as failed > wear/app/src/main/kotlin/com/faceclaw/wear/ui/RemoteScreen.kt > ● 232 [efficiency] WristTwistDetector (gyroscope, default-on) samples at 100 Hz via DisposableEffect and is not stopped when > the activity stops, draining the watch battery while the process is cached > > * Review of PR #14 (Wear OS companion app + remote controls, 85 files, +5813/−102) is done — 10 confirmed findings reported above, ranked by > severity. The headline items are the phone-mirror tap coordinate double-correction (which misroutes every mirror tap in the default > configuration) and two independent paths on the watch that can drop LONG_PRESS_RELEASE, each of which makes the shell escape menu open on the > glasses by itself. > > A few things beyond the reported list, for context: > > Refuted during verification (looked like bugs, aren't): the watch ack-timeout-during-render theory (input injection never blocks on BLE > transmit), mirror swipes stealing page scroll (no enclosing ScrollView), and a suspected raw-settings-key notification gap (the store-level > broadcast covers raw keys). > > Confirmed but below the 10-finding cap — worth folding in if you're touching those areas anyway: > - shell.closeAssistant() no-ops via popIfTop when another layer sits above the assistant, yet still acks ok to the watch. > - Landscape preview sizing still assumes the old 260-wide panel after the XML moved to 360 (overlap under ~568 dip). > - The shell escape menu and tool-debug dialogs hardcode SIDEBAR_WIDTH + 8, wrong in the new full-panel mode. > - A CapabilityClient IPC fires every 250 ms during streamed assistant replies even on watchless phones; backgrounded preview updates bypass > their 150 ms floor. > - Cleanups: the sidebarWidth() > 0 || focus === "sidebar" predicate appears three times, sidebar slot math is duplicated in chrome-layer.ts, > dead protocol-version plumbing, dead padPointers field, and display-mode is missing from wear/PROTOCOL.md. > > If you'd like, I can apply fixes for any subset of these, or post the findings as inline PR comments (/code-review 14 --comment). <hr> Thanks, let's go ahead and write fixes for all of the issues you described. > Everything builds and tests pass. All 16 issues from the review are fixed — the 10 reported findings plus the six below-cap items. Here's what changed, by fix: > > **Watch app (Kotlin)** > - **Stuck long-press via onStop** — `MainActivity.onStop` now sends `LONG_PRESS_RELEASE` when a stem hold is in progress before dropping the link, so the phone's escape-menu countdown can't run to completion on its own. > - **Stuck long-press via the gesture loop** — `detectTouchpadGestures` wraps the loop in try/finally that pairs every `onLongPressStart` with an `onLongPressEnd`, covering both the consumed-event `break` and coroutine cancellation mid-hold. > - **Cached state never loading** — `PhoneLink.loadStateItem` now queries `wear://*/faceclaw/state` (wildcard host) instead of a hostless Uri that matched nothing, so the watch shows the cached dashboard state when the phone is unreachable. > - **Sensors draining battery while stopped** — the gyroscope/accelerometer/off-body detectors are now driven by a `WhileStartedEffect` helper tied to the activity lifecycle: they stop on `ON_STOP` (palmed screen, back to watch face) and restart on `ON_START`, instead of running at 100 Hz for as long as the process is cached. > - **Dead protocol plumbing** — removed the unused `Protocol.PROTOCOL_VERSION` const and the never-read `PhoneState.protocol` field; the wire field stays and PROTOCOL.md now documents it as reserved. > > **Phone Java** > - **Stale input replay** — `FaceclawWearBridge` now queues only idempotent `/faceclaw/state/request` messages for replay after the JS listener registers; gestures and commands that arrived while the dashboard was down are acked as failed and dropped, never executed up to 30 seconds later. > > **Phone TypeScript** > - **Mirror taps misrouting (the top finding)** — `mirrorFraction` no longer subtracts `getLocationOnScreen()`; NativeScript's `getX`/`getY` are view-local DIPs (I verified this in the installed `@nativescript/core` gesture source), so every mirror tap was being shifted up-left by the status bar + action bar. Taps now land where you touch. > - **Mirror touch ignoring the wake setting** — `handleMirrorTouch` now honours `watchWakesDisplaySetting`: with it off, only a double-tap reaches a dark display, matching the rest of the watch scheme. > - **Blind click through open menus** — `LayerStack.hitTest` now consumes a mirror tap when any layer is stacked (menu, dialog) instead of reporting a miss, so a tap on visible app content can no longer activate whatever menu row happened to be highlighted (e.g. "Close window"). > - **Modal corruption after display-mode switch** — `MODAL_INTERIOR` is now a live `modalInterior()` function, and `ShellModalLayer.paint` re-syncs the inner stack's size each paint, so notification modals render correctly after switching Band ↔ Full panel. > - **View-model leak** — `MainViewModel` keeps its controller/settings unsubscribers and exposes `attach()`/`dispose()`; `main-page.ts` disposes on unload and re-attaches on load (re-attach is needed because suspend/resume fires unloaded/loaded without a navigation, and `subscribe` delivers a fresh snapshot so nothing is missed). > - **`closeAssistant` no-op** — new `LayerStack.popThrough` closes the assistant overlay plus anything stacked above it (follow-up voice dialog, alert); the watch ack now says "The assistant was not open." when there was nothing to close. > - **Watchless-phone IPC churn** — assistant stream events and alerts are skipped when no watch is reachable, so a streaming reply no longer triggers a CapabilityClient lookup every 250 ms. > - **Backgrounded preview floor bypass** — the 150 ms floor check (and timestamp) in `updateCompositePreview` now runs before the foreground-activity check, so backgrounded per-frame refreshes actually throttle. > - **Full-panel hardcodes** — the shell escape menu and tool-debug dialogs position at `sidebarWidth() + 8` (0 in full-panel mode) instead of the fixed 64 px strip. > - **Cleanups** — the strip-visible predicate is now one `sidebarStripVisible()` helper in geometry.ts (used in three places); sidebar slot math is a single `slotPosition()` shared by drawing and hit-testing; dead `padPointers` removed; landscape preview sizing updated to the XML's 360 dp panel; `wear/PROTOCOL.md` documents the `display-mode` command and `displayMode` state field. > > Verification: `./build.sh` (webpack + Gradle) succeeds, `npx tsc --noEmit` is clean, the wear `:app:assembleDebug` builds, and all 59 tests pass. The behavioral fixes — especially mirror-tap coordinates, the two long-press-release paths, and the watch state cache — still need a hardware pass to confirm on device. </details>
333 lines
12 KiB
TypeScript
333 lines
12 KiB
TypeScript
import { G2_LENS_HEIGHT, G2_LENS_WIDTH, GrayImage } from "../graphics/image";
|
|
import { type Plane } from "../graphics/plane";
|
|
import { spanCurrent } from "../native/frame-timings";
|
|
import { type ConfigSettingString } from "./dashboard-settings";
|
|
|
|
/**
|
|
* Where an input came from. The ring and the glasses' arms are the stock
|
|
* sources; "watch" is the Wear OS remote, which components may treat with a
|
|
* richer scheme (see isWatchInput) — the ring's semantics are never changed
|
|
* by it.
|
|
*/
|
|
export type InputSource = "ring" | "left-arm" | "right-arm" | "watch";
|
|
|
|
export type DashboardInputEvent =
|
|
| { type: "click"; source: InputSource }
|
|
| { type: "double-click"; source: InputSource }
|
|
/** Ring scroll (or a watch crown turn, then tagged source "watch"). */
|
|
| { type: "scroll-up"; source?: InputSource }
|
|
| { type: "scroll-down"; source?: InputSource }
|
|
| { type: "long-press"; source: InputSource }
|
|
| { type: "long-press-release"; source: InputSource }
|
|
/**
|
|
* Spatial (four-way) input, which only a watch can produce: the ring's
|
|
* scroll is a one-dimensional cursor, these are directions. Components with
|
|
* a spatial meaning for them opt in with Layer.acceptsDirectional /
|
|
* ShellWindow.acceptsDirectional and must then handle all four; for the
|
|
* rest, directionalFallback turns up/down into scroll, right into "select"
|
|
* (click) and left into "back" (double-click), so a swipe always does
|
|
* something sensible.
|
|
*/
|
|
| { type: "swipe-up"; source: "watch" }
|
|
| { type: "swipe-down"; source: "watch" }
|
|
| { type: "swipe-left"; source: "watch" }
|
|
| { type: "swipe-right"; source: "watch" }
|
|
/**
|
|
* The stock display lifecycle woke while no EvenHub page was running. This
|
|
* is wake-only, unlike a normal double-click (which turns an on screen off).
|
|
*/
|
|
| { type: "display-wake" }
|
|
/**
|
|
* The on-glasses "Hey Even" wakeword fired. Delivered on sid 0x07 by the
|
|
* stock firmware regardless of CFW; the CFW additionally suppresses the stock
|
|
* Even AI app so this is ours to handle. Its configured action may be applied
|
|
* while the screen is off, unlike ordinary input events.
|
|
*/
|
|
| { type: "wakeword" }
|
|
| { type: "unknown"; kind: string; eventSource: number; eventType: number };
|
|
|
|
/** True for input from the Wear OS remote (any type that carries a source). */
|
|
export function isWatchInput(event: DashboardInputEvent): boolean {
|
|
return "source" in event && event.source === "watch";
|
|
}
|
|
|
|
export type DirectionalInputEvent = Extract<
|
|
DashboardInputEvent,
|
|
{ type: "swipe-up" | "swipe-down" | "swipe-left" | "swipe-right" }
|
|
>;
|
|
|
|
export function isDirectionalInput(event: DashboardInputEvent): event is DirectionalInputEvent {
|
|
return (
|
|
event.type === "swipe-up" ||
|
|
event.type === "swipe-down" ||
|
|
event.type === "swipe-left" ||
|
|
event.type === "swipe-right"
|
|
);
|
|
}
|
|
|
|
/**
|
|
* The ring-vocabulary equivalent of a directional swipe: up/down scroll,
|
|
* right selects, left backs out.
|
|
*/
|
|
export function directionalFallback(event: DashboardInputEvent): DashboardInputEvent {
|
|
switch (event.type) {
|
|
case "swipe-up":
|
|
return { type: "scroll-up" };
|
|
case "swipe-down":
|
|
return { type: "scroll-down" };
|
|
case "swipe-right":
|
|
return { type: "click", source: "ring" };
|
|
case "swipe-left":
|
|
return { type: "double-click", source: "ring" };
|
|
default:
|
|
return event;
|
|
}
|
|
}
|
|
|
|
export type LayerActions = {
|
|
/** Ask for a dashboard repaint+transmit, e.g. when async data arrives. */
|
|
requestRender: () => void;
|
|
disconnect: () => Promise<void> | void;
|
|
startTextSettingEdit: (setting: ConfigSettingString) => Promise<void> | void;
|
|
/** Open one phone form for multiple related settings (for example email + password). */
|
|
startTextSettingsEdit: (
|
|
settings: readonly ConfigSettingString[],
|
|
title: string,
|
|
onFinish?: () => void,
|
|
) => Promise<void> | void;
|
|
endTextSettingEdit: () => Promise<void> | void;
|
|
/**
|
|
* Start push-to-talk voice capture with the provider chosen in settings.
|
|
* With `endpointing`, the capture also ends itself when the speaker stops
|
|
* (hands-free); otherwise it runs until stopVoiceCapture.
|
|
*/
|
|
startVoiceCapture: (endpointing?: boolean) => Promise<void> | void;
|
|
/** Stop push-to-talk; for a cloud provider this also commits for a final result. */
|
|
stopVoiceCapture: () => Promise<void> | void;
|
|
/** Start continuous capture (Transcribe); shares the mic with push-to-talk. */
|
|
startContinuousVoiceCapture: () => Promise<void> | void;
|
|
stopContinuousVoiceCapture: () => Promise<void> | void;
|
|
/** Play a CFW tone-sequencer payload (see sound-effects.ts). */
|
|
playBuzzerSequence: (payload: Uint8Array) => Promise<void> | void;
|
|
};
|
|
|
|
/** Do-nothing actions, for stacks whose layers never use them (or as a base to spread over). */
|
|
export const noopLayerActions: LayerActions = {
|
|
requestRender: () => {},
|
|
disconnect: () => {},
|
|
startTextSettingEdit: () => {},
|
|
startTextSettingsEdit: () => {},
|
|
endTextSettingEdit: () => {},
|
|
startVoiceCapture: () => {},
|
|
stopVoiceCapture: () => {},
|
|
startContinuousVoiceCapture: () => {},
|
|
stopContinuousVoiceCapture: () => {},
|
|
playBuzzerSequence: () => {},
|
|
};
|
|
|
|
/**
|
|
* Hands a layer its paint canvas: a fresh transparent (0-filled) image the
|
|
* size of the stack. Calling it also declares that the layers below should
|
|
* stay visible beneath this layer — each layer becomes its own plane, so a
|
|
* layer that never calls paintBelow fully replaces what is under it (its
|
|
* plane is submitted alone), while one that does gets composited above the
|
|
* planes below. Repeat calls return the same canvas.
|
|
*
|
|
* Note the plane model's occlusion rule: raster painted onto the canvas
|
|
* covers lower planes entirely (including their text), but glyphs drawn into
|
|
* the *same* image always render above that image's own raster.
|
|
*/
|
|
export type PaintBelow = () => GrayImage;
|
|
|
|
function notifyRemoved(layer: Layer | undefined): void {
|
|
try {
|
|
layer?.onRemoved?.();
|
|
} catch (error) {
|
|
console.warn("layer onRemoved failed", error);
|
|
}
|
|
}
|
|
|
|
export interface LayerContext {
|
|
readonly stack: LayerStack;
|
|
readonly actions: LayerActions;
|
|
}
|
|
|
|
export interface Layer {
|
|
readonly paintOverBase?: boolean;
|
|
paint(ctx: LayerContext, paintBelow: PaintBelow): GrayImage;
|
|
handleInput(event: DashboardInputEvent, ctx: LayerContext): Promise<void> | void;
|
|
/**
|
|
* True when handleInput gives swipe-left / swipe-right a meaning of its own.
|
|
* Otherwise the stack hands the layer directionalFallback(event) instead.
|
|
*/
|
|
readonly acceptsDirectional?: boolean;
|
|
/**
|
|
* A touch at (x, y) in this layer's own canvas coordinates, from the phone's
|
|
* mirror view. Return true if the layer acted on it (selected / opened what
|
|
* is there); false sends a plain select (click) instead.
|
|
*/
|
|
hitTest?(x: number, y: number, ctx: LayerContext): Promise<boolean> | boolean;
|
|
/** Called when the layer leaves the stack by any path (pop or clearToBase). */
|
|
onRemoved?(): void;
|
|
/**
|
|
* Accept a finished text string aimed at this window (voice input, and
|
|
* anything else the shell routes through sendTextToForegroundWindow).
|
|
* Layers that have somewhere to put text implement it; the rest don't, and
|
|
* the shell then has no app destination to offer.
|
|
*/
|
|
receiveTextInput?(text: string, ctx: LayerContext): void;
|
|
}
|
|
|
|
export class LayerStack {
|
|
private readonly layers: Layer[];
|
|
private readonly ctx: LayerContext;
|
|
private baseWidth: number;
|
|
private baseHeight: number;
|
|
private readonly focusedFn: () => boolean;
|
|
|
|
constructor(
|
|
baseLayer: Layer,
|
|
actions: LayerActions,
|
|
baseSize?: { width: number; height: number },
|
|
// Whether this stack is the current input target; drives the strength of
|
|
// selection highlights (a visible-but-unfocused window dims its selection).
|
|
// Defaults to always-focused for shell overlays and standalone stacks.
|
|
isFocused: () => boolean = () => true,
|
|
) {
|
|
this.layers = [baseLayer];
|
|
this.ctx = {
|
|
stack: this,
|
|
actions,
|
|
};
|
|
this.baseWidth = baseSize?.width ?? G2_LENS_WIDTH;
|
|
this.baseHeight = baseSize?.height ?? G2_LENS_HEIGHT;
|
|
this.focusedFn = isFocused;
|
|
}
|
|
|
|
isFocused(): boolean {
|
|
return this.focusedFn();
|
|
}
|
|
|
|
push(layer: Layer): void {
|
|
this.layers.push(layer);
|
|
}
|
|
|
|
pop(): void {
|
|
if (this.layers.length > 1) {
|
|
notifyRemoved(this.layers.pop());
|
|
}
|
|
}
|
|
|
|
/** Whether the top layer matches (without popping it). */
|
|
topMatches(predicate: (layer: Layer) => boolean): boolean {
|
|
return this.layers.length > 1 && predicate(this.layers[this.layers.length - 1]!);
|
|
}
|
|
|
|
/** Pop the top layer only if it matches; returns whether a layer was popped. */
|
|
popIfTop(predicate: (layer: Layer) => boolean): boolean {
|
|
if (this.layers.length > 1 && predicate(this.layers[this.layers.length - 1]!)) {
|
|
notifyRemoved(this.layers.pop());
|
|
return true;
|
|
}
|
|
return false;
|
|
}
|
|
|
|
clearToBase(): void {
|
|
for (const layer of this.layers.splice(1)) {
|
|
notifyRemoved(layer);
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Pop layers down to and including the given one, wherever it sits in the
|
|
* stack (e.g. dismissing the assistant while a follow-up voice dialog is
|
|
* open above it). Returns false if the layer isn't stacked (the base layer
|
|
* can never be popped). Removal callbacks fire top-down.
|
|
*/
|
|
popThrough(layer: Layer): boolean {
|
|
const index = this.layers.indexOf(layer);
|
|
if (index <= 0) return false;
|
|
for (const removed of this.layers.splice(index).reverse()) {
|
|
notifyRemoved(removed);
|
|
}
|
|
return true;
|
|
}
|
|
|
|
isAtBase(): boolean {
|
|
return this.layers.length === 1;
|
|
}
|
|
|
|
getBaseSize(): { width: number; height: number } {
|
|
return { width: this.baseWidth, height: this.baseHeight };
|
|
}
|
|
|
|
/** Resize the base viewport (e.g. an EvenHub window switching to the tall canvas). */
|
|
setBaseSize(size: { width: number; height: number }): void {
|
|
this.baseWidth = size.width;
|
|
this.baseHeight = size.height;
|
|
}
|
|
|
|
setActions(actions: Partial<LayerActions>): void {
|
|
Object.assign(this.ctx.actions, actions);
|
|
}
|
|
|
|
/**
|
|
* Paint the stack as planes, bottom to top: the top layer's plane last,
|
|
* preceded by the planes of the layers it asked to remain visible (see
|
|
* PaintBelow). All planes share the stack's base size at offset (0, 0).
|
|
*/
|
|
paint(): Plane[] {
|
|
return this.paintLayer(this.layers.length - 1);
|
|
}
|
|
|
|
async handleInput(event: DashboardInputEvent): Promise<void> {
|
|
const top = this.layers[this.layers.length - 1]!;
|
|
const delivered = isDirectionalInput(event) && !top.acceptsDirectional ? directionalFallback(event) : event;
|
|
await top.handleInput(delivered, this.ctx);
|
|
}
|
|
|
|
/**
|
|
* Pass a mirror touch to the base layer when nothing is stacked on it.
|
|
* With a layer stacked (a menu, a dialog) the touch is consumed without
|
|
* acting: positions mean nothing to the overlay, and falling back to a
|
|
* blind synthetic click would activate whatever happens to be highlighted
|
|
* rather than what was tapped.
|
|
*/
|
|
async hitTest(x: number, y: number): Promise<boolean> {
|
|
if (this.layers.length !== 1) return true;
|
|
const base = this.layers[0]!;
|
|
if (!base.hitTest) return false;
|
|
return await base.hitTest(x, y, this.ctx);
|
|
}
|
|
|
|
/** Hand text to the top layer; false if it doesn't take text input. */
|
|
receiveTextInput(text: string): boolean {
|
|
const layer = this.layers[this.layers.length - 1]!;
|
|
if (!layer.receiveTextInput) return false;
|
|
layer.receiveTextInput(text, this.ctx);
|
|
return true;
|
|
}
|
|
|
|
|
|
private paintLayer(index: number): Plane[] {
|
|
const layer = this.layers[index]!;
|
|
let canvas: GrayImage | null = null;
|
|
let belowRequested = false;
|
|
const image = spanCurrent(`paint[${index}]:${layer.constructor.name}`, () =>
|
|
layer.paint(this.ctx, () => {
|
|
belowRequested = true;
|
|
if (!canvas) {
|
|
canvas = new GrayImage(this.baseWidth, this.baseHeight, 0);
|
|
}
|
|
return canvas;
|
|
}),
|
|
);
|
|
const ownPlane: Plane = { image, x: 0, y: 0 };
|
|
if (index <= 0 || !belowRequested) {
|
|
return [ownPlane];
|
|
}
|
|
const below = layer.paintOverBase ? this.paintLayer(0) : this.paintLayer(index - 1);
|
|
return [...below, ownPlane];
|
|
}
|
|
}
|