UI refinements to watch app and phone preview

<details>
<summary>Prompts Used</summary>
Thanks. Next up, let's add a "Phone display" section to the Settings app (above the Watch section). We'll add a "Preview color" setting (options: White, Green; default White), which controls whether the mirror display is white/grayscale (best looking) or green (matches the physical glasses). We'll move the "Touch mirror" toggle setting (currently on the phone main screen) into the phone display settings menu. We'll add a "Show brightness control" toggle setting (default off) and a "Show screen size" toggle setting (default off), which control the visibility of the corresponding elements on the phone main screen.

Let's also make it so that double-tapping the watch (or simulated watch on the phone, mirror, etc) while the app-switcher is selected turns off the glasses display, in the same way that double-tappign the R1 ring does.

> All four pieces are in and everything builds (webpack, Gradle with the Java changes, and a clean `tsc`). Summary:
>
> **"Phone display" Settings section** — added between Roam and Watch in the on-glasses Settings app, with four entries:
> - **Preview color** (White/Green, default White) — new enum setting `phone.previewColor`. The colorization happens in the Java preview path: `PreviewBitmapUtil.fromGray` gained a `green` variant whose LUT maps luminance to green-on-black instead of grayscale, threaded through `getCompositePreviewBitmap` → `getCompositePreview(green)` → the controller, which reads the setting per refresh. A settings change also schedules an immediate preview refresh, so flipping the color updates the mirror right away instead of waiting for the next composited frame.
> - **Touch mirror** — moved from the phone main screen. It's now a proper `ConfigSettingBoolean` reusing the existing `phone.mirrorTouch` storage key, so anyone who already turned it off keeps that choice. The label+switch pair is gone from the ring row in `remote-controls.xml`, and the view model reads the setting per gesture.
> - **Show brightness control** and **Show screen size** (both default off) — new booleans controlling visibility of the brightness slider/Auto toggle and the display-mode picker on the phone main screen. The whole display-controls row collapses when both are off; the bindings refresh through the existing `onAnySettingChanged` → `refreshDisplayControls` path, so toggling them from the glasses updates the phone screen live. Note this means the main screen is more minimal out of the box — brightness and screen size remain reachable via the Settings app either way.
>
> **Watch double-tap at the app switcher** — the watch-scheme special case in `handleSidebarInput` is removed: a double-click from any source (watch, phone pad/d-pad, mirror double-tap — they all arrive as source `"watch"`) now turns the display off when the sidebar is selected, exactly like the ring's. The original code deliberately excluded the watch on the theory that sleeping was a trap since any watch gesture re-wakes; with your "Watch input wakes display" setting now gated (and double-tap waking a dark display regardless), the toggle behavior is symmetric: double-tap at the switcher sleeps, double-tap again wakes.
>
> Hardware-test notes: the green preview tint and the collapsed-controls layout are worth one visual check on the phone, and the double-tap-to-sleep from the watch is a quick on-device confirm.
</details>
This commit is contained in:
James Babcock committed 2026-08-25 01:37:29 -07:00
1 parent dc1d4d30a9
commit 1f9f3b35d3
9 files changed
+115 -30

No files matched your search

@@ -569,12 +569,17 @@ public class FaceclawBleCommunicator implements FaceclawBleListener, Runnable {
* foreground), including worker-app frames the TS side never sees.
*/
public android.graphics.Bitmap getCompositePreviewBitmap(double brightenGamma) {
return getCompositePreviewBitmap(brightenGamma, false);
}
/** As above; `green` renders the preview green-on-black (Settings > Phone display > Preview color). */
public android.graphics.Bitmap getCompositePreviewBitmap(double brightenGamma, boolean green) {
SurfaceCompositor.Composite composite = compositor.previewComposite();
if (composite == null) {
return null;
}
return PreviewBitmapUtil.fromGray(
java.nio.ByteBuffer.wrap(composite.gray), composite.width, composite.height, brightenGamma);
java.nio.ByteBuffer.wrap(composite.gray), composite.width, composite.height, brightenGamma, green);
}
/** Save the current composite as a 4-bit grayscale PNG; returns the path or "". */
@@ -15,13 +15,18 @@ public final class PreviewBitmapUtil {
private PreviewBitmapUtil() {}
public static Bitmap fromGray(ByteBuffer gray, int width, int height, double brightenGamma) {
return fromGray(gray, width, height, brightenGamma, false);
}
/** As above; `green` renders green-on-black (matching the physical glasses) instead of grayscale. */
public static Bitmap fromGray(ByteBuffer gray, int width, int height, double brightenGamma, boolean green) {
if (gray == null || width <= 0 || height <= 0 || gray.remaining() < width * height) {
throw new IllegalArgumentException("invalid gray preview buffer");
}
int[] lut = new int[256];
for (int g = 0; g < 256; g++) {
int v = (int) Math.max(0, Math.min(255, Math.round(255 * Math.pow(g / 255.0, brightenGamma))));
lut[g] = 0xff000000 | (v << 16) | (v << 8) | v;
lut[g] = green ? (0xff000000 | (v << 8)) : (0xff000000 | (v << 16) | (v << 8) | v);
}
byte[] bytes = new byte[width * height];
gray.get(bytes);
+5 -2
View File
@@ -58,7 +58,7 @@ import { type InProcessAppOptions, type InProcessWindow } from "../ui/shell/in-p
import { loadPersistedOpenApps, savePersistedOpenApps } from "../ui/shell/open-apps-persistence";
import { appViewportRect, SIDEBAR_WIDTH, sidebarStripVisible, type WindowHeightMode } from "../ui/shell/geometry";
import { type LayerActions } from "../ui/layers";
import { assistantAllowProactiveSetting, assistantBackendSetting, assistantBridgeHostSetting, assistantBridgePortSetting, assistantBridgeTokenSetting, brightnessSetting, brightnessSettingToLevel, displayModeSetting, watchWakesDisplaySetting, elevenLabsApiKeySetting, getStringSettingById, openAiApiKeySetting, nightscoutApiTokenSetting, firmwareDebugFlagsSetting, lockScreenEnabledSetting, nightscoutSiteUrlSetting, onAnySettingChanged, saveVoiceRecordingsSetting, sonioxApiKeySetting, screenTimeoutSetting, screenTimeoutSettingToMs, suspendEvenHubWhenScreenOffSetting, verticalPositionSetting, voiceProviderSetting, wakeWordActionSetting, type ConfigSettingString } from "../ui/dashboard-settings";
import { assistantAllowProactiveSetting, assistantBackendSetting, assistantBridgeHostSetting, assistantBridgePortSetting, assistantBridgeTokenSetting, brightnessSetting, brightnessSettingToLevel, displayModeSetting, watchWakesDisplaySetting, elevenLabsApiKeySetting, getStringSettingById, openAiApiKeySetting, nightscoutApiTokenSetting, firmwareDebugFlagsSetting, lockScreenEnabledSetting, nightscoutSiteUrlSetting, onAnySettingChanged, previewColorSetting, saveVoiceRecordingsSetting, sonioxApiKeySetting, screenTimeoutSetting, screenTimeoutSettingToMs, suspendEvenHubWhenScreenOffSetting, verticalPositionSetting, voiceProviderSetting, wakeWordActionSetting, type ConfigSettingString } from "../ui/dashboard-settings";
import { isIgnoringBatteryOptimizations, requestIgnoreBatteryOptimizations } from "../native/battery-optimization";
import {
getInstalledEvenHubAppById,
@@ -330,6 +330,9 @@ class DashboardController {
this.applyDisplayModeIfChanged();
this.syncAssistantBridgeIfChanged();
this.syncLockScreenSettingIfChanged();
// A Preview color change should show on the mirror at once, not at the
// next composited frame (cheap: floor-limited, dedup'd on the Java side).
this.schedulePreviewUpdate();
});
// Connect to the external agent bridge at boot if configured; the
// connection stays up (with re-dial) so proactive tool calls work
@@ -2195,7 +2198,7 @@ class DashboardController {
this.lastRecordCaptureAtMs = now;
this.communicator.recordScreenFrame();
}
const preview = this.communicator.getCompositePreview();
const preview = this.communicator.getCompositePreview(previewColorSetting.get() === "green");
if (preview) {
this.setDisplayPreview(preview);
}
+7 -3
View File
@@ -456,10 +456,14 @@ export class FaceclawCommunicatorBridge {
});
}
/** Phone-UI preview of the current composited screen, or null if none yet. */
getCompositePreview(): ImageSource | null {
/**
* Phone-UI preview of the current composited screen, or null if none yet.
* `green` tints it green-on-black (the Preview color setting) instead of
* grayscale.
*/
getCompositePreview(green = false): ImageSource | null {
if (!global.isAndroid) return null;
const bitmap = this.communicator.getCompositePreviewBitmap(PREVIEW_BRIGHTEN_GAMMA);
const bitmap = this.communicator.getCompositePreviewBitmap(PREVIEW_BRIGHTEN_GAMMA, green);
return bitmap ? new ImageSource(bitmap) : null;
}
+24 -10
View File
@@ -18,11 +18,13 @@ import {
DISPLAY_MODE_VALUES,
displayModeLabel,
displayModeSetting,
mirrorTouchSetting,
onAnySettingChanged,
showBrightnessControlSetting,
showScreenSizeSetting,
type BrightnessSetting,
type DisplayModeSetting,
} from "../ui/dashboard-settings";
import { getBooleanSetting, setBooleanSetting } from "../native/settings-store";
import { isValidMacAddress, loadDeviceAddresses } from "../g2/device-addresses";
import { isAutoReconnectSuppressed, resumeAutoReconnect } from "../g2/reconnect-policy";
import { formatErrorMessage } from "../util/format-error";
@@ -742,14 +744,10 @@ export class MainViewModel extends Observable {
// ---- touching the mirror itself ----
// Toggled from Settings > Phone display > Touch mirror; read per gesture,
// so no change notification is needed.
get mirrorTouchEnabled(): boolean {
return getBooleanSetting(MIRROR_TOUCH_KEY, true);
}
onMirrorTouchChange(args: { value?: boolean; object?: { checked?: boolean } }): void {
const on = typeof args.value === "boolean" ? args.value : Boolean(args.object?.checked);
setBooleanSetting(MIRROR_TOUCH_KEY, on);
this.notifyPropertyChange("mirrorTouchEnabled", on);
return mirrorTouchSetting.get();
}
private mirrorFraction(args: GestureEventData & { getX?: () => number; getY?: () => number }): { nx: number; ny: number } | null {
@@ -785,6 +783,21 @@ export class MainViewModel extends Observable {
}
// ---- display mode and brightness, beside the mirror ----
//
// Both controls are hidden unless enabled in Settings > Phone display
// (the settings themselves stay reachable through the Settings app).
get screenSizeControlVisibility(): "visible" | "collapse" {
return showScreenSizeSetting.get() ? "visible" : "collapse";
}
get brightnessControlVisibility(): "visible" | "collapse" {
return showBrightnessControlSetting.get() ? "visible" : "collapse";
}
get displayControlsVisibility(): "visible" | "collapse" {
return showScreenSizeSetting.get() || showBrightnessControlSetting.get() ? "visible" : "collapse";
}
get displayModeLabel(): string {
return displayModeLabel(displayModeSetting.get()) + " ▾";
@@ -848,6 +861,9 @@ export class MainViewModel extends Observable {
this.notifyPropertyChange("brightnessSliderEnabled", this.brightnessSliderEnabled);
this.notifyPropertyChange("brightnessPercent", this.brightnessPercent);
this.notifyPropertyChange("displayModeLabel", this.displayModeLabel);
this.notifyPropertyChange("screenSizeControlVisibility", this.screenSizeControlVisibility);
this.notifyPropertyChange("brightnessControlVisibility", this.brightnessControlVisibility);
this.notifyPropertyChange("displayControlsVisibility", this.displayControlsVisibility);
}
private readLayoutOrientation(): LayoutOrientation {
@@ -863,8 +879,6 @@ export class MainViewModel extends Observable {
}
}
const MIRROR_TOUCH_KEY = "phone.mirrorTouch";
/** NativeScript swipe direction -> the watch-scheme directional gesture. */
function swipeKind(direction: SwipeDirection): "swipe-up" | "swipe-down" | "swipe-left" | "swipe-right" {
switch (direction) {
+7 -7
View File
@@ -7,13 +7,14 @@
MainViewModel.
-->
<StackLayout xmlns="http://schemas.nativescript.org/tns.xsd" class="remote-controls">
<GridLayout columns="auto,*,auto" class="display-controls">
<Button col="0" text="{{ displayModeLabel }}" tap="{{ onDisplayModeTap }}" class="mode-button" />
<GridLayout col="1" columns="auto,*" verticalAlignment="center" class="brightness">
<!-- Hidden by default; enabled in Settings > Phone display. -->
<GridLayout columns="auto,*,auto" class="display-controls" visibility="{{ displayControlsVisibility }}">
<Button col="0" text="{{ displayModeLabel }}" tap="{{ onDisplayModeTap }}" class="mode-button" visibility="{{ screenSizeControlVisibility }}" />
<GridLayout col="1" columns="auto,*" verticalAlignment="center" class="brightness" visibility="{{ brightnessControlVisibility }}">
<Label col="0" text="☀" class="brightness-icon" verticalAlignment="center" />
<Slider col="1" width="100%" minValue="0" maxValue="100" value="{{ brightnessPercent }}" valueChange="{{ onBrightnessChange }}" isEnabled="{{ brightnessSliderEnabled }}" />
</GridLayout>
<StackLayout col="2" orientation="horizontal" verticalAlignment="center" class="auto-toggle">
<StackLayout col="2" orientation="horizontal" verticalAlignment="center" class="auto-toggle" visibility="{{ brightnessControlVisibility }}">
<Label text="Auto" verticalAlignment="center" />
<Switch checked="{{ brightnessAuto }}" checkedChange="{{ onBrightnessAutoChange }}" />
</StackLayout>
@@ -34,14 +35,13 @@
<Button col="2" text="Hey Even" tap="{{ onSyntheticMicTap }}" class="action-button" />
</GridLayout>
<GridLayout columns="auto,*,*,*,*,*,auto,auto" class="ring-row">
<!-- The "Touch mirror" toggle lives in Settings > Phone display. -->
<GridLayout columns="auto,*,*,*,*,*" class="ring-row">
<Label col="0" text="Ring" class="ring-label" verticalAlignment="center" />
<Button col="1" text="▲" tap="{{ onSyntheticUpTap }}" class="ring-button" />
<Button col="2" text="▼" tap="{{ onSyntheticDownTap }}" class="ring-button" />
<Button col="3" text="●" tap="{{ onSyntheticRightTap }}" class="ring-button" />
<Button col="4" text="●●" tap="{{ onSyntheticLeftTap }}" class="ring-button" />
<Button col="5" text="‒" tap="{{ onSyntheticLongPressTap }}" class="ring-button" />
<Label col="6" text="Touch mirror" class="ring-label" verticalAlignment="center" />
<Switch col="7" checked="{{ mirrorTouchEnabled }}" checkedChange="{{ onMirrorTouchChange }}" />
</GridLayout>
</StackLayout>
+42
View File
@@ -282,6 +282,48 @@ export const lockScreenEnabledSetting = new ConfigSettingBoolean({
"Lock the glasses after they are taken off while the phone is locked. Unlocking the phone unlocks the glasses.",
});
// Phone display: the phone app's mirror of the glasses screen and the
// controls around it on the main page.
export type PreviewColor = "white" | "green";
export const previewColorSetting = new ConfigSettingEnum<PreviewColor>({
id: "preview-color",
label: "Preview color",
storageKey: "phone.previewColor",
defaultValue: "white",
values: ["white", "green"],
formatValue: (value) => (value === "green" ? "Green" : "White"),
description:
"How the phone's mirror of the glasses display renders: white/grayscale (clearest), or green to match the physical glasses.",
});
export const mirrorTouchSetting = new ConfigSettingBoolean({
id: "mirror-touch",
label: "Touch mirror",
// Key predates this setting object (the toggle used to live on the phone's
// main screen); keeping it preserves the user's choice.
storageKey: "phone.mirrorTouch",
defaultValue: true,
description:
"Let touches on the phone's mirror act on the glasses UI: tap selects what it lands on, double-tap goes back, a hold opens the menu, swipes navigate.",
});
export const showBrightnessControlSetting = new ConfigSettingBoolean({
id: "show-brightness-control",
label: "Show brightness control",
storageKey: "phone.showBrightnessControl",
defaultValue: false,
description: "Show the brightness slider above the mirror on the phone's main screen.",
});
export const showScreenSizeSetting = new ConfigSettingBoolean({
id: "show-screen-size",
label: "Show screen size",
storageKey: "phone.showScreenSize",
defaultValue: false,
description: "Show the screen-size (display mode) picker above the mirror on the phone's main screen.",
});
// Wear OS watch remote (app/g2/wear-remote.ts, wear/). All three are read on
// every watch message, so a change applies immediately.
export const watchRemoteEnabledSetting = new ConfigSettingBoolean({
+15
View File
@@ -38,7 +38,9 @@ import {
displayModeSetting,
elevenLabsApiKeySetting,
mapboxApiKeySetting,
mirrorTouchSetting,
openAiApiKeySetting,
previewColorSetting,
roamApiTokenSetting,
roamGraphNameSetting,
sonioxApiKeySetting,
@@ -46,6 +48,8 @@ import {
firmwareDebugFlagsSetting,
lockScreenEnabledSetting,
saveVoiceRecordingsSetting,
showBrightnessControlSetting,
showScreenSizeSetting,
suspendEvenHubWhenScreenOffSetting,
terminalAutoReconnectSetting,
terminalLaunchPresetsSetting,
@@ -150,6 +154,17 @@ function settingsSections(): SettingsSection[] {
textSettingMenuItem(roamApiTokenSetting),
],
},
{
label: "Phone display",
// The phone app's mirror of the glasses screen and its controls
// (app/phone-ui/): all read live by the main page.
items: [
enumSettingMenuItem(previewColorSetting),
toggleSettingMenuItem(mirrorTouchSetting),
toggleSettingMenuItem(showBrightnessControlSetting),
toggleSettingMenuItem(showScreenSizeSetting),
],
},
{
label: "Watch",
// Wear OS remote (wear/); the status line above the items says whether
+3 -6
View File
@@ -8,7 +8,6 @@ import {
directionalFallback,
type InputSource,
isDirectionalInput,
isWatchInput,
Layer,
LayerActions,
LayerContext,
@@ -686,11 +685,9 @@ class Shell {
private handleSidebarInput(event: DashboardInputEvent): ShellInputOutcome {
switch (event.type) {
case "double-click":
// The ring's double-tap at the root turns the display off. For the
// watch, "back" at the root is simply the end of the road: it wakes
// the display on any gesture, so sleeping here would be a trap; the
// watch's Status screen turns the display off on purpose.
if (isWatchInput(event)) return { shell: false, window: false };
// A double-tap at the root (the app switcher selected) turns the
// display off — from the ring and the watch scheme alike; watch-scheme
// gestures wake it again (or a ring double-tap does).
this.sleep();
return { shell: true, window: false };
case "scroll-up":