Commit Graph
81 Commits
Author SHA1 Message Date
James Babcock 71c1971992 Use animated menus in a bunch more places 2026-09-23 23:57:50 -07:00
James Babcock 313ccd86d9 Ungate some settings on iOS 2026-09-20 18:33:24 -07:00
James Babcock ccb320c66f Add tokens and faceclaw-input script, which allows using a CLI utility as an input device for the glasses 2026-09-20 00:34:58 -07:00
James Babcock e2b35ada1a Port lock screen to iOS 2026-09-19 02:55:13 -07:00
James Babcock 15a3e129b2 iOS wakeword support 2026-09-19 01:37:36 -07:00
James Babcock d1df4bf046 Fix missing EvenHub app icons; ungate Assistant section of settings 2026-09-19 01:27:01 -07:00
James Babcock 617cdc39eb Port assistant integration (with cloud AIs) to iOS 2026-09-19 00:02:15 -07:00
James Babcock e8e144f3f2 Merge main into ios-port and update the iOS CFW transport 2026-09-17 23:33:51 -07:00
Chris LeeandClaude Sonnet 5 2d717a9cdb Add on-device Whisper as a second offline transcription option
sherpa-onnx's native library already reads an offline Whisper config
(verified against sherpa-onnx/jni/offline-recognizer.cc at v1.13.0);
only the Java wrapper was a package-private stub. This adds a public
OfflineWhisperModelConfig builder and wires base.en (int8) in as a
second on-device model next to Moonshine.

- FaceclawVoiceController: setOnboardModelKind() selects the model
  before start("onboard"); defaults to Moonshine. Whisper skips the
  700ms live-partial redecode (each call re-encodes the whole utterance
  plus ~10s of tail padding) and gates near-silent segments before
  decode.
- asr-model.ts: ASR_MODELS registry with per-model download state.
- Settings > Voice: "On-device (Moonshine)", "On-device (Whisper)",
  and "OpenAI (Whisper)" for the existing cloud provider (stored
  value unchanged); one model row per on-device model.
- Microphones live captions pass "moonshine" explicitly (unchanged).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 22:46:22 -04:00
James Babcock 937c73d81b Add battery indicator for Android Wear watch 2026-09-12 19:35:38 -07:00
James Babcock ca839f45d6 Fancier battery displays: Stacked view, segmenting 2026-09-09 14:46:47 -07:00
James Babcock a31ddae4d2 Merge branch 'main' into ios-port 2026-09-08 20:08:24 -07:00
James Babcock 79b6c276f8 iOS porting: Image-sending performance: Pipelining, bounding-box updates 2026-09-08 13:18:50 -07:00
James Babcock 1154474754 Add a phone preview Rotation setting; improve landscape layout 2026-09-08 02:43:51 -07:00
James Babcock b07ae0b479 iOS TTF support; voice input 2026-09-07 16:57:49 -07:00
James Babcock e5ba7abc18 iOS porting: Battery indicators, app icon, add the rest of the apps 2026-09-07 10:18:30 -07:00
James Babcock 1524766996 Add screen-size setting to Navigate and Terminal apps 2026-09-05 10:45:39 -07:00
James Babcock 0fac92c7e4 Navigate app: Saved destinations 2026-09-03 22:53:17 -07:00
James Babcock 85508b87ca Add "Show BLE bandwidth usage" option to developer settings 2026-09-01 00:33:07 -07:00
James Babcock e98f19d7e4 Add "Use microphone control" option to developer settings 2026-08-30 21:18:32 -07:00
James Babcock ccd28b7ac9 Move Roam settings out of the Settings app and into the Roam app itself 2026-08-30 15:34:33 -07:00
James Babcock 80d688dd4e Move and rename DashboardInputEvent->InputEvent 2026-08-30 13:02:44 -07:00
James Babcock 63b7a13df1 Gate R1 direct connection behind a dev setting
<details>
<summary>Prompts Used</summary>
Currently we receive user input events from the R1 ring mainly via the G2 glasses, which have their own connection to ring, and we also try to connect to the ring directly; but the direct connection currently winds up not workign reliably. Let's add a "Ring connection mode" to the Developer section of the Settings app, with options "Only via glasses" (new default) and "Direct". If the "Only via glasses" option is selected, we don't open a bluetooth connection to the ring at all.

> Done. Build + typecheck pass.
>
> **Changes:**
>
> - `app/ui/dashboard-settings.ts:393` — new `ringConnectionModeSetting` (`ConfigSettingEnum<RingConnectionMode>`), values `"glasses"` (default, displayed "Only via glasses") and `"direct"`, stored under `developer.ringConnectionMode`.
> - `app/ui/dashboard/settings-menus.ts:182` — added it as the first row of the Developer section.
> - `app/g2/dashboard-controller.ts:988` — `connect()` now computes `ringAddress = mode === "direct" ? addresses.ring : ""` and passes that to `FaceclawCommunicatorBridge` (and into the connect log line).
>
> The gate works by handing the Java communicator an empty ring address: `FaceclawBleCommunicator.hasRingAddress()` then returns false, so `tryConnectRing`/`shouldAttemptRingConnect`/the teardown `disconnect(ringAddress)` all short-circuit — no GATT connection to the ring is ever opened, and the retry loop never fires. Gestures still arrive over the glasses' own link to the ring, unchanged.
>
> Since the mode is read at connect time, changing it while connected takes effect on the next connection; the setting's description says so. If you'd rather it apply live (drop/raise the ring link without cycling the glasses), that needs a small new Java method on the communicator — say the word and I'll add it.
</details>
2026-08-29 11:41:49 -07:00
James Babcock f858f10ce2 Wear watchface design
Makes the button-row appear and disappear with a slide-in/out gesture.
Rearranges the other UI elements for a more watchface-like view.

<details>
<summary>Prompts used</summary>

In faceclaw/wear, we have an Android Wear watch app which can be used as a glasses controller. Currently, its UI is a bit chaotic: we have a bottom row of buttons, the time, a foreground-window title, some help text, and a ring of indicators. The core interaction is tapping, double tapping, long tapping, and swiping (which have the same effect as on the R1 ring), which aren't location sensitive except that they can't hit the row of buttons. There is also a crown, which can be rotated as a prev/next input.

First, let's make a crown-direction setting (in the Watch section of the on-glasses Settings app), which is a toggle that can swap which crown-rotation direction is considered next vs prev. The default will be off, which will be opposite the direction that is currently default.

Next, we will make it so that the row of buttons is hidden by default. They will slide in from the bottom when the user swipes up from off-screen (distinguished from a swipe-up gesture sent to the glasses by the starting position of the swipe). They will slide out with a swipe down starting from anywhere below the top of the buttons (distinguishing from a swipe-down gesture sent to the glasses by the starting position of the swipe).

Next, we will remove the circle and four arrows; change the time from being small, curved and at the top of the screen to being larger, straight and vertically centered. We'll delete the gesture current hint ("● select  ●● back"), and take the current foreground-window display (which is currently either "Disconnected", "Screen off", or an app title) and place it below the time. When the current state is "Screen off", it will display as "●● Wake". We'll then make it so that when the glasses screen is off, all gestures except for double tap (which wakes the screen) are ignored.

gpt-5.6-Sol:

> Implemented the Wear controller redesign.
>
> - Added on-glasses Settings → Watch toggle: “Clockwise crown = next.” Default off maps clockwise to previous, opposite the former default.
> - Replaced the obsolete “Watch input wakes display” setting.
> - Bottom buttons are hidden initially, slide up from a bottom-edge swipe, and dismiss with a downward swipe anywhere in the tray.
> - Removed the ring, arrows, and gesture hint.
> - Added a large centered straight clock with foreground status beneath it; screen-off status is now “●● Wake.”
> - While the glasses display is off, non-double-tap gestures are ignored locally and defensively on the phone.
> - Updated faceclaw/wear/README.md and faceclaw/wear/PROTOCOL.md.
>
> Verification:
>
> - TypeScript type-check passed.
> - All 59 tests passed.
> - git diff --check passed.
> - Wear Gradle compilation could not start because the sandbox blocks Gradle’s local lock socket/cache access.

</details>
2026-08-28 10:28:37 -07:00
James Babcock b9e5e56ac8 Refine the UI of the main screen bottom area
<details>
<summary>Prompts Used</summary>
Thanks. Next up, we're going to work on the main screen bottom input area.

We will have a distinct state for keyboard-input mode (ie when you select a setting with a text field). While in that state, we'll hide other inputs (ie the simulated watch and ring input buttons) and put the text input (which is currently a modal on top of the screen preview) in the space below the screen preview. If it doesn't fit, given the reduced screen height while the keyboard is open, it will overlap the screen preview. When leaving a text-input screen, we will close the keyboard.

When the keyboard isn't open, we will have a tab bar-style row of buttons underneath the screen preview, with three tabs: Settings, Watch, and Ring. The Settings tab will have brightness and screen size options (we will drop the options in the Phone Display section of the Settings app which would hide these settings). The Ring tab will have a set of buttons corresponding to simulated R1 ring inputs (the main page has recently had this, but currently doesn't seem to). The Watch tab will have a simulated watch face for simulated Android Wear watch inputs (this is currently on the main page).

> Done — builds and typechecks cleanly. Here's the shape of it:
>
> **Keyboard-input mode**
> - When a text setting becomes active, the whole tabbed controls area collapses (`controlsVisibility` on the `rc:remote-controls` instances in both orientations), leaving the space below the preview for the editor.
> - The text editor overlay is now anchored to the **bottom** of the page instead of the top. Combined with the keyboard resizing the page, it sits in the space below the preview and only rides up over the preview when the keyboard leaves too little room. To make that resize behavior deterministic, I added `android:windowSoftInputMode="adjustResize"` to the activity in the manifest (previously unset, so Android picked pan-vs-resize itself; note this applies app-wide, which is generally the better behavior for the other pages with text fields too).
> - When the glasses navigate away from the text-input screen (`activeTextSettingId` goes null), the page now dismisses the soft keyboard, and the tab area comes back.
>
> **Tab bar** (`remote-controls.xml` rewritten)
> - Three tabs — Settings, Watch, Ring — as a segmented row of buttons; the selected tab is remembered across page navigations (module-level, in-memory), defaulting to Watch.
> - **Settings**: screen-size picker (labeled row) and the brightness slider + Auto switch, now always shown.
> - **Watch**: the simulated watch face unchanged (touchpad with d-pad overlay and legend) plus the Back/Menu row.
> - **Ring**: a 3×2 grid of larger labeled buttons for the R1 inputs — ▲ Up, ▼ Down, ● Click, ●● Double, ‒ Hold, and Hey Even (the wakeword, which moved here from the old shared action row since it's a ring-scheme input).
>
> **Settings app**: `showBrightnessControlSetting` and `showScreenSizeSetting` are deleted outright (definitions, menu items, and the view-model visibility getters). The Phone display section keeps Preview color and Touch mirror. Users' old stored values for the deleted keys are simply orphaned, which is harmless.
>
> Things worth checking on hardware: the editor's position with the keyboard open (especially on a short screen where it should overlap the preview), keyboard dismissal when backing out of a text setting on the glasses, and dark-mode contrast of the tab buttons (I inverted the selected/unselected colors for `.ns-dark`).
<hr>
Keyboard-open view: Looks good. One minor (preexisting) issue: The input title (eg "Agent bridge port") appears twice.

Keyboard-closed view: It looks like the tabs are vertically centered when they should be closer to the screen preview; and, the tab contents don't seem to be visible at all.

> Both issues fixed, verified on the tablet, and the fixed build is installed on both devices.
>
> **Tab bar centered / contents invisible** — I reproduced it on the tablet and traced it with a native view-hierarchy dump. The visibility bindings and tab switching were actually working fine; the bug was a NativeScript layout quirk: a `GridLayout` with no `rows` attribute gets a single implicit row that behaves like `*` (star). The old page measured the controls inside an `auto` grid row (unbounded, so the star row collapsed to content), but the restructured page puts them in the outer grid's `*` row (bounded measure) — so the rows-less tab-bar grid ballooned to fill all remaining height, centered its buttons in the stretch, and left zero height for the tab contents below it. The fix is explicit `rows="auto"` on the tab bar, both Settings rows, and the Back/Menu row in `remote-controls.xml`. Verified all three tabs on-device: Watch shows the touchpad + Back/Menu, Settings shows the screen-size picker and brightness row, Ring shows the six input buttons, all sitting directly under the preview.
>
> **Duplicate input title** — for a single-field edit, the editor title and the field's own label are the same string (`editorTitle` defaults to the field label), and the page rendered both. The per-field label is now only shown when a second field exists (where the labels actually distinguish the fields); single-field edits show just the one heading.
>
> One thing I noticed but couldn't test while disconnected: the keyboard-mode change is untouched except for that label, so it should still look as you saw it. There's also a pre-existing console error on page load — `keyboardType="text"` is not a valid NativeScript keyboard type (only email/number/etc. are), so the binding logs an error and falls back to the default keyboard. Harmless, but the clean fix would be to leave `keyboardType` unset for plain text; happy to do that if you want.
<hr>
Thanks, that is now working. Next up let's add a microphone button to the tab bar, as the rightmost button. Use an icon rather than text, and make it narrower than the others. The button will leave the tab selection unchanged, and trigger as if the wakeword ("Hey Even") was spoken. This will replace the "Hey Even" button in the Ring section.

In the Watch section, we will vertically center the emulated-watch UI within the remaining space beneath the tab bar, and set a maximum width corresponding to an aspect ratio for the simulated watch face of 1.5:1.

> Done and verified on the tablet; both devices have the new build.
>
> **Mic button** — added as a fourth, narrower (56dp) column at the right end of the tab bar, showing the 🎤 emoji. It uses the plain tab style, never shows the selected state, and taps route to the existing `onSyntheticMicTap` (ring "wakeword"), leaving the tab selection untouched. The "Hey Even" button is gone from the Ring tab, which now has just the five ring inputs (Up / Down / Click on the top row, Double / Hold below).
>
> **Watch tab centering + aspect cap** — the component root is now a `rows="auto,*"` grid (tab bar on top, the three sections sharing the star cell), and the portrait page gives the controls the full star row (with Stop recording moved to an `auto` row beneath, so it still appears when recording). The watch section — face plus the Back/Menu row — is `verticalAlignment="center"` in the leftover height and width-capped at `min(available width, 1.5 × 230dp face height ≈ 345dp)` via a new `watchFaceWidth` getter that recomputes on orientation change (in landscape the side panel's 328dp effectively governs). Verified on the tablet: the face sits centered between the tab bar and screen bottom at the capped width, and Settings/Ring stay top-aligned.
>
> One aesthetic note: on the Samsung tablet the 🎤 emoji renders as a gray studio mic rather than a colorful one — legible, but if you'd rather have a crisper icon I could switch to a font-icon or a tinted drawable.
<hr>
Thanks. For the microphone button, let's use `~/Downloads/noun-microphone-1368757.svg`.

> Done — the mic button now uses your Noun Project SVG instead of the emoji, verified on the tablet and installed on both devices.
>
> Since NativeScript Buttons are text-only, I converted the SVG's two paths into an Android VectorDrawable (`App_Resources/Android/src/main/res/drawable/ic_tab_mic.xml`, with a comment crediting the source file) and made the mic control a button-styled GridLayout tile holding a tinted `Image`. It keeps the 56dp width and the `.tab-button` look, plus a small `android-elevation` so it casts the same shadow as the real MaterialButtons beside it. The icon tints `#333333` in light mode and `#cccccc` in dark mode via `tint-color`, matching the tab label colors, and the tap still fires the wakeword without touching tab selection.
>
> One licensing note: Noun Project icons are CC BY unless you have a Pro license — if you don't, the icon's creator should be credited somewhere user-visible (e.g. an about/credits page) before a public release.
<hr>
(I do have a Nounproject pro license)
Thanks, the icon renders correctly, but it's off-center (top and left rather than centered on both axes), and the height and drop shadow of the mic button now doesn't match the other buttons in the tab strip.

> Fixed and verified on the tablet; both devices have the build.
>
> Two things were wrong, both variations of the same NativeScript grid quirk from before:
>
> - **Off-center icon**: the tile's row was effectively auto-sized — row heights are locked in at measure time, and inside the tab bar's `auto` row the tile was measured unbounded, so its cell hugged the 20dp icon and the layout-time stretch dumped the leftover space below/right of it. Giving the tile an explicit `height: 48` bounds its measure, so its `rows="*"` cell genuinely fills and the icon's center alignments work — confirmed by native bounds (icon at exactly (72−30)/2 within the tile on both axes).
> - **Height/shadow mismatch**: MaterialButtons render 48dp tall inside a 60dp layout box (6dp built-in top/bottom insets), while the tile was stretching to the full 60dp row. It now gets `margin: 6 2` plus the 48dp height, so its edges and its elevation shadow align with the neighboring buttons.
>
> The 48dp is derived from the tab buttons' current font/padding metrics, so if the tab-button styling ever changes materially, that value would need to follow — I left a comment in `app.css` tying them together.
</details>
2026-08-26 00:44:22 -07:00
James Babcock 1f9f3b35d3 UI refinements to watch app and phone preview
<details>
<summary>Prompts Used</summary>
Thanks. Next up, let's add a "Phone display" section to the Settings app (above the Watch section). We'll add a "Preview color" setting (options: White, Green; default White), which controls whether the mirror display is white/grayscale (best looking) or green (matches the physical glasses). We'll move the "Touch mirror" toggle setting (currently on the phone main screen) into the phone display settings menu. We'll add a "Show brightness control" toggle setting (default off) and a "Show screen size" toggle setting (default off), which control the visibility of the corresponding elements on the phone main screen.

Let's also make it so that double-tapping the watch (or simulated watch on the phone, mirror, etc) while the app-switcher is selected turns off the glasses display, in the same way that double-tappign the R1 ring does.

> All four pieces are in and everything builds (webpack, Gradle with the Java changes, and a clean `tsc`). Summary:
>
> **"Phone display" Settings section** — added between Roam and Watch in the on-glasses Settings app, with four entries:
> - **Preview color** (White/Green, default White) — new enum setting `phone.previewColor`. The colorization happens in the Java preview path: `PreviewBitmapUtil.fromGray` gained a `green` variant whose LUT maps luminance to green-on-black instead of grayscale, threaded through `getCompositePreviewBitmap` → `getCompositePreview(green)` → the controller, which reads the setting per refresh. A settings change also schedules an immediate preview refresh, so flipping the color updates the mirror right away instead of waiting for the next composited frame.
> - **Touch mirror** — moved from the phone main screen. It's now a proper `ConfigSettingBoolean` reusing the existing `phone.mirrorTouch` storage key, so anyone who already turned it off keeps that choice. The label+switch pair is gone from the ring row in `remote-controls.xml`, and the view model reads the setting per gesture.
> - **Show brightness control** and **Show screen size** (both default off) — new booleans controlling visibility of the brightness slider/Auto toggle and the display-mode picker on the phone main screen. The whole display-controls row collapses when both are off; the bindings refresh through the existing `onAnySettingChanged` → `refreshDisplayControls` path, so toggling them from the glasses updates the phone screen live. Note this means the main screen is more minimal out of the box — brightness and screen size remain reachable via the Settings app either way.
>
> **Watch double-tap at the app switcher** — the watch-scheme special case in `handleSidebarInput` is removed: a double-click from any source (watch, phone pad/d-pad, mirror double-tap — they all arrive as source `"watch"`) now turns the display off when the sidebar is selected, exactly like the ring's. The original code deliberately excluded the watch on the theory that sleeping was a trap since any watch gesture re-wakes; with your "Watch input wakes display" setting now gated (and double-tap waking a dark display regardless), the toggle behavior is symmetric: double-tap at the switcher sleeps, double-tap again wakes.
>
> Hardware-test notes: the green preview tint and the collapsed-controls layout are worth one visual check on the phone, and the double-tap-to-sleep from the watch is a quick on-device confirm.
</details>
2026-08-25 01:37:29 -07:00
James Babcock 646335cb64 Merge branch 'main' of https://github.com/jimrandomh/faceclaw into android-wear-integration
# Conflicts:
#	app/phone-ui/main-page.xml
#	app/phone-ui/main-view-model.ts
2026-08-24 22:24:03 -07:00
James Babcock 208b14e928 Update ACKNOWLEDGEMENTS 2026-08-24 13:17:09 -07:00
James Babcock 92a2530e0e Merge branch 'main' of https://github.com/jimrandomh/faceclaw into android-wear-integration 2026-08-24 00:23:16 -07:00
Kalani Helekunihi 8fa04d1b46 Add Wear OS companion app and remote controls
Add a Wearable Data Layer bridge for syncing dashboard state and commands.

Introduce a standalone watch app with assistant, app launcher, touchpad remote, settings, and connectivity UI.
2026-08-23 15:02:35 -05:00
James Babcock 2b71eddab5 Optimize APK size
Removes unused keyword-spotting model. Makes on-device voice model a
runtime download rather than bundling it. Removes native-code library
versions for 32-bit and x86_64 processors.
2026-08-22 23:04:01 -07:00
James Babcock 35e6977c6c UI refinement pass
Remove gesture-help bottom-bars from many places in the UI, adjust
layouts to be able to handle larger fonts, and other small UI fixes.
2026-08-22 18:48:20 -07:00
James Babcock eef9fbaab4 Adjust UI to account for user-configured variable font size 2026-08-19 20:48:18 -07:00
James Babcock 96842ceb23 Use TTF fonts throughout the UI; also add some more functions to app extension API 2026-08-19 19:35:24 -07:00
James Babcock deac81f98f EvenHub app uses logo; move login from the Settings app into the EvenHub app 2026-08-19 00:49:18 -07:00
James Babcock eab5a17b6d Centralized version number; bump to 0.4.0 2026-08-16 22:38:58 -07:00
James Babcock df85fb4072 Auto-get OpenUdid rather than it being a config setting 2026-08-16 21:01:20 -07:00
James Babcock cde29cc4d0 Basic EvenHub store compatibility 2026-08-16 20:59:42 -07:00
James Babcock 6df7e9bc81 Terminal connection string management, multi-host 2026-08-10 01:47:26 -07:00
James Babcock 6f45888627 Add support for a local model (Qwen3) 2026-08-09 23:52:21 -07:00
James Babcock 2562761038 Terminal auto-reconnect 2026-08-09 16:08:19 -07:00
James Babcock b5213170e3 Add link to ACKNOWLEDGEMENTS.md to the Settings app, and update it 2026-08-06 11:10:55 -07:00
James Babcock ba81e46624 Add Soniox as an API speech-to-text option 2026-08-04 20:15:39 -07:00
James Babcock 4022f06d49 Build an integration with Roam Research 2026-08-03 15:21:52 -07:00
James Babcock 9b3832d7a7 Refactor: Organize files by app 2026-08-03 00:08:52 -07:00
James Babcock 0f4adb0e86 Rename README to README.md 2026-08-02 19:35:36 -07:00
James Babcock 960e2fd6e1 Add a lock screen, controlled by wear-detection 2026-08-02 15:38:07 -07:00
James BabcockandClaude Fable 5 48bac85cbe External assistant backend: agent bridge client + MCP server
Phase 3 of the voice-assistant design: assistant.backend=external routes
utterances over a dial-out websocket to faceclaw-agent-bridge (an OpenClaw
plugin on the user's machine). The phone serves its ToolRegistry as MCP on
the same socket, with proactive gating and rate limiting enforced locally.
New Assistant settings section with backend/bridge settings.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XCdkQyB3RUVPTuEmhxVEdp
2026-08-01 18:28:47 -07:00
James Babcock 4d1182599b Add privacy policy; update README; incorporate both into Settings app 2026-07-31 22:19:51 -07:00
James Babcock 7d2a41bd56 Apps are variable height; add vertical position setting 2026-07-31 14:52:51 -07:00