Commit Graph
178 Commits
Author SHA1 Message Date
James Babcock c007186e05 Fix crash on install of larger EvenHub apps 2026-09-27 18:14:50 -07:00
James Babcock 80f16c0cd6 Merge branch 'main' of https://github.com/jimrandomh/faceclaw into microphone-fixes 2026-09-27 17:57:23 -07:00
James Babcock 3601a998a3 CPU usage optimizations 2026-09-27 16:33:43 -07:00
James Babcock 7d0becfaa7 Fix some microphone-related bugs, add a debug panel 2026-09-26 14:12:29 -07:00
James BabcockandClaude Opus 5.5 1812818ae2 Stop leaking the remaining JS ArrayBuffers passed to Java
The same NativeScript conversion leak as the frame path (see
java-direct-buffer.ts) applied to every other ArrayBuffer handed to a
Java ByteBuffer parameter:

- Shell scenes (submitShellScene on the bridge and the preview display),
  submitted on every shell render, which the frame-path fix predates.
  These now go through a reused JavaDirectBuffer like frames do.
- Glyph, firmware-glyph and image atlas registration, Mapbox image
  decode, the preview placeholder bitmap, firmware hash/write, binary
  file writes and the EvenHub privacy-policy download, plus the Kotlin
  protocol smoke test. These are rare or large, so they cross through a
  new copyToJavaByteBuffer: a fresh Java direct buffer per call that is
  freed normally, rather than one pinned for the life of the isolate.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-25 22:28:45 -07:00
Chris LeeandClaude Opus 5 54eae0c6b9 Stop leaking every submitted frame's pixel buffer in native memory
Passing a JS ArrayBuffer to a Java ByteBuffer parameter leaks it in
@nativescript/android 9.0.4 through 9.1.1: JsArgConverter wraps the backing store with
NewDirectByteBuffer and takes a JNI global ref that it never deletes. The
ref keeps the ByteBuffer reachable, so makeInstanceWeakAndCheckIfAlive
always answers "alive", the JS finalizer re-arms, and V8 never frees the
ArrayBuffer. submitSurfaceFrame passed a fresh copy of each frame this way,
so every frame leaked a full surface (576x260, 640x452 or 640x480 bytes)
plus one global ref, including frames that Java then discards as unchanged.
The draw-list buffer and buzzer payloads leaked the same way.

Each payload now goes through a reused Java direct ByteBuffer
(app/native/java-direct-buffer.ts): JS writes into it through an
ArrayBuffer.from view, and the Java object itself is passed, which takes
NativeScript's ordinary object path. Every consumer (SurfaceCompositor
applyAndComposite, playBuzzerSequence) copies before returning, so one
buffer per JS context is safe.

Covered: the communicator bridge (frames, draw lists, buzzer, which also
carries EvenHub tones), the preview-only display, and all eight worker
apps. Workers submit through the Android getActiveDisplay() (frames) and
worker-buzzer (tones), so the reused buffers live there rather than in
each worker; the worker sources are shared with iOS, whose active display
takes the ArrayBuffer itself. The
smaller one-off crossings (glyph and image atlas registration, the gray
preview helper, firmware and file writes) are left as they are.

tests/java-direct-buffer.test.cjs covers the helper against a fake
ByteBuffer, and the Android worker-buzzer test stubs it.

(Rebased onto main by James Babcock: worker changes moved into
active-display.ts and worker-buzzer.ts.)

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-25 22:24:22 -07:00
James Babcock 4f3ad37c20 Custom auto-brightness, and fade in/out effects 2026-09-25 00:57:43 -07:00
James Babcock 915f155819 Add Glanceboard depth setting 2026-09-24 02:26:20 -07:00
James Babcock 4a194e37bb Refactor iOS version to use a bunch more shared code 2026-09-22 21:26:51 -07:00
James Babcock f9724ddcab Convert remaining Java to Kotlin 2026-09-22 17:26:09 -07:00
James Babcock ea9bfae6af Refactor: g2flash protocol doesn't have so many magic numbers 2026-09-22 16:27:04 -07:00
James Babcock 2e4d20e130 Clean up unused worker threads for closed apps; other performance optimizations 2026-09-22 14:56:54 -07:00
James Babcock 9560f3ac8d Phone CPU usage optimization 2026-09-22 14:19:23 -07:00
James Babcock 392554dd37 Depth effects 2026-09-22 10:59:51 -07:00
James Babcock 4320021222 Display lists 2026-09-22 00:25:40 -07:00
James Babcock 0c3d7517bc Resource management for the texture cache 2026-09-20 18:23:33 -07:00
James Babcock 07828db0bf Refactor texture cache usage to shared Kotlin for iOS 2026-09-20 15:12:13 -07:00
James Babcock ccb320c66f Add tokens and faceclaw-input script, which allows using a CLI utility as an input device for the glasses 2026-09-20 00:34:58 -07:00
James Babcock 6c3144cc31 Make iOS app main screen match the Android UI 2026-09-19 21:54:18 -07:00
James Babcock 32063e57cd iOS calendar integration 2026-09-19 20:52:36 -07:00
James Babcock 7b19c00443 Forward ring input events before filtering, and include swipe info 2026-09-19 19:02:11 -07:00
James Babcock e50da605e0 Add support for buzzer sound effects on iOS 2026-09-19 12:03:57 -07:00
James Babcock cef879fec0 Display proper app name in iOS notifications 2026-09-19 11:46:46 -07:00
James Babcock 67823f66ef Implement notifications on iOS 2026-09-19 11:07:37 -07:00
James Babcock 64bf600ef9 Porting onboarding/flashing flow io iOS 2026-09-19 09:19:26 -07:00
James Babcock 68b459751e iOS QR scanning for opening EvenHub apps 2026-09-19 03:06:56 -07:00
James Babcock e2b35ada1a Port lock screen to iOS 2026-09-19 02:55:13 -07:00
James Babcock 15a3e129b2 iOS wakeword support 2026-09-19 01:37:36 -07:00
James Babcock d1df4bf046 Fix missing EvenHub app icons; ungate Assistant section of settings 2026-09-19 01:27:01 -07:00
James Babcock 7e58c7c05f EvenHub compatibility for iOS 2026-09-19 00:48:26 -07:00
James Babcock 617cdc39eb Port assistant integration (with cloud AIs) to iOS 2026-09-19 00:02:15 -07:00
James Babcock 9e18745e53 Port navigate and weather 2026-09-18 23:46:27 -07:00
James Babcock 2b3e068a23 Port Compass to iOS 2026-09-18 23:30:07 -07:00
James Babcock 5ae2c245c6 Nightscout works on iOS 2026-09-18 23:16:22 -07:00
James Babcock 68810197a8 Refactor Java to shared kotlin 2026-09-18 22:48:20 -07:00
James Babcock bc8e80c533 Kotlin on iOS 2026-09-18 19:59:13 -07:00
James Babcock 63b2a7d545 Merge branch 'main' into ios-port
# Conflicts:
#	app/app.ts
2026-09-18 19:38:10 -07:00
James Babcock 1e1c3f799d Introduce Kotlin 2026-09-18 19:35:50 -07:00
James Babcock e8e144f3f2 Merge main into ios-port and update the iOS CFW transport 2026-09-17 23:33:51 -07:00
James Babcock 25c398d60f Wait for final results from Whisper before completing
<details>
<summary>Prompts Used</summary>
User: This checkout corresponds to https://github.com/jimrandomh/faceclaw/pull/27. Review this branch.

GPT-6 Astra:
> Found one issue:
>
> - **[P2] Wait for Whisper’s final result before completing dictation** — [FaceclawVoiceController.java:729](/Users/jbabcock/repositories/g2/faceclaw/App_Resources/Android/src/main/java/com/faceclaw/app/FaceclawVoiceController.java:729). Skipping partial decodes leaves short utterances without text until release, but the dialog still finalizes after its existing timeout. If decoding exceeds roughly 2.7 seconds (native stop’s 1.5-second wait plus the dialog’s 1.2-second timeout), auto-send can abandon a short command or send an incomplete longer one. Continue can append the late editing instruction literally instead of refining the message. I reproduced these outcomes with delayed transcript events. These flows should await recognition completion.
>
> Validation: TypeScript and a clean Android build passed. All 21 test failures also occur on `main`. Actual Whisper latency was not tested on hardware.

<hr>

User: Thanks, let's fix that issue.

GPT-6 Astra:
> Fixed. Dictation now waits for native recognition to finish before fallback timers run. Auto-send, Continue, and AI Chat preserve delayed Whisper results; cancellation ignores late transcripts.
>
> TypeScript and Android builds pass. All 11 new regression tests pass. The full suite still has the same 21 pre-existing failures.
</details>
2026-09-17 22:12:33 -07:00
Chris LeeandClaude Sonnet 5 2d717a9cdb Add on-device Whisper as a second offline transcription option
sherpa-onnx's native library already reads an offline Whisper config
(verified against sherpa-onnx/jni/offline-recognizer.cc at v1.13.0);
only the Java wrapper was a package-private stub. This adds a public
OfflineWhisperModelConfig builder and wires base.en (int8) in as a
second on-device model next to Moonshine.

- FaceclawVoiceController: setOnboardModelKind() selects the model
  before start("onboard"); defaults to Moonshine. Whisper skips the
  700ms live-partial redecode (each call re-encodes the whole utterance
  plus ~10s of tail padding) and gates near-silent segments before
  decode.
- asr-model.ts: ASR_MODELS registry with per-model download state.
- Settings > Voice: "On-device (Moonshine)", "On-device (Whisper)",
  and "OpenAI (Whisper)" for the existing cloud provider (stored
  value unchanged); one model row per on-device model.
- Microphones live captions pass "moonshine" explicitly (unchanged).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 22:46:22 -04:00
James Babcock 937c73d81b Add battery indicator for Android Wear watch 2026-09-12 19:35:38 -07:00
James Babcock dfde776016 Voice input tells you if it can't record due to microphone permissions 2026-09-12 00:47:27 -07:00
James Babcock fb89517a5a Navigate app links to mapbox.com to get a token 2026-09-11 22:21:57 -07:00
James Babcock 6d85c0ac3e Onboarding "Disconnect Other Apps" page handles case where Even app isn't installed 2026-09-11 21:58:31 -07:00
James Babcock b0cf1e4f67 Switch from capability strings to numeric versioning for firmwares 2026-09-11 00:22:36 -07:00
James Babcock 5a97b218e6 Add bluetooth settings options to bandwidth benchmark 2026-09-10 12:21:01 -07:00
James Babcock 50cbc3a756 Merge branch 'main' into ble-bandwidth-test 2026-09-09 21:59:38 -07:00
James Babcock adf9fa89c4 Add R1 ring battery indicator 2026-09-09 11:31:44 -07:00
James Babcock a31ddae4d2 Merge branch 'main' into ios-port 2026-09-08 20:08:24 -07:00