Refactor a bunch of Java to Kotlin, making it usable in an iOS port

This commit is contained in:
James Babcock committed 2026-09-18 22:46:08 -07:00
1 parent c0f8f7e9c2
commit 410fd5840b
162 files changed
+9650 -6224

No files matched your search

+79
View File
@@ -0,0 +1,79 @@
# Shared Kotlin
Both `main` and `ios-port` build the same Kotlin Multiplatform sources in
`native/kotlin/shared/src`. NativeScript's `before-prepare` hook builds an AAR
for Android or a static `FaceclawKit.xcframework` for iOS, then stages it in the
local `@faceclaw/kotlin` plugin. Generated binaries and Gradle caches are ignored.
- `commonMain/kotlin`: wire protocol, fragmentation/reassembly, event decoding,
image/RLE/texture planning, retained surface composition, glyph/image atlases,
LVGL font parsing, spectral noise suppression, PNG/GIF/WAV encodings and
callback interfaces.
- `androidMain/kotlin`: monotonic clock, reentrant locks, JVM zlib and file I/O,
zero-copy `ByteBuffer` readers, firmware hashing/file writes and WAV file storage.
- `iosMain/kotlin`: monotonic clock, recursive locks, native zlib and POSIX file
reads, plus `NSData` facades for NativeScript's Objective-C bridge.
The `com.faceclaw.app` package, `@JvmStatic` methods and `@JvmField` properties
preserve existing Java/NativeScript callers. Android's `java.nio.ByteBuffer`
entry points for atlases/composition now take `AndroidByteReader`, keeping the
bulk buffer crossing without putting Java APIs in common code. iOS uses
`IosByteReader` and `NSData`; binary bridges avoid per-element JS/native calls.
## Platform boundaries
`ProtocolPlatform` supplies clocks, locks and persistent deflaters. `expect` /
`actual` functions select default platform services at compilation. Stateful
transports and caches retain reentrant locking; callbacks still run on their
caller's thread. The migration does not introduce implicit UI-thread dispatch.
`OutboundMessage` and `CfwMessageWindow` require their owner's synchronization.
`CfwTransport` uses ordered `SYNC_FLUSH` records with one persistent compression
context. Lens changes and replay resets reset that context. Owners can close it
explicitly; iOS also cleans up abandoned native allocations. GATT write modes
are mapped by platform adapters.
`migrated-java-sources.json` lists each removed Java source and its Kotlin
replacement. Android's build removes precisely those stale generated copies
before compilation. `android-java-boundaries.json` inventories every retained
Java file and its Android API boundary: services/receivers, GATT, media,
WebView, SQLite, rendering, networking and JNI. The sherpa-onnx Java classes
remain the vendored Android JNI ABI. PNG/GIF adapters retain only Android
storage responsibilities; their codecs are shared.
On `ios-port`, platform adapters route the existing TypeScript session and UI
logic through shared Kotlin for protocol builders, CRC/framing/reassembly,
event decoding, CFW compression/ACKs, changed-image regions, surface composition,
and LVGL glyphs. The original pure TypeScript implementations remain as Node
test references; NativeScript selects the `.ios.ts` native adapters. Features
not yet present in the iOS app can use the shared audio/recording and texture
classes without another Java port.
## Build and validation
Requires JDK 21, Android SDK 35 and the existing `wear/gradlew` wrapper. iOS
additionally requires macOS, Xcode and an arm64 simulator. Configure paths in
`build_paths.sh`. Both production and tests use Kotlin 2.4.20 and Android KMP
plugin 8.13.2.
```sh
npm ci
# Once when upgrading an existing generated Android platform to runtime 9.1.1:
npx nativescript platform clean android
npm run test:kotlin
npm run test:kotlin:ios
./build.sh
# In ios-port:
./scripts/ios.sh build ios --emulator
```
`tests/kotlin` compiles production sources directly. Common tests run on JVM
and iOS: wire vectors, framing/corruption/expiry, streaming compression, replay,
composition, PNG pixels, font glyphs, and Java SHA-256 golden outputs for DSP
and GIF. Android additionally runs the existing Java protocol fixtures and
concurrency checks; iOS exercises `NSData` and compositor adapters. Debug app
startup checks real NativeScript metadata and bidirectional callbacks, logging
`FACECLAW_KOTLIN_BRIDGE_PASS` and `FACECLAW_KOTLIN_PROTOCOL_PASS`.
These checks require no BLE hardware. Physical glasses/ring behavior still
needs device testing, particularly sustained rendering and microphone traffic.
+136
View File
@@ -0,0 +1,136 @@
[
{
"reason": "BluetoothGatt, BluetoothLeScanner, Android callback threads",
"files": [
"com/faceclaw/app/FaceclawBleCommunicator.java",
"com/faceclaw/app/FaceclawBleManager.java",
"com/faceclaw/app/FaceclawDeviceDiscovery.java",
"com/faceclaw/app/FaceclawDeviceInfoProbe.java",
"com/faceclaw/app/FaceclawFirmwareFlasher.java",
"com/faceclaw/app/FaceclawFlashPromptCommunicator.java"
]
},
{
"reason": "AlarmManager, PendingIntent, Android activities/services/receivers",
"files": [
"com/faceclaw/app/FaceclawAlarmActivity.java",
"com/faceclaw/app/FaceclawAlarmReceiver.java",
"com/faceclaw/app/FaceclawAlarmRescheduleReceiver.java",
"com/faceclaw/app/FaceclawAlarmService.java",
"com/faceclaw/app/FaceclawAlarms.java",
"com/faceclaw/app/FaceclawForegroundService.java"
]
},
{
"reason": "Android providers, sensors, permissions and preferences",
"files": [
"com/faceclaw/app/FaceclawCalendarProvider.java",
"com/faceclaw/app/FaceclawLocationProvider.java",
"com/faceclaw/app/FaceclawLocationTracker.java",
"com/faceclaw/app/FaceclawSettings.java",
"com/faceclaw/app/FaceclawSettingsPortReceiver.java",
"com/faceclaw/app/FaceclawResourceUsage.java"
]
},
{
"reason": "Android MediaSession, notifications, status icons and wearable services",
"files": [
"com/faceclaw/app/FaceclawMediaBrowser.java",
"com/faceclaw/app/FaceclawMediaController.java",
"com/faceclaw/app/FaceclawMediaNotificationListenerService.java",
"com/faceclaw/app/FaceclawSystemStatusIconProvider.java",
"com/faceclaw/app/FaceclawWearBridge.java",
"com/faceclaw/app/FaceclawWearListenerService.java"
]
},
{
"reason": "Android WebView/JavascriptInterface, package manager and intent integration",
"files": [
"com/faceclaw/app/FaceclawEvenAppDetector.java",
"com/faceclaw/app/FaceclawEvenHubDocumentStart.java",
"com/faceclaw/app/FaceclawEvenHubJsBridge.java",
"com/faceclaw/app/FaceclawEvenHubWebView.java",
"com/faceclaw/app/FaceclawEvenHubWebViewClient.java",
"com/faceclaw/app/FaceclawEvenHubWebViewHost.java"
]
},
{
"reason": "Android AudioRecord/AudioTrack/MediaCodec, Handler delivery and JNI speech/model engines",
"files": [
"com/faceclaw/app/AndroidNoiseEffects.java",
"com/faceclaw/app/FaceclawAudioTranscoder.java",
"com/faceclaw/app/FaceclawCaptionEngine.java",
"com/faceclaw/app/FaceclawDiarizer.java",
"com/faceclaw/app/FaceclawLc3Decoder.java",
"com/faceclaw/app/FaceclawLlamaRunner.java",
"com/faceclaw/app/FaceclawSpeakerId.java",
"com/faceclaw/app/FaceclawVoiceController.java"
]
},
{
"reason": "Android SQLite persistence and cursor/content APIs",
"files": [
"com/faceclaw/app/FaceclawConversationStore.java"
]
},
{
"reason": "Android OkHttp backend, Handler/Log delivery, download lifecycle",
"files": [
"com/faceclaw/app/FaceclawHttp.java",
"com/faceclaw/app/FaceclawModelDownloader.java",
"com/faceclaw/app/FaceclawSseRequest.java",
"com/faceclaw/app/FaceclawWebSocket.java"
]
},
{
"reason": "Android Bitmap/Canvas/Paint/typeface, SVG rasterization and camera scanning",
"files": [
"com/faceclaw/app/FontFileRenderer.java",
"com/faceclaw/app/IconRenderer.java",
"com/faceclaw/app/ImageFileLoader.java",
"com/faceclaw/app/PreviewBitmapUtil.java",
"com/faceclaw/app/FaceclawQrScanner.java"
]
},
{
"reason": "Android preview/UI delivery and app-scoped export paths; codecs/composition are shared",
"files": [
"com/faceclaw/app/FaceclawPreviewCompositor.java",
"com/faceclaw/app/GifScreenRecorder.java",
"com/faceclaw/app/ScreenshotUtil.java"
]
},
{
"reason": "Android elapsedRealtime, worker wait/notify and Context-backed trace export",
"files": [
"com/faceclaw/app/FrameTimings.java",
"com/faceclaw/app/util/InterruptibleSleep.java"
]
},
{
"reason": "Vendored sherpa-onnx JNI ABI: Java classes/fields are consumed by the Android native library; iOS uses its native engine APIs.",
"files": [
"com/k2fsa/sherpa/onnx/FastClusteringConfig.java",
"com/k2fsa/sherpa/onnx/FeatureConfig.java",
"com/k2fsa/sherpa/onnx/HomophoneReplacerConfig.java",
"com/k2fsa/sherpa/onnx/OfflineModelConfig.java",
"com/k2fsa/sherpa/onnx/OfflineMoonshineModelConfig.java",
"com/k2fsa/sherpa/onnx/OfflineRecognizer.java",
"com/k2fsa/sherpa/onnx/OfflineRecognizerConfig.java",
"com/k2fsa/sherpa/onnx/OfflineRecognizerResult.java",
"com/k2fsa/sherpa/onnx/OfflineSpeakerDiarization.java",
"com/k2fsa/sherpa/onnx/OfflineSpeakerDiarizationCallback.java",
"com/k2fsa/sherpa/onnx/OfflineSpeakerDiarizationConfig.java",
"com/k2fsa/sherpa/onnx/OfflineSpeakerDiarizationSegment.java",
"com/k2fsa/sherpa/onnx/OfflineSpeakerSegmentationModelConfig.java",
"com/k2fsa/sherpa/onnx/OfflineSpeakerSegmentationPyannoteModelConfig.java",
"com/k2fsa/sherpa/onnx/OfflineStream.java",
"com/k2fsa/sherpa/onnx/OfflineWhisperModelConfig.java",
"com/k2fsa/sherpa/onnx/OnlineStream.java",
"com/k2fsa/sherpa/onnx/QnnConfig.java",
"com/k2fsa/sherpa/onnx/SpeakerEmbeddingExtractor.java",
"com/k2fsa/sherpa/onnx/SpeakerEmbeddingExtractorConfig.java",
"com/k2fsa/sherpa/onnx/SpeakerEmbeddingManager.java"
]
}
]
+4
View File
@@ -0,0 +1,4 @@
plugins {
kotlin("multiplatform") version "2.4.20" apply false
id("com.android.kotlin.multiplatform.library") version "8.13.2" apply false
}
+198
View File
@@ -0,0 +1,198 @@
[
{
"java": "com/faceclaw/app/LvglFontFile.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/graphics/LvglFontFile.kt"
},
{
"java": "com/faceclaw/app/FaceclawAlarmListener.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/callbacks/FaceclawAlarmListener.kt"
},
{
"java": "com/faceclaw/app/FaceclawAmbientLightListener.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/callbacks/FaceclawAmbientLightListener.kt"
},
{
"java": "com/faceclaw/app/FaceclawAudioPacketListener.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/callbacks/FaceclawAudioPacketListener.kt"
},
{
"java": "com/faceclaw/app/FaceclawBleCommunicatorListener.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/callbacks/FaceclawBleCommunicatorListener.kt"
},
{
"java": "com/faceclaw/app/FaceclawBleListener.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/callbacks/FaceclawBleListener.kt"
},
{
"java": "com/faceclaw/app/FaceclawCaptionEngineListener.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/callbacks/FaceclawCaptionEngineListener.kt"
},
{
"java": "com/faceclaw/app/FaceclawCompassListener.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/callbacks/FaceclawCompassListener.kt"
},
{
"java": "com/faceclaw/app/FaceclawDeviceDiscoveryListener.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/callbacks/FaceclawDeviceDiscoveryListener.kt"
},
{
"java": "com/faceclaw/app/FaceclawDeviceInfoProbeListener.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/callbacks/FaceclawDeviceInfoProbeListener.kt"
},
{
"java": "com/faceclaw/app/FaceclawEvenHubListener.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/callbacks/FaceclawEvenHubListener.kt"
},
{
"java": "com/faceclaw/app/FaceclawFirmwareFlasherListener.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/callbacks/FaceclawFirmwareFlasherListener.kt"
},
{
"java": "com/faceclaw/app/FaceclawFlashPromptListener.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/callbacks/FaceclawFlashPromptListener.kt"
},
{
"java": "com/faceclaw/app/FaceclawImuListener.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/callbacks/FaceclawImuListener.kt"
},
{
"java": "com/faceclaw/app/FaceclawLlamaListener.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/callbacks/FaceclawLlamaListener.kt"
},
{
"java": "com/faceclaw/app/FaceclawLocationListener.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/callbacks/FaceclawLocationListener.kt"
},
{
"java": "com/faceclaw/app/FaceclawLocationTrackerListener.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/callbacks/FaceclawLocationTrackerListener.kt"
},
{
"java": "com/faceclaw/app/FaceclawMediaBrowserListener.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/callbacks/FaceclawMediaBrowserListener.kt"
},
{
"java": "com/faceclaw/app/FaceclawMediaControllerListener.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/callbacks/FaceclawMediaControllerListener.kt"
},
{
"java": "com/faceclaw/app/FaceclawMicStatusListener.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/callbacks/FaceclawMicStatusListener.kt"
},
{
"java": "com/faceclaw/app/FaceclawModelDownloaderListener.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/callbacks/FaceclawModelDownloaderListener.kt"
},
{
"java": "com/faceclaw/app/FaceclawNoiseSuppressor.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/audio/FaceclawNoiseSuppressor.kt"
},
{
"java": "com/faceclaw/app/FaceclawNotificationListener.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/callbacks/FaceclawNotificationListener.kt"
},
{
"java": "com/faceclaw/app/FaceclawQrScannerListener.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/callbacks/FaceclawQrScannerListener.kt"
},
{
"java": "com/faceclaw/app/FaceclawRingEventDecoder.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/g2protocol/FaceclawRingEventDecoder.kt"
},
{
"java": "com/faceclaw/app/FaceclawSettingsListener.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/callbacks/FaceclawSettingsListener.kt"
},
{
"java": "com/faceclaw/app/FaceclawSseListener.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/callbacks/FaceclawSseListener.kt"
},
{
"java": "com/faceclaw/app/FaceclawVoiceControllerListener.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/callbacks/FaceclawVoiceControllerListener.kt"
},
{
"java": "com/faceclaw/app/FaceclawWearListener.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/callbacks/FaceclawWearListener.kt"
},
{
"java": "com/faceclaw/app/FaceclawWebSocketListener.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/callbacks/FaceclawWebSocketListener.kt"
},
{
"java": "com/faceclaw/app/FwGlyphAtlas.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/graphics/FwGlyphAtlas.kt"
},
{
"java": "com/faceclaw/app/g2protocol/BleImageOptimizer.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/g2protocol/BleImageOptimizer.kt"
},
{
"java": "com/faceclaw/app/g2protocol/BleMagicPool.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/g2protocol/BleMagicPool.kt"
},
{
"java": "com/faceclaw/app/g2protocol/BleProtocol.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/g2protocol/BleProtocol.kt"
},
{
"java": "com/faceclaw/app/g2protocol/CfwMessageWindow.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/g2protocol/CfwMessageWindow.kt"
},
{
"java": "com/faceclaw/app/g2protocol/CfwTransport.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/g2protocol/CfwTransport.kt"
},
{
"java": "com/faceclaw/app/g2protocol/ConnectionOptions.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/g2protocol/ConnectionOptions.kt"
},
{
"java": "com/faceclaw/app/g2protocol/G2Event.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/g2protocol/G2Event.kt"
},
{
"java": "com/faceclaw/app/g2protocol/MessageBuilder.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/g2protocol/MessageBuilder.kt"
},
{
"java": "com/faceclaw/app/g2protocol/OutboundMessage.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/g2protocol/OutboundMessage.kt"
},
{
"java": "com/faceclaw/app/g2protocol/TextureCacheState.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/g2protocol/TextureCacheState.kt"
},
{
"java": "com/faceclaw/app/g2protocol/TexturePlanner.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/g2protocol/TexturePlanner.kt"
},
{
"java": "com/faceclaw/app/GlyphAtlas.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/graphics/GlyphAtlas.kt"
},
{
"java": "com/faceclaw/app/ImageAtlas.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/graphics/ImageAtlas.kt"
},
{
"java": "com/faceclaw/app/SurfaceCompositor.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/graphics/SurfaceCompositor.kt"
},
{
"java": "com/faceclaw/app/util/BmpUtil.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/util/BmpUtil.kt"
},
{
"java": "com/faceclaw/app/util/CollectionUtils.java",
"kotlin": "shared/src/commonMain/kotlin/com/faceclaw/app/util/CollectionUtils.kt"
},
{
"java": "com/faceclaw/app/FaceclawFirmwareUtil.java",
"kotlin": "shared/src/androidMain/kotlin/com/faceclaw/app/FaceclawFirmwareUtil.kt"
},
{
"java": "com/faceclaw/app/FaceclawWavRecorder.java",
"kotlin": "shared/src/androidMain/kotlin/com/faceclaw/app/FaceclawWavRecorder.kt"
}
]
+1
View File
@@ -0,0 +1 @@
// Native APIs are exposed by NativeScript's platform metadata.
+7
View File
@@ -0,0 +1,7 @@
{
"name": "@faceclaw/kotlin",
"version": "0.1.0",
"private": true,
"main": "index.js",
"nativescript": { "platforms": { "android": "9.1.1", "ios": "9.0.3" } }
}
+4
View File
@@ -0,0 +1,4 @@
pluginManagement { repositories { google(); mavenCentral(); gradlePluginPortal() } }
dependencyResolutionManagement { repositories { google(); mavenCentral() } }
rootProject.name = "FaceclawKotlin"
include(":shared")
+25
View File
@@ -0,0 +1,25 @@
import org.jetbrains.kotlin.gradle.dsl.JvmTarget
import org.jetbrains.kotlin.gradle.plugin.mpp.apple.XCFramework
plugins {
kotlin("multiplatform")
id("com.android.kotlin.multiplatform.library")
}
kotlin {
android {
namespace = "com.faceclaw.shared"
compileSdk = 35
minSdk = 24
compilerOptions { jvmTarget.set(JvmTarget.JVM_17) }
withHostTestBuilder {}.configure {}
}
val framework = XCFramework("FaceclawKit")
listOf(iosArm64(), iosSimulatorArm64()).forEach { target ->
target.binaries.framework {
baseName = "FaceclawKit"
isStatic = true
binaryOption("bundleId", "com.faceclaw.shared")
framework.add(this)
}
}
}
@@ -0,0 +1,20 @@
package com.faceclaw.app
import java.nio.ByteBuffer
/** NativeScript's ArrayBuffer-backed ByteBuffer stays on the JVM side. */
class AndroidByteReader(buffer: ByteBuffer) : ByteReader() {
// NativeScript can reuse the same ByteBuffer for an ArrayBuffer. Reading one
// frame must not advance the cursor used by a later atlas/preview crossing.
private val buffer = buffer.duplicate()
override fun remaining(): Int = buffer.remaining()
override fun get(): Byte = buffer.get()
override fun get(index: Int): Byte = buffer.get(index)
override fun get(bytes: ByteArray, offset: Int, length: Int): ByteReader {
buffer.get(bytes, offset, length)
return this
}
}
@@ -0,0 +1,31 @@
package com.faceclaw.app
import java.io.FileOutputStream
import java.nio.ByteBuffer
import java.security.MessageDigest
/** Android's bulk ByteBuffer boundary avoids per-byte NativeScript crossings. */
object FaceclawFirmwareUtil {
@JvmStatic fun bytesToHex(bytes: ByteArray?): String = BinaryEncoding.bytesToHex(bytes)
@JvmStatic
fun sha256Hex(data: ByteBuffer): String {
val digest = MessageDigest.getInstance("SHA-256")
digest.update(data.duplicate())
return bytesToHex(digest.digest())
}
@JvmStatic
fun writeFile(path: String, data: ByteBuffer) {
try {
FileOutputStream(path).use { file ->
file.channel.use { channel ->
val view = data.duplicate()
while (view.hasRemaining()) channel.write(view)
}
}
} catch (e: java.io.IOException) {
throw RuntimeException("firmware write failed: ${e.message}", e)
}
}
}
@@ -0,0 +1,72 @@
package com.faceclaw.app
import android.util.Log
import java.io.File
import java.io.IOException
import java.io.RandomAccessFile
/** Android streaming file adapter; the PCM/WAV format is shared Kotlin. */
class FaceclawWavRecorder
@Throws(IOException::class)
constructor(
private val path: String,
private val sampleRate: Int,
private val channels: Int,
) {
private var file: RandomAccessFile?
private var dataBytes = 0L
init {
val parent = File(path).parentFile
if (parent != null && !parent.exists() && !parent.mkdirs())
throw IOException("could not create $parent")
file =
RandomAccessFile(path, "rw").also {
it.setLength(0)
it.write(BinaryEncoding.wavHeader(0, sampleRate, channels))
}
}
@Synchronized
fun append(pcm16le: ByteArray?) {
val current = file ?: return
if (pcm16le == null || pcm16le.isEmpty()) return
try {
current.write(pcm16le)
dataBytes += pcm16le.size
} catch (e: IOException) {
Log.w("FaceclawWavRecorder", "append failed", e)
}
}
@Synchronized fun getDurationMs(): Long = dataBytes * 1000L / (sampleRate * channels * 2L)
fun getPath(): String = path
@Synchronized
fun finish(): Long {
val current = file ?: return dataBytes
file = null
try {
current.use {
it.seek(0)
it.write(
BinaryEncoding.wavHeader(
minOf(dataBytes, Int.MAX_VALUE.toLong()).toInt(),
sampleRate,
channels,
)
)
}
} catch (e: IOException) {
Log.w("FaceclawWavRecorder", "finish failed", e)
}
return dataBytes
}
@Synchronized
fun discard() {
finish()
File(path).delete()
}
}
@@ -0,0 +1,41 @@
package com.faceclaw.app
internal actual fun protocolPlatform(): ProtocolPlatform = AndroidProtocolPlatform
internal actual fun readFileData(path: String): ByteArray {
val file = java.io.File(path)
require(file.isFile && file.canRead() && file.length() <= Int.MAX_VALUE)
return file.readBytes()
}
internal actual fun deflateData(bytes: ByteArray): ByteArray {
val encoder = java.util.zip.Deflater()
try {
encoder.setInput(bytes)
encoder.finish()
val out = ByteSink()
val buffer = ByteArray(8192)
while (!encoder.finished()) out.write(buffer, 0, encoder.deflate(buffer))
return out.toByteArray()
} finally {
encoder.end()
}
}
internal actual fun inflateData(bytes: ByteArray, size: Int): ByteArray {
val decoder = java.util.zip.Inflater()
try {
decoder.setInput(bytes)
val result = ByteArray(size)
var offset = 0
while (offset < size && !decoder.finished()) {
val count = decoder.inflate(result, offset, size - offset)
if (count == 0) break
offset += count
}
check(offset == size) { "recorded frame truncated: $offset of $size" }
return result
} finally {
decoder.end()
}
}
@@ -0,0 +1,48 @@
package com.faceclaw.app
import android.bluetooth.BluetoothGattCharacteristic
import android.os.SystemClock
import java.io.ByteArrayOutputStream
import java.util.concurrent.locks.ReentrantLock
import java.util.zip.Deflater
object AndroidProtocolPlatform : ProtocolPlatform {
override fun elapsedRealtimeMs(): Long = SystemClock.elapsedRealtime()
override fun createLock(): ProtocolLock =
object : ProtocolLock {
private val lock = ReentrantLock()
override fun lock() = lock.lock()
override fun unlock() = lock.unlock()
}
override fun createDeflater(): ProtocolDeflater = AndroidDeflater()
@JvmStatic
fun writeType(mode: GattWriteMode): Int =
when (mode) {
GattWriteMode.WITH_RESPONSE -> BluetoothGattCharacteristic.WRITE_TYPE_DEFAULT
GattWriteMode.WITHOUT_RESPONSE -> BluetoothGattCharacteristic.WRITE_TYPE_NO_RESPONSE
}
}
private class AndroidDeflater : ProtocolDeflater {
private val deflater = Deflater()
override fun reset() = deflater.reset()
override fun syncFlush(message: ByteArray): ByteArray {
deflater.setInput(message)
val out = ByteArrayOutputStream()
val chunk = ByteArray(4096)
do {
val count = deflater.deflate(chunk, 0, chunk.size, Deflater.SYNC_FLUSH)
out.write(chunk, 0, count)
} while (count == chunk.size)
return out.toByteArray()
}
override fun close() = deflater.end()
}
@@ -0,0 +1,11 @@
package com.faceclaw.shared
import android.os.Looper
internal actual object PlatformInfo {
actual val name: String = "Android"
actual fun threadName(): String = Thread.currentThread().name
actual fun isMainThread(): Boolean = Looper.myLooper() == Looper.getMainLooper()
}
@@ -0,0 +1,10 @@
package com.faceclaw.app
/** Platform selection happens at compilation; protocol algorithms stay shared. */
internal expect fun protocolPlatform(): ProtocolPlatform
internal expect fun deflateData(bytes: ByteArray): ByteArray
internal expect fun inflateData(bytes: ByteArray, size: Int): ByteArray
internal expect fun readFileData(path: String): ByteArray
@@ -0,0 +1,263 @@
package com.faceclaw.app
import kotlin.math.roundToInt
/**
* Spectral suppression for PCM from glasses microphones: 16 ms sqrt-Hann frames at 50% overlap,
* asymmetric noise-floor tracking and a smoothed Wiener gain. Android capture-session effects live
* in AndroidNoiseEffects. This processor has no platform dependencies and must be owned by a single
* audio worker.
*/
class FaceclawNoiseSuppressor {
companion object {
private const val FFT_SIZE: Int = 256
private val HOP: Int = (FFT_SIZE / 2)
private val OVERSUBTRACTION: Float = 1.6f
/** About -16 dB of residual: keeps the noise bed natural instead of gated. */
private val GAIN_FLOOR: Float = 0.15f
private val GAIN_SMOOTHING: Float = 0.6f
/** Frames of pure noise-floor learning after start/reset (~100 ms). */
private const val NOISE_WARMUP_FRAMES: Int = 12
/** In-place iterative radix-2 FFT (inverse includes the 1/N scale). */
private fun fft(re: FloatArray, im: FloatArray, inverse: Boolean): Unit {
var n: Int = re.size
run {
var i: Int = 1
var j: Int = 0
while ((i < n)) {
var bit: Int = (n shr 1)
run {
while (((j and bit) != 0)) {
j = (j xor bit)
bit = (bit shr 1)
}
}
j = (j xor bit)
if ((i < j)) {
var t: Float = re[i]
re[i] = re[j]
re[j] = t
t = im[i]
im[i] = im[j]
im[j] = t
}
i++
}
}
run {
var len: Int = 2
while ((len <= n)) {
var ang: Double = (((2 * kotlin.math.PI) / len) * (if (inverse) 1 else -1))
var wRe: Float = (kotlin.math.cos(ang)).toFloat()
var wIm: Float = (kotlin.math.sin(ang)).toFloat()
run {
var i: Int = 0
while ((i < n)) {
var curRe: Float = 1f
var curIm: Float = 0f
run {
var k: Int = 0
while ((k < (len / 2))) {
var a: Int = (i + k)
var b: Int = (a + (len / 2))
var xr: Float = ((re[b] * curRe) - (im[b] * curIm))
var xi: Float = ((re[b] * curIm) + (im[b] * curRe))
re[b] = (re[a] - xr)
im[b] = (im[a] - xi)
re[a] += xr
im[a] += xi
var nextRe: Float = ((curRe * wRe) - (curIm * wIm))
curIm = ((curRe * wIm) + (curIm * wRe))
curRe = nextRe
k++
}
}
i += len
}
}
len = (len shl 1)
}
}
if (inverse) {
run {
var i: Int = 0
while ((i < n)) {
re[i] /= n
im[i] /= n
i++
}
}
}
}
}
private val sampleRate: Int
private val window: FloatArray = FloatArray(FFT_SIZE)
private val history: FloatArray = FloatArray(FFT_SIZE)
private val overlap: FloatArray = FloatArray(HOP)
private val noise: FloatArray = FloatArray(((FFT_SIZE / 2) + 1))
private val gain: FloatArray = FloatArray(((FFT_SIZE / 2) + 1))
private val re: FloatArray = FloatArray(FFT_SIZE)
private val im: FloatArray = FloatArray(FFT_SIZE)
private val pending: ShortArray = ShortArray(HOP)
private var pendingCount: Int = 0
private var warmupFrames: Int = 0
constructor(sampleRate: Int) {
this.sampleRate = sampleRate
run {
var i: Int = 0
while ((i < FFT_SIZE)) {
window[i] = (kotlin.math.sin(((kotlin.math.PI * (i + 0.5)) / FFT_SIZE))).toFloat()
i++
}
}
reset()
}
fun getSampleRate(): Int {
return sampleRate
}
/** Forget the learned noise floor (e.g. after the suppressor was toggled off). */
fun reset(): Unit {
history.fill(0f)
overlap.fill(0f)
noise.fill(0f)
gain.fill(1f)
pendingCount = 0
warmupFrames = 0
}
/**
* Suppress noise in 16-bit little-endian mono PCM. Streaming: input not filling a whole 8 ms
* hop is carried into the next call, so the output may be up to one hop shorter than the input
* (and longer on a later call); total latency is one hop.
*/
fun process(pcm16le: ByteArray?): ByteArray? {
if (((pcm16le == null) || (pcm16le.size < 2))) {
return pcm16le
}
var sampleCount: Int = (pcm16le.size / 2)
var hops: Int = ((pendingCount + sampleCount) / HOP)
var out: ByteArray = ByteArray(((hops * HOP) * 2))
var outPos: Int = 0
var inPos: Int = 0
while ((inPos < sampleCount)) {
var take: Int = minOf((HOP - pendingCount), (sampleCount - inPos))
run {
var i: Int = 0
while ((i < take)) {
var lo: Int = (pcm16le[((inPos + i) * 2)] and 0xff)
var hi: Int = pcm16le[(((inPos + i) * 2) + 1)].toInt()
pending[(pendingCount + i)] = (((hi shl 8) or lo)).toShort()
i++
}
}
pendingCount += take
inPos += take
if ((pendingCount == HOP)) {
outPos = processHop(out, outPos)
pendingCount = 0
}
}
return out
}
private fun processHop(out: ByteArray, outPos: Int): Int {
var outPos = outPos
history.copyInto(history, 0, HOP, HOP + (FFT_SIZE - HOP))
run {
var i: Int = 0
while ((i < HOP)) {
history[((FFT_SIZE - HOP) + i)] = (pending[i] / 32768f)
i++
}
}
run {
var i: Int = 0
while ((i < FFT_SIZE)) {
re[i] = (history[i] * window[i])
im[i] = 0f
i++
}
}
fft(re, im, false)
var warming: Boolean = (warmupFrames < NOISE_WARMUP_FRAMES)
if (warming) {
warmupFrames++
}
run {
var b: Int = 0
while ((b <= (FFT_SIZE / 2))) {
var power: Float = ((re[b] * re[b]) + (im[b] * im[b]))
if (warming) {
noise[b] =
(if ((warmupFrames == 1)) power
else (noise[b] + ((power - noise[b]) / warmupFrames)))
} else {
if ((power < noise[b])) {
noise[b] += ((power - noise[b]) * 0.2f)
} else {
noise[b] = minOf(power, ((noise[b] * 1.006f) + 1e-10f))
}
}
var clean: Float = (power - (OVERSUBTRACTION * noise[b]))
var g: Float = (if ((clean > 0f)) (clean / power) else 0f)
if ((g < GAIN_FLOOR)) {
g = GAIN_FLOOR
}
g = ((GAIN_SMOOTHING * gain[b]) + ((1f - GAIN_SMOOTHING) * g))
gain[b] = g
re[b] *= g
im[b] *= g
if (((b > 0) && (b < (FFT_SIZE / 2)))) {
re[(FFT_SIZE - b)] *= g
im[(FFT_SIZE - b)] *= g
}
b++
}
}
fft(re, im, true)
run {
var i: Int = 0
while ((i < HOP)) {
var value: Float = ((overlap[i] + (re[i] * window[i])) * 32767f)
var sample: Int = value.roundToInt()
if ((sample > 32767)) {
sample = 32767
}
if ((sample < -32768)) {
sample = -32768
}
out[(outPos + (i * 2))] = ((sample and 0xff)).toByte()
out[((outPos + (i * 2)) + 1)] = (((sample shr 8) and 0xff)).toByte()
i++
}
}
run {
var i: Int = 0
while ((i < HOP)) {
overlap[i] = (re[(HOP + i)] * window[(HOP + i)])
i++
}
}
return (outPos + (HOP * 2))
}
}
@@ -0,0 +1,6 @@
package com.faceclaw.app
/** Phone-side alarm actions (dismiss / snooze), delivered to the JS engine; implemented in JS. */
interface FaceclawAlarmListener {
fun onPhoneAction(id: Long, action: String?, minutes: Int): Unit
}
@@ -0,0 +1,10 @@
package com.faceclaw.app
interface FaceclawAmbientLightListener {
/**
* A CFW ambient-light report (settings-channel field 105) from the master temple. body is the
* raw 24-byte ['A','L',ver,reason,...] record; it is decoded on the TS side
* (app/native/ambient-light.ts).
*/
fun onAmbientLight(body: ByteArray?): Unit
}
@@ -0,0 +1,5 @@
package com.faceclaw.app
interface FaceclawAudioPacketListener {
fun onAudioPacket(data: ByteArray?, arm: String?, arrivalMs: Long): Unit
}
@@ -0,0 +1,37 @@
package com.faceclaw.app
interface FaceclawBleCommunicatorListener {
fun onLog(line: String?): Unit
fun onStateChange(phase: String?, status: String?): Unit
fun onRingEvent(
kind: String?,
containerName: String?,
eventType: Int,
eventSource: Int,
systemExitReasonCode: Int,
frameId: Int,
): Unit
fun onBatteryState(
headsetBattery: Int,
headsetCharging: Int,
ringBattery: Int,
ringCharging: Int,
): Unit
fun onSilentMode(silent: Boolean): Unit
fun onWearState(wearing: Boolean): Unit
fun onPhoneLockState(locked: Boolean): Unit
fun onEvenAppConflict(message: String?): Unit
fun onFrameMetrics(paintMs: Int, transmitMs: Int, tileCount: Int): Unit
fun onFrameFinished(frameId: Int, outcome: String?): Unit
fun onFirmwareInfo(leftVersion: String?, rightVersion: String?, extension: String?): Unit
}
@@ -0,0 +1,7 @@
package com.faceclaw.app
interface FaceclawBleListener {
fun onNotification(address: String?, characteristicUuid: String?, data: ByteArray?): Unit
fun onConnectionStateChange(address: String?, connected: Boolean): Unit
}
@@ -0,0 +1,22 @@
package com.faceclaw.app
interface FaceclawCaptionEngineListener {
/**
* One finished utterance. text may be empty when no ASR model is loaded or recognition produced
* nothing; embedding is the L2-normalized voice-print or null when the speaker model is
* unavailable or the utterance was too short. startMs/endMs are on the engine's sample clock
* (milliseconds of audio since start()).
*/
fun onUtterance(
text: String?,
embedding: FloatArray?,
startMs: Long,
endMs: Long,
peakRms: Double,
): Unit
/** A speech onset was detected; the utterance is now being collected. */
fun onSpeechStart(startMs: Long): Unit
fun onStatus(status: String?): Unit
}
@@ -0,0 +1,14 @@
package com.faceclaw.app
/** Receives stock compass heading and calibration notifications on the thread that subscribed. */
interface FaceclawCompassListener {
fun onCompassEvent(
command: Int,
headingDegrees: Int,
magneticAccuracy: Int,
magneticAnomalies: Int,
orientationSource: Int,
diagnosticFlags: Int,
sampleTimeMs: Long,
): Unit
}
@@ -0,0 +1,17 @@
package com.faceclaw.app
/** Callbacks from FaceclawDeviceDiscovery's live scan to the TypeScript layer. */
interface FaceclawDeviceDiscoveryListener {
/**
* One advertisement (or one bonded device) as a JSON object: {address, name, manufacturerData
* (hex, company id included), rssi|null, txPower|null, connectable|null, bonded, source,
* seenAtMs}. Protocol decoding happens in TypeScript (see app/g2/even-advertisement.ts).
*/
fun onAdvertisement(json: String?): Unit
/** The platform refused to start or continue the scan. */
fun onScanFailed(errorCode: Int, message: String?): Unit
/** Diagnostic line for the pairing log. */
fun onLog(line: String?): Unit
}
@@ -0,0 +1,15 @@
package com.faceclaw.app
/** Callbacks from FaceclawDeviceInfoProbe to the TypeScript layer. */
interface FaceclawDeviceInfoProbeListener {
fun onLog(line: String?): Unit
/** Lifecycle: connecting/authenticating for each arm, then querying. */
fun onState(state: String?, detail: String?): Unit
/** Terminal success: firmware versions and the firmware-extension string (empty on stock). */
fun onResult(leftVersion: String?, rightVersion: String?, extension: String?): Unit
/** Terminal failure (couldn't connect or read the version). */
fun onError(message: String?): Unit
}
@@ -0,0 +1,18 @@
package com.faceclaw.app
/**
* Callbacks from a hosted EvenHub app's WebView to the TypeScript host. All methods are delivered
* on the Android main thread (NativeScript's JS thread), regardless of which WebView-internal
* thread produced them.
*/
interface FaceclawEvenHubListener {
/**
* The app invoked window.flutter_inappwebview.callHandler(handlerName, ...args). argsJson is
* the JSON-encoded argument array; callId identifies the promise to resolve via
* window.__fcResolve(callId, ok, value).
*/
fun onEvenAppMessage(handlerName: String?, argsJson: String?, callId: Int): Unit
/** The WebView finished loading the app's page. */
fun onPageFinished(url: String?): Unit
}
@@ -0,0 +1,30 @@
package com.faceclaw.app
/** Callbacks from FaceclawFirmwareFlasher to the TypeScript layer. */
interface FaceclawFirmwareFlasherListener {
fun onLog(line: String?): Unit
/**
* Fine-grained progress for the UI bar. `lens` is "left"/"right"; component/block indices are
* 1-based against their counts. `bytesSent` / `bytesTotal` cover the whole lens image, so a bar
* driven by them moves evenly even though the components differ wildly in size.
*/
fun onProgress(
lens: String?,
componentIndex: Int,
componentCount: Int,
blockIndex: Int,
blockCount: Int,
bytesSent: Long,
bytesTotal: Long,
): Unit
/**
* Coarse lifecycle. `state` is one of: validating, connecting, flashing, rebooting, done,
* error. `detail` carries the lens name or a message.
*/
fun onState(state: String?, detail: String?): Unit
/** Terminal callback: success or failure, with an explanatory message. */
fun onComplete(success: Boolean, detail: String?): Unit
}
@@ -0,0 +1,25 @@
package com.faceclaw.app
/** Callbacks from FaceclawFlashPromptCommunicator to the TypeScript layer. */
interface FaceclawFlashPromptListener {
fun onLog(line: String?): Unit
/**
* Lifecycle updates. `state` is one of: connecting, connected, prompting, battery, result,
* cancelled, timeout, disconnected, error. `detail` carries the "approved"/"declined" text for
* result, or an error/explanation message.
*/
fun onState(state: String?, detail: String?): Unit
/**
* Battery percent of each arm, read after the user approved (or right after auth when the
* prompt is skipped); -1 when an arm didn't answer. Always fires before onResult(true).
*/
fun onBattery(rightPercent: Int, leftPercent: Int): Unit
/**
* Fires once the user picks a menu row (or, with skipPrompt, once the battery is read): true =
* flash, false = cancel.
*/
fun onResult(approved: Boolean): Unit
}
@@ -0,0 +1,9 @@
package com.faceclaw.app
/**
* Receives IMU (accelerometer) readings decoded from the glasses' sys-event stream. Registered on
* FaceclawBleCommunicator via addImuListener; callbacks are delivered on the Android main thread.
*/
interface FaceclawImuListener {
fun onImuData(x: Double, y: Double, z: Double, eventSource: Int): Unit
}
@@ -0,0 +1,16 @@
package com.faceclaw.app
/**
* Callbacks for one on-phone LLM generation. Like the other bridge listeners, implementations are
* typically created on the NativeScript JS thread and FaceclawLlamaRunner posts these callbacks
* back to that thread's Looper.
*/
interface FaceclawLlamaListener {
/** A piece of generated text (may be a partial word). */
fun onToken(piece: String?): Unit
/** Generation finished: "stop" (EOG), "length", or "cancelled". */
fun onDone(stopReason: String?): Unit
fun onError(message: String?): Unit
}
@@ -0,0 +1,13 @@
package com.faceclaw.app
/** One-shot Android location callbacks for the TypeScript Weather bridge. */
interface FaceclawLocationListener {
fun onLocation(
latitude: Double,
longitude: Double,
accuracyMeters: Float,
timestampMs: Long,
): Unit
fun onError(message: String?): Unit
}
@@ -0,0 +1,19 @@
package com.faceclaw.app
/** Continuous location callbacks for the TypeScript navigation bridge. */
interface FaceclawLocationTrackerListener {
/**
* A new fix. bearingDeg/speedMps are -1 when the fix doesn't carry them (common when
* stationary); accuracyMeters is -1 when unknown.
*/
fun onLocation(
latitude: Double,
longitude: Double,
accuracyMeters: Float,
bearingDeg: Float,
speedMps: Float,
timestampMs: Long,
): Unit
fun onError(message: String?): Unit
}
@@ -0,0 +1,10 @@
package com.faceclaw.app
interface FaceclawMediaBrowserListener {
fun onConnectResult(requestId: Int, connected: Boolean, rootId: String?, error: String?): Unit
fun onBrowseResult(requestId: Int, childrenJson: String?, error: String?): Unit
/** The service connection dropped after connecting (player crashed or was killed). */
fun onDisconnected(): Unit
}
@@ -0,0 +1,23 @@
package com.faceclaw.app
interface FaceclawMediaControllerListener {
/** All observed session owners, before filtering or choosing the active player. */
fun onSessionAppsChanged(appsJson: String?): Unit
fun onStateChange(
playbackState: String?,
packageName: String?,
appName: String?,
title: String?,
artist: String?,
album: String?,
positionMs: Long,
durationMs: Long,
playbackSpeed: Float,
canPlayPause: Boolean,
canSkipNext: Boolean,
canSkipPrevious: Boolean,
accessEnabled: Boolean,
status: String?,
): Unit
}
@@ -0,0 +1,10 @@
package com.faceclaw.app
interface FaceclawMicStatusListener {
/**
* A CFW mic_control field-104 status record from one temple. body is the raw 21-byte
* ['M','C',ver,...] record (decoded on the TS side); arm is "L" or "R" for the temple's own BLE
* link, "?" if unmatched.
*/
fun onMicStatus(body: ByteArray?, arm: String?): Unit
}
@@ -0,0 +1,11 @@
package com.faceclaw.app
/** Progress callbacks for FaceclawModelDownloader, posted to the creating thread's Looper. */
interface FaceclawModelDownloaderListener {
fun onProgress(bytesDownloaded: Long, totalBytes: Long): Unit
/** The file is fully downloaded, verified, and moved to its final path. */
fun onDone(path: String?): Unit
fun onError(message: String?): Unit
}
@@ -0,0 +1,5 @@
package com.faceclaw.app
interface FaceclawNotificationListener {
fun onNotificationPosted(key: String?): Unit
}
@@ -0,0 +1,10 @@
package com.faceclaw.app
/** Callbacks for one FaceclawQrScanner.scan; exactly one method fires. */
interface FaceclawQrScannerListener {
fun onResult(text: String?): Unit
fun onCancelled(): Unit
fun onError(message: String?): Unit
}
@@ -0,0 +1,6 @@
package com.faceclaw.app
/** Change-notification callback for FaceclawSettings; implemented in JS. */
interface FaceclawSettingsListener {
fun onSettingChanged(key: String?): Unit
}
@@ -0,0 +1,16 @@
package com.faceclaw.app
/** Callbacks for FaceclawSseRequest, delivered on the constructing thread's Looper. */
interface FaceclawSseListener {
/** One line of a successful (2xx) streaming response body, without the newline. */
fun onLine(line: String?): Unit
/** The server answered with a non-2xx status; body is the full error body. */
fun onHttpError(code: Int, body: String?): Unit
/** The 2xx response body ended normally. */
fun onComplete(): Unit
/** Network-level failure (connect, TLS, mid-stream drop). Not called after cancel(). */
fun onFailure(message: String?): Unit
}
@@ -0,0 +1,44 @@
package com.faceclaw.app
interface FaceclawVoiceControllerListener {
fun onStatus(status: String?): Unit
/**
* Best transcript of the current utterance so far. REPLACE semantics: text is the complete
* transcript, not a delta — display it verbatim, replacing any previous partial. isFinal marks
* the end of the utterance.
*/
fun onTranscript(text: String?, isFinal: Boolean): Unit
/** Capture has finished, after the final transcript or a startup/recognition error. */
fun onStopped(captureId: Int): Unit
/**
* Decoded microphone audio: 16 kHz mono signed 16-bit little-endian PCM. Emitted in every mode
* so raw-PCM consumers (levels, recording, captions) keep working while the onboard recognizer
* owns the transcript.
*/
fun onPcm(pcm16le: ByteArray?): Unit
/**
* Per-packet DSP metadata the glasses firmware appends to each 50 ms audio packet, computed on
* the raw stereo capture before its mono downmix: the direction-of-arrival in signed degrees (0
* = straight ahead, positive to the wearer's right) and a signal-strength ratio that serves as
* a confidence/energy proxy for the angle.
*/
fun onFrameMeta(angleDegrees: Int, ssr: Int): Unit
/**
* The speaker stopped, in a hands-free session that has no button release to end it. Only fires
* when endpointing was enabled for the session; see FaceclawVoiceController.setEndpointing.
*/
fun onSpeechEnd(): Unit
/**
* Result of speaker verification against the enrolled wearer voice-print, emitted once per
* capture session just before the final transcript. Only fires when verification was configured
* for the session; fails open (isWearer=true) when the utterance was too short to verify or the
* speaker model could not be loaded.
*/
fun onSpeakerVerified(isWearer: Boolean, similarity: Float): Unit
}
@@ -0,0 +1,17 @@
package com.faceclaw.app
/**
* Callbacks from the Android Wear (Wear OS) bridge; implemented in JS. Every call is delivered on
* the thread that registered the listener (see FaceclawWearBridge.setListener).
*/
interface FaceclawWearListener {
/**
* A message from the watch app. `path` is the Data Layer message path (e.g. "/faceclaw/input");
* `json` is the UTF-8 payload (always a JSON object, "{}" when the watch sent none); `nodeId`
* identifies the watch so a reply can be addressed to it.
*/
fun onMessage(path: String?, json: String?, nodeId: String?): Unit
/** The set of reachable watches running the Faceclaw watch app changed. */
fun onWatchConnection(reachable: Boolean, watchName: String?): Unit
}
@@ -0,0 +1,12 @@
package com.faceclaw.app
/** Callbacks for FaceclawWebSocket, delivered on the Android main thread. */
interface FaceclawWebSocketListener {
fun onOpen(): Unit
fun onTextMessage(message: String?): Unit
fun onClosed(code: Int, reason: String?): Unit
fun onFailure(message: String?): Unit
}
@@ -0,0 +1,735 @@
package com.faceclaw.app
import kotlin.jvm.JvmField
import kotlin.jvm.JvmStatic
class BleImageOptimizer {
companion object {
private const val TAG: String = "BleImageOptimizer"
/**
* Split an image into fragments (packets), taking advantage of the fact that each packet
* can declare a logical size longer than its byte payload to get free trailing
* zero-padding.
*
* Each fragment's logical size stays within maxFragmentSize, while trailing zeros inside
* that span can be omitted from the payload.
*/
@JvmStatic
fun planImageFragments(
bmp: ByteArray,
maxFragmentSize: Int,
): MutableList<BleProtocol.ImageFragment> {
return planImageFragments(bmp, maxFragmentSize, false)
}
@JvmStatic
fun planImageFragments(
bmp: ByteArray?,
maxFragmentSize: Int,
reserveFinalByte: Boolean,
): MutableList<BleProtocol.ImageFragment> {
if (((bmp == null) || (bmp.size == 0))) {
return mutableListOf(BleProtocol.ImageFragment(0, ByteArray(0), 0))
}
if ((maxFragmentSize <= 0)) {
throw IllegalArgumentException("maxFragmentSize must be positive")
}
val bulkLength: Int = (if (reserveFinalByte) (bmp.size - 1) else bmp.size)
val fragmentCount: Int =
((((bulkLength + maxFragmentSize) - 1) / maxFragmentSize) +
(if ((bulkLength < bmp.size)) 1 else 0))
val fragments: MutableList<BleProtocol.ImageFragment> = ArrayList(fragmentCount)
run {
var index: Int = 0
while (((index * maxFragmentSize) < bulkLength)) {
var start: Int = (index * maxFragmentSize)
var end: Int = minOf((start + maxFragmentSize), bulkLength)
fragments.add(buildFragment(index, bmp, start, end))
index++
}
}
if ((bulkLength < bmp.size)) {
fragments.add(
BleProtocol.ImageFragment(
fragments.size,
bmp.copyOfRange(bulkLength, bmp.size),
(bmp.size - bulkLength),
)
)
}
return fragments
}
private fun buildFragment(
index: Int,
bmp: ByteArray,
start: Int,
end: Int,
): BleProtocol.ImageFragment {
var dataEnd: Int = end
while (((dataEnd > start) && (bmp[(dataEnd - 1)].toInt() == 0))) {
dataEnd -= 1
}
if ((dataEnd == start)) {
return BleProtocol.ImageFragment(index, ByteArray(0), (end - start))
}
return BleProtocol.ImageFragment(index, bmp.copyOfRange(start, dataEnd), (end - start))
}
@JvmStatic
fun buildIncrementalImagePayload(
previous: ByteArray?,
next: ByteArray?,
width: Int,
height: Int,
frameId: Int,
): IncrementalPlan? {
var stride: Int = ((width + 1) shr 1)
if (
((((((width <= 0) || (height <= 0)) || (previous == null)) || (next == null)) ||
(previous.size != (stride * height))) || (next.size != (stride * height)))
) {
return null
}
var minByteX: Int = stride
var maxByteX: Int = -1
var minY: Int = height
var maxY: Int = -1
var changedBytes: Int = 0
var columnChanged: BooleanArray = BooleanArray(stride)
run {
var y: Int = 0
while ((y < height)) {
var rowOffset: Int = (y * stride)
run {
var x: Int = 0
while ((x < stride)) {
if ((previous[(rowOffset + x)] != next[(rowOffset + x)])) {
changedBytes++
columnChanged[x] = true
if ((x < minByteX)) {
minByteX = x
}
if ((x > maxByteX)) {
maxByteX = x
}
if ((y < minY)) {
minY = y
}
maxY = y
}
x++
}
}
y++
}
}
if ((maxY < 0)) {
return null
}
var clusterCount: Int = 0
var inCluster: Boolean = false
run {
var x: Int = minByteX
while ((x <= maxByteX)) {
if (columnChanged[x]) {
if (!inCluster) {
clusterCount++
}
inCluster = true
} else {
inCluster = false
}
x++
}
}
var left: Int = ((minByteX * 2) and 3.inv())
var rightExclusive: Int = minOf(width, ((((maxByteX + 1) * 2) + 3) and 3.inv()))
var top: Int = (minY and 1.inv())
var bottomExclusive: Int = minOf(height, ((maxY + 2) and 1.inv()))
var boxWidth: Int = (rightExclusive - left)
var boxHeight: Int = (bottomExclusive - top)
if (((boxWidth >= width) && (boxHeight >= height))) {
return null
}
var out: ByteArray =
encodeMode3Rect(next, stride, left, top, boxWidth, boxHeight, frameId)
return IncrementalPlan(out, changedBytes, ((boxWidth shr 1) * boxHeight), clusterCount)
}
/**
* The aligned changed bounding box between two same-size packed frames, or null when they
* are identical: pixel-space {left, top, width, height} with left/width multiples of 4 and
* top/height multiples of 2 (mode-3 header quantization). Exposed so the texture-cache
* planner can reuse the exact rect geometry the single-bbox path would send.
*/
@JvmStatic
fun computeChangedBox(
previous: ByteArray?,
next: ByteArray?,
width: Int,
height: Int,
): IntArray? {
var stride: Int = ((width + 1) shr 1)
if (
((((((width <= 0) || (height <= 0)) || (previous == null)) || (next == null)) ||
(previous.size != (stride * height))) || (next.size != (stride * height)))
) {
return null
}
var minByteX: Int = stride
var maxByteX: Int = -1
var minY: Int = height
var maxY: Int = -1
run {
var y: Int = 0
while ((y < height)) {
var rowOffset: Int = (y * stride)
run {
var x: Int = 0
while ((x < stride)) {
if ((previous[(rowOffset + x)] != next[(rowOffset + x)])) {
if ((x < minByteX)) {
minByteX = x
}
if ((x > maxByteX)) {
maxByteX = x
}
if ((y < minY)) {
minY = y
}
maxY = y
}
x++
}
}
y++
}
}
if ((maxY < 0)) {
return null
}
var left: Int = ((minByteX * 2) and 3.inv())
var rightExclusive: Int = minOf(width, ((((maxByteX + 1) * 2) + 3) and 3.inv()))
var top: Int = (minY and 1.inv())
var bottomExclusive: Int = minOf(height, ((maxY + 2) and 1.inv()))
return intArrayOf(left, top, (rightExclusive - left), (bottomExclusive - top))
}
/**
* The multi-rect split of the changed region (see computeSplitRects), or null when it is
* one region / too fragmented / the frames aren't comparable. Exposed for the texture-cache
* planner.
*/
@JvmStatic
fun computeChangedRects(
previous: ByteArray?,
next: ByteArray?,
width: Int,
height: Int,
maxRects: Int,
): MutableList<IntArray>? {
var stride: Int = ((width + 1) shr 1)
if (
((((((width <= 0) || (height <= 0)) || (previous == null)) || (next == null)) ||
(previous.size != (stride * height))) || (next.size != (stride * height)))
) {
return null
}
var rects: MutableList<IntArray>? =
computeSplitRects(previous, next, stride, width, height, maxRects)
return (if (((rects == null) || (rects.size < 2))) null else rects)
}
/**
* Encode one CFW mode-3 rectangle delta:
* [3][left/4][top/2][width/4][height/2][fid_lo][fid_hi][rle(box pixels)] left/width must be
* multiples of 4 (=> left>>1, width>>1 are whole bytes), top/ height multiples of 2. The
* box pixels are top-down rows of `next` (4bpp packed, width>>1 bytes/row), run-length
* encoded before deflate. Shared by the single-bbox path, each rect of a mode-8 multi-rect
* batch, and the texture-cache planner.
*/
@JvmStatic
fun encodeMode3Rect(
next: ByteArray,
stride: Int,
left: Int,
top: Int,
boxWidth: Int,
boxHeight: Int,
fid: Int,
): ByteArray {
var regionStride: Int = (boxWidth shr 1)
var region: ByteArray = ByteArray((regionStride * boxHeight))
run {
var y: Int = 0
while ((y < boxHeight)) {
next.copyInto(
region,
(y * regionStride),
(((top + y) * stride) + (left shr 1)),
(((top + y) * stride) + (left shr 1)) + regionStride,
)
y++
}
}
var compressed: ByteArray = rleEncode(region)
var out: ByteArray = ByteArray((7 + compressed.size))
out[0] = 3
out[1] = ((left / 4)).toByte()
out[2] = ((top / 2)).toByte()
out[3] = ((boxWidth / 4)).toByte()
out[4] = ((boxHeight / 2)).toByte()
out[5] = ((fid and 0xff)).toByte()
out[6] = (((fid shr 8) and 0xff)).toByte()
compressed.copyInto(out, 7, 0, 0 + compressed.size)
return out
}
/** A full repaint in independently decodable commands below the uint16 record limit. */
@JvmStatic
fun encodeFullFrameBands(
packed: ByteArray,
width: Int,
height: Int,
firstFid: Int,
): MutableList<ByteArray> {
if (
(((((((width <= 0) || (width > 640)) || ((width and 3) != 0)) || (height <= 0)) ||
(height > 480)) || ((height and 1) != 0)) ||
(packed.size != ((width * height) / 2)))
) {
throw IllegalArgumentException("unsupported custom framebuffer dimensions")
}
var commands: MutableList<ByteArray> = ArrayList()
var fid: Int = firstFid
run {
var top: Int = 0
while ((top < height)) {
commands.add(
encodeMode3Rect(
packed,
(width / 2),
0,
top,
width,
minOf(64, (height - top)),
fid,
)
)
fid = (if ((fid >= 0xfffe)) 1 else (fid + 1))
top += 64
}
}
return commands
}
private const val MULTI_RECT_V_GAP_ROWS: Int = 4
private const val MULTI_RECT_H_GAP_BYTES: Int = 6
/**
* Build a mode-8 multi-rect payload for the change between two same-size 4bpp frames,
* assigning consecutive frame ids starting at fidStart (each rect needs a distinct fid —
* the CFW skips duplicate fids). Returns null when the change is a single region (use the
* single-rect path), when it fragments past maxRects, or when the frames aren't comparable.
*/
@JvmStatic
fun buildMultiRectImagePayload(
previous: ByteArray?,
next: ByteArray?,
width: Int,
height: Int,
fidStart: Int,
maxRects: Int,
): MultiRectPlan? {
var stride: Int = ((width + 1) shr 1)
if (
((((((width <= 0) || (height <= 0)) || (previous == null)) || (next == null)) ||
(previous.size != (stride * height))) || (next.size != (stride * height)))
) {
return null
}
var rects: MutableList<IntArray>? =
computeSplitRects(previous, next, stride, width, height, maxRects)
if (((rects == null) || (rects.size < 2))) {
return null
}
var subs: MutableList<ByteArray> = ArrayList(rects.size)
var coveredBytes: Int = 0
var fid: Int = fidStart
var total: Int = 2
for (r in rects) {
var sub: ByteArray = encodeMode3Rect(next, stride, r[0], r[1], r[2], r[3], fid)
subs.add(sub)
coveredBytes += ((r[2] shr 1) * r[3])
total += (2 + sub.size)
fid = (if ((fid >= 0xfffe)) 1 else (fid + 1))
}
var out: ByteArray = ByteArray(total)
out[0] = 8
out[1] = (rects.size).toByte()
var pos: Int = 2
for (sub in subs) {
out[pos] = ((sub.size and 0xff)).toByte()
out[(pos + 1)] = (((sub.size shr 8) and 0xff)).toByte()
pos += 2
sub.copyInto(out, pos, 0, 0 + sub.size)
pos += sub.size
}
return MultiRectPlan(out, rects.size, coveredBytes, fid)
}
/**
* Split the changed region between two packed 4bpp frames into a small set of aligned
* rectangles that together cover every changed pixel: maximal vertical bands of changed
* rows (merging gaps < V_GAP), each split into horizontal column clusters (merging gaps <
* H_GAP), with each cluster's rows tightened to where it actually changes. Rects are
* pixel-space {left,top,width,height}, left/width aligned to 4px and top/height to 2px.
* Returns null if there's no change or the split exceeds maxRects (caller falls back to a
* single bounding box).
*/
private fun computeSplitRects(
previous: ByteArray,
next: ByteArray,
stride: Int,
width: Int,
height: Int,
maxRects: Int,
): MutableList<IntArray>? {
var rowChanged: BooleanArray = BooleanArray(height)
var anyChange: Boolean = false
run {
var y: Int = 0
while ((y < height)) {
var off: Int = (y * stride)
run {
var x: Int = 0
while ((x < stride)) {
if ((previous[(off + x)] != next[(off + x)])) {
rowChanged[y] = true
anyChange = true
break
}
x++
}
}
y++
}
}
if (!anyChange) {
return null
}
var rects: MutableList<IntArray> = ArrayList()
var col: BooleanArray = BooleanArray(stride)
var y: Int = 0
while ((y < height)) {
if (!rowChanged[y]) {
y++
continue
}
var bandTop: Int = y
var bandBot: Int = y
var cur: Int = (y + 1)
while ((cur < height)) {
if (rowChanged[cur]) {
bandBot = cur
cur++
continue
}
var g: Int = cur
while (((g < height) && !rowChanged[g])) {
g++
}
if (((g < height) && ((g - cur) < MULTI_RECT_V_GAP_ROWS))) {
cur = g
} else {
break
}
}
col.fill(false)
run {
var yy: Int = bandTop
while ((yy <= bandBot)) {
if (!rowChanged[yy]) {
yy++
continue
}
var off: Int = (yy * stride)
run {
var x: Int = 0
while ((x < stride)) {
if ((previous[(off + x)] != next[(off + x)])) {
col[x] = true
}
x++
}
}
yy++
}
}
var x: Int = 0
while ((x < stride)) {
if (!col[x]) {
x++
continue
}
var cL: Int = x
var cR: Int = x
var cx: Int = (x + 1)
while ((cx < stride)) {
if (col[cx]) {
cR = cx
cx++
continue
}
var g: Int = cx
while (((g < stride) && !col[g])) {
g++
}
if (((g < stride) && ((g - cx) < MULTI_RECT_H_GAP_BYTES))) {
cx = g
} else {
break
}
}
var tTop: Int = height
var tBot: Int = -1
run {
var yy: Int = bandTop
while ((yy <= bandBot)) {
if (!rowChanged[yy]) {
yy++
continue
}
var off: Int = (yy * stride)
run {
var xx: Int = cL
while ((xx <= cR)) {
if ((previous[(off + xx)] != next[(off + xx)])) {
if ((yy < tTop)) {
tTop = yy
}
tBot = yy
break
}
xx++
}
}
yy++
}
}
if ((tBot >= 0)) {
var left: Int = ((cL * 2) and 3.inv())
var rightExclusive: Int = minOf(width, ((((cR + 1) * 2) + 3) and 3.inv()))
var top: Int = (tTop and 1.inv())
var bottomExclusive: Int = minOf(height, ((tBot + 2) and 1.inv()))
rects.add(
intArrayOf(left, top, (rightExclusive - left), (bottomExclusive - top))
)
if ((rects.size > maxRects)) {
return null
}
}
x = (cR + 1)
}
y = (bandBot + 1)
}
return rects
}
/**
* Run-length encode headerless 4bpp pixels for CFW load_image_z mode 6. Wire format:
* [6][rle(4bpp pixels)]. Always use mode 6: the logical image can be larger than its
* EvenHub carrier, so a raw BMP fallback would carry dimensions the legacy container loader
* cannot accept.
*/
@JvmStatic
fun maybeCompress(packed: ByteArray?, width: Int, height: Int): ByteArray {
if (((packed == null) || (packed.size == 0))) {
return packed ?: ByteArray(0)
}
var z: ByteArray = rleEncode(packed)
var out: ByteArray = ByteArray((z.size + 1))
out[0] = 6
z.copyInto(out, 1, 0, 0 + z.size)
return out
}
/**
* Run-length encode packed 4bpp pixels for CFW modes 3 and 6, whose payload is RLE tokens;
* deflate belongs to the transport. Runs are over the pixel NIBBLES of {@code pix} in wire
* order (high nibble = left pixel), including the pad nibble that ends each row at odd
* widths — i.e. {@code pix} read as 2*length nibbles. One token is:
* <pre>
* [cnt4|color4] cnt 1..15
* [0|color4][cnt8] cnt 1..255
* [0|color4][0][cntLo][cntHi] cnt 1..65535, little-endian
* </pre>
* with the low nibble always the color; runs longer than 65535 split across tokens. Worst
* case (every nibble its own run) is exactly one byte per nibble, which is what the buffer
* is sized for.
*/
@JvmStatic
fun rleEncode(pix: ByteArray): ByteArray {
var n: Int = (pix.size * 2)
var out: ByteArray = ByteArray(n)
var o: Int = 0
var i: Int = 0
while ((i < n)) {
var v: Int = nibbleAt(pix, i)
var j: Int = (i + 1)
while (((j < n) && (nibbleAt(pix, j) == v))) {
j++
}
var run: Int = (j - i)
while ((run > 0)) {
var c: Int = minOf(run, 0xffff)
if ((c <= 15)) {
out[o++] = (((c shl 4) or v)).toByte()
} else {
if ((c <= 255)) {
out[o++] = (v).toByte()
out[o++] = (c).toByte()
} else {
out[o++] = (v).toByte()
out[o++] = 0
out[o++] = ((c and 0xff)).toByte()
out[o++] = ((c shr 8)).toByte()
}
}
run -= c
}
i = j
}
return out.copyOf(o)
}
private fun nibbleAt(pix: ByteArray, i: Int): Int {
var b: Int = (pix[(i shr 1)] and 0xff)
return (if (((i and 1) != 0)) (b and 0x0f) else (b shr 4))
}
}
constructor() {}
class TileImagePlan {
@JvmField val tileIndex: Int
@JvmField val tile: BleProtocol.ImageTileOptions
@JvmField val packed: ByteArray?
@JvmField val width: Int
@JvmField val height: Int
@JvmField val payload: ByteArray
@JvmField val sessionId: Int
@JvmField var fragments: MutableList<BleProtocol.ImageFragment> = mutableListOf()
constructor(
tileIndex: Int,
tile: BleProtocol.ImageTileOptions,
packed: ByteArray?,
width: Int,
height: Int,
sessionId: Int,
) {
this.tileIndex = tileIndex
this.tile = tile
this.packed = (if ((packed == null)) ByteArray(0) else packed)
this.width = width
this.height = height
this.payload = maybeCompress(this.packed, width, height)
this.sessionId = sessionId
}
constructor(
tileIndex: Int,
tile: BleProtocol.ImageTileOptions,
packed: ByteArray?,
width: Int,
height: Int,
sessionId: Int,
payloadOverride: ByteArray,
) {
this.tileIndex = tileIndex
this.tile = tile
this.packed = (if ((packed == null)) ByteArray(0) else packed)
this.width = width
this.height = height
this.payload = payloadOverride
this.sessionId = sessionId
}
}
/**
* A mode-3 incremental payload plus diagnostics about how tightly the bounding box fit the
* actual change. {@code changedBytes} is how many packed bytes actually differ; {@code
* boxBytes} is how many the box sends. A large box with few changed bytes / multiple clusters
* means distant small edits were merged into one oversized box (which can spill past the
* fragment size into an extra serialized BLE message).
*/
class IncrementalPlan {
@JvmField val payload: ByteArray
@JvmField val changedBytes: Int
@JvmField val boxBytes: Int
@JvmField val clusterCount: Int
constructor(payload: ByteArray, changedBytes: Int, boxBytes: Int, clusterCount: Int) {
this.payload = payload
this.changedBytes = changedBytes
this.boxBytes = boxBytes
this.clusterCount = clusterCount
}
}
/**
* A CFW mode-8 multi-segment payload (`[8][count]` then per-rect `[len16][mode-3 submsg]`)
* carrying several tight rectangle deltas instead of one bounding box, plus diagnostics. Each
* rect is applied to the firmware shadow with the panel push deferred, then presented once.
* Used when the change splits into regions whose bounding box wastes bytes (distant edits, or a
* sparse box); the caller falls back to the single-rect path when this returns null or isn't
* smaller.
*/
class MultiRectPlan {
@JvmField val payload: ByteArray
@JvmField val rectCount: Int
@JvmField val coveredBytes: Int
@JvmField val nextFid: Int
constructor(payload: ByteArray, rectCount: Int, coveredBytes: Int, nextFid: Int) {
this.payload = payload
this.rectCount = rectCount
this.coveredBytes = coveredBytes
this.nextFid = nextFid
}
}
class ImageUpdateStats {
@JvmField val paintMs: Int
@JvmField val tileCount: Int
@JvmField val frameId: Int
@JvmField var firstWriteStartedAtMs: Long = 0L
constructor(paintMs: Int, tileCount: Int, frameId: Int) {
this.paintMs = maxOf(0, paintMs)
this.tileCount = tileCount
this.frameId = frameId
}
}
}
@@ -0,0 +1,43 @@
package com.faceclaw.app
import kotlin.jvm.JvmField
/**
* Allocates one-byte message identifiers in LRU order. Release records survive reuse so the
* communicator can diagnose late ACKs after a timeout.
*/
class BleMagicPool(private val platform: ProtocolPlatform) {
companion object {
const val MIN_MAGIC = 100
const val MAX_MAGIC = 255
}
private val lock = platform.createLock()
private val available = ArrayDeque((MIN_MAGIC..MAX_MAGIC).toList())
private val allocated = BooleanArray(256)
private val releaseRecords = mutableMapOf<String, ReleaseRecord>()
fun allocate(): Int = lock.withLock {
check(available.isNotEmpty()) { "no BLE magic values available" }
val magic = available.removeFirst()
allocated[magic] = true
magic
}
fun release(sid: Int, magic: Int, label: String?, reason: String?) = lock.withLock {
if (magic !in MIN_MAGIC..MAX_MAGIC) return@withLock
releaseRecords["$sid:$magic"] = ReleaseRecord(label, reason, platform.elapsedRealtimeMs())
if (!allocated[magic]) return@withLock
allocated[magic] = false
available.addLast(magic)
}
fun getReleaseRecord(sid: Int, magic: Int): ReleaseRecord? = lock.withLock {
releaseRecords["$sid:$magic"]
}
class ReleaseRecord(label: String?, reason: String?, @JvmField val releasedAtMs: Long) {
@JvmField val label: String = label ?: ""
@JvmField val reason: String = reason ?: ""
}
}
@@ -0,0 +1,44 @@
package com.faceclaw.app
import kotlin.jvm.JvmStatic
/** Ordered completion and go-back-N recovery for the custom-message window. */
class CfwMessageWindow private constructor() {
companion object {
// Flappy capture: p99 ACK 63ms, max 77ms. Allow full-frame processing
// while recovering a lost reply well before the stock timeout.
const val ACK_TIMEOUT_MS = 500
const val MAX_RETRIES = 3
@JvmStatic
fun acknowledgedHead(messages: Iterable<OutboundMessage>): OutboundMessage? {
for (message in messages) {
if (message.sid != CfwTransport.SID) continue
return if (!message.cfwRetryPending && message.cfwAckLenses == CfwTransport.BOTH)
message
else null
}
return null
}
@JvmStatic
fun replayWindow(messages: Iterable<OutboundMessage>, nowMs: Long): List<OutboundMessage> {
val replay = mutableListOf<OutboundMessage>()
var failed = false
for (message in messages) {
if (message.sid != CfwTransport.SID) continue
failed =
failed ||
message.cfwRetryPending ||
(message.cfwAckLenses != CfwTransport.BOTH &&
message.ackDeadlineAtMs > 0 &&
message.ackDeadlineAtMs <= nowMs)
replay.add(message)
}
// Replaying only the named failure can strand an older unresolved
// attempt at the head. Rewind the entire unresolved custom window.
if (!failed) replay.clear()
return replay
}
}
}
@@ -0,0 +1,190 @@
package com.faceclaw.app
import kotlin.jvm.JvmField
import kotlin.jvm.JvmStatic
/** Faceclaw/10 records: clear flags, wire length and decoded CRC; persistent zlib body. */
class CfwTransport(private val platform: ProtocolPlatform) {
private val lock = platform.createLock()
private var deflater: ProtocolDeflater? = null
private var resetPending = true
private var previousLenses = 0
fun reset() = lock.withLock {
deflater?.reset()
resetPending = true
previousLenses = 0
}
/** Release native compression resources. A later encode starts a fresh stream. */
fun close() = lock.withLock {
deflater?.close()
deflater = null
resetPending = true
previousLenses = 0
}
/** Compress only when writing, in exactly the order seen by the receiver. */
fun encode(message: ByteArray?, streamId: Int, lenses: Int, mtu: Int): List<ByteArray> =
lock.withLock {
validate(message, streamId, lenses, mtu)
val input = message!!
if (previousLenses != lenses) reset()
val reset = resetPending
val encoder = deflater ?: platform.createDeflater().also { deflater = it }
var body = encoder.syncFlush(input)
var flags = lenses or COMPRESSED or (if (reset) RESET_CONTEXT else 0)
if (body.isEmpty()) flags = flags and COMPRESSED.inv() // repeated empty sync flush
if (body.size > MAX_MESSAGE) {
// The attempted deflate advanced history: reset both ends before reuse.
reset()
body = input
flags = lenses or RESET_CONTEXT
} else {
resetPending = false
previousLenses = lenses
}
frameRecord(body, crc(input), streamId, flags, mtu)
}
companion object {
const val SID = 0xf0
const val BOTH = 3
const val MAX_MESSAGE = 65535
const val COMPRESSED = 4
const val RESET_CONTEXT = 8
private fun validate(message: ByteArray?, streamId: Int, lenses: Int, mtu: Int) {
require(
message != null &&
message.size <= MAX_MESSAGE &&
streamId in 0..255 &&
lenses in 1..BOTH &&
mtu in 23..517
) {
"invalid CFW stream parameters"
}
}
/** One logical command per stream. Packets are bounded by actual ATT MTU. */
@JvmStatic
fun frame(message: ByteArray?, streamId: Int, lenses: Int, mtu: Int): List<ByteArray> {
validate(message, streamId, lenses, mtu)
return frameRecord(message!!, crc(message), streamId, lenses or RESET_CONTEXT, mtu)
}
private fun frameRecord(
body: ByteArray,
checksum: Int,
streamId: Int,
flags: Int,
mtu: Int,
): List<ByteArray> {
val lenses = flags and BOTH
val capacity = minOf(252, mtu - 14)
val stream = ByteArray(body.size + 5)
stream[0] = flags.toByte()
stream[1] = body.size.toByte()
stream[2] = (body.size ushr 8).toByte()
stream[3] = checksum.toByte()
stream[4] = (checksum ushr 8).toByte()
body.copyInto(stream, 5)
val packets = mutableListOf<ByteArray>()
for (offset in stream.indices step capacity) {
val count = minOf(capacity, stream.size - offset)
val packet = ByteArray(count + 11)
packet[0] = 0xaa.toByte()
packet[1] = 0x21
packet[2] = (streamId + packets.size).toByte()
packet[3] = (count + 3).toByte()
packet[4] = 1
packet[5] = 1
packet[6] = SID.toByte()
packet[8] =
(lenses or
(if (offset == 0) 0x80 else 0) or
(if (offset + count == stream.size) 0x40 else 0))
.toByte()
stream.copyInto(packet, 9, offset, offset + count)
val crc = crc(packet, 8, count + 1)
packet[packet.size - 2] = crc.toByte()
packet[packet.size - 1] = (crc ushr 8).toByte()
packets.add(packet)
}
return packets
}
@JvmStatic fun crc(data: ByteArray): Int = crc(data, 0, data.size)
private fun crc(data: ByteArray, offset: Int, count: Int): Int {
var crc = 0xffff
for (i in offset until offset + count) {
crc = crc xor ((data[i].toInt() and 255) shl 8)
repeat(8) {
crc = ((crc shl 1) xor (if (crc and 0x8000 != 0) 0x1021 else 0)) and 65535
}
}
return crc
}
private fun u16(data: ByteArray, offset: Int): Int =
(data[offset].toInt() and 255) or ((data[offset + 1].toInt() and 255) shl 8)
@JvmStatic fun parseAck(packet: ByteArray?): Ack? = parseAcks(packet)?.first()
/**
* Validate the entire primary ACK and up to three preceding successes before applying any.
*/
@JvmStatic
fun parseAcks(packet: ByteArray?): Array<Ack>? {
if (
packet == null ||
packet.size !in 19..40 ||
(packet.size - 19) % 7 != 0 ||
packet[0].toInt() and 255 != 0xaa ||
packet[1].toInt() != 0x12 ||
packet[3].toInt() and 255 != packet.size - 8 ||
packet[4].toInt() != 1 ||
packet[5].toInt() != 1 ||
packet[6].toInt() and 255 != SID ||
packet[7].toInt() != 0 ||
(packet[8].toInt() != 1 && packet[8].toInt() != 3) ||
(packet[12].toInt() != 1 && packet[12].toInt() != 2) ||
(packet[8].toInt() == 3 && packet.size != 19) ||
crc(packet, 8, packet.size - 10) != u16(packet, packet.size - 2)
)
return null
return Array(1 + (packet.size - 19) / 7) { i ->
if (i == 0)
Ack(
packet[8].toInt() == 3,
packet[9].toInt() and 255,
u16(packet, 10),
packet[12].toInt(),
u16(packet, 13),
u16(packet, 15),
)
else {
val offset = 17 + (i - 1) * 7
Ack(
false,
packet[offset].toInt() and 255,
u16(packet, offset + 1),
packet[12].toInt(),
u16(packet, offset + 3),
u16(packet, offset + 5),
)
}
}
}
}
class Ack(
@JvmField val nack: Boolean,
@JvmField val streamId: Int,
@JvmField val messageId: Int,
@JvmField val lens: Int,
@JvmField val size: Int,
@JvmField val checksum: Int,
)
}
@@ -0,0 +1,78 @@
package com.faceclaw.app
import kotlin.jvm.JvmField
class ConnectionOptions {
companion object {
@JvmField val WRITE_MODE = GattWriteMode.WITHOUT_RESPONSE
const val DESIRED_MTU = 512
const val CONNECT_TIMEOUT_MS = 5_000
const val SERVICES_TIMEOUT_MS = 5_000
const val DESCRIPTOR_TIMEOUT_MS = 5_000
const val RING_DESIRED_MTU = 247
const val RING_RECONNECT_DELAY_MS = 2_000
const val WRITE_TIMEOUT_MS = 2_000
const val PRELUDE_TIMEOUT_MS = 2_000
// Application security-auth exchange (sid 0x80). The success notification
// only arrives once the BLE link is encrypted, which on a first-time
// connection means SMP pairing — possibly with an OS dialog the user has
// to accept — so the stock-firmware paths wait generously.
const val SECURITY_AUTH_TIMEOUT_MS = 30_000
// The main communicator's wait is soft (it proceeds on timeout so firmware
// that never answers can't stall every reconnect), so it is kept short.
const val SECURITY_AUTH_SOFT_TIMEOUT_MS = 6_000
const val ACK_TIMEOUT_MS = 3_500
const val HEARTBEAT_FAILURE_DEADLINE_MS = 10_000
const val HEARTBEAT_READY_MS = 4_000
const val HEARTBEAT_URGENT_MS = 6_000
const val BATTERY_REFRESH_INTERVAL_MS = 5 * 60_000
const val BATTERY_INPUT_QUIET_MS = 5_000
// Poll cadence while in charging mode; also bounds how quickly we notice
// the glasses coming back out of the case.
const val CHARGING_BATTERY_POLL_MS = 30_000
const val IMAGE_FRAGMENT_SIZE = 3800
const val IMAGE_RETRY_DELAY_MS = 2_000
const val MAX_CONSECUTIVE_ACK_TIMEOUTS = 8
const val EVEN_APP_WRITE_FAILURE_WINDOW_MS = 15_000
const val IDLE_SLEEP_MS = 100
const val RECONNECT_DELAY_MS = 2_000
// Cap on rects per batch. Each rect consumes a distinct CFW frame id, and the
// firmware's duplicate-fid ring holds 16, so keep several batches of history
// within it. Above this the split is abandoned for a single bounding box.
const val MULTI_RECT_MAX_RECTS = 6
// Don't bother splitting when the single bounding box already compresses this
// small (one tight rect); only attempt multi-rect above it or when the box
// spans multiple horizontal clusters.
const val MULTI_RECT_MIN_PAYLOAD = 900
}
@JvmField val sendImagesToLeft = true
@JvmField val skipSessionIds = true
// Max messages in flight (awaiting ack) at once — full pipelining. >1 lets a
// frame be sent before the previous frame's ack returns, hiding the BLE ack
// round-trip that otherwise serialized every update. Requires firmware that
// accepts pipelined image messages (the CFW snapshot/deferred FIFO). At 1 the
// communicator behaves exactly as before (serial sends + last-packet prewrite).
@JvmField val WINDOW_SIZE = 3
// Send changed-region (mode 3 bounding box) updates instead of full frames
// when the previous frame is known to be displayed. Requires firmware with
// the experimental bbox-incremental mode. Disabled: the firmware-side
// display buffer is not always the previous frame (occasionally two frames
// back, apparently display-driver buffer swapping), so partial updates
// composite onto stale content per-lens. Re-enable once the firmware
// guarantees the compositing base.
@JvmField val INCREMENTAL_FRAMES = true
// Send several tight rectangle deltas as one CFW mode-8 multi-segment message
// instead of a single bounding box, when the change splits into regions whose
// bounding box wastes bytes (distant edits, or a sparse box). Requires firmware
// with mode-8 multi-rect support; needs INCREMENTAL_FRAMES (same seeded shadow).
// Falls back to the single bounding box whenever multi-rect isn't smaller.
@JvmField val MULTI_RECT_FRAMES = true
// Ship text as on-glasses cached-glyph draws (CFW modes 12/14) instead of
// pixels, punching glyph ink out of the baked deltas (see TexturePlanner).
// Also gated at runtime on the firmware advertising texcache12 + texstr14.
@JvmField val TEXTURE_CACHE_FRAMES = true
}
@@ -0,0 +1,145 @@
package com.faceclaw.app
import kotlin.jvm.JvmField
import kotlin.jvm.JvmStatic
class FaceclawRingEventDecoder {
companion object {
@JvmStatic
fun decode(raw: ByteArray?): DirectRingEvent? {
if (((raw == null) || (raw.size == 0))) {
return null
}
var chargerGesture: DirectRingEvent? = decodeChargerGesture(raw)
if ((chargerGesture != null)) {
return chargerGesture
}
return decodeDirectGesture(raw)
}
private fun decodeChargerGesture(raw: ByteArray): DirectRingEvent? {
if ((raw.size != 11)) {
return null
}
if (
((((u8(raw[0]) != 0x00) || (u8(raw[1]) != 0x09)) || (u8(raw[2]) != 0x61)) ||
(u8(raw[3]) != 0x00))
) {
return null
}
var code: Int = u8(raw[4])
var param: Int = (u8(raw[5]) or (u8(raw[6]) shl 8))
var tick: Long =
((((u8(raw[7]) or (u8(raw[8]) shl 8)) or (u8(raw[9]) shl 16)) or
(u8(raw[10]) shl 24))
.toLong() and 0xffffffffL)
var detail: String =
"code=0x${code.toString(16).padStart(2, '0')}(${chargerCodeName(code)}) param=0x${param.toString(16).padStart(4, '0')} tick=$tick"
when (code) {
0x00 -> {
return event(BleProtocol.EVENT_RING_LONG_PRESS, "LONG_PRESS", detail)
}
0x01 -> {
return event(BleProtocol.EVENT_CLICK, "TAP", detail)
}
0x02 -> {
return event(BleProtocol.EVENT_DOUBLE_CLICK, "DOUBLE_TAP", detail)
}
0x04 -> {
return event(BleProtocol.EVENT_SCROLL_TOP, "SWIPE_UP", detail)
}
0x05 -> {
return event(BleProtocol.EVENT_SCROLL_BOTTOM, "SWIPE_DOWN", detail)
}
0x08 -> {
return event(
BleProtocol.EVENT_RING_LONG_PRESS_RELEASE,
"LONG_PRESS_RELEASE",
detail,
)
}
else -> {
return null
}
}
}
private fun decodeDirectGesture(raw: ByteArray): DirectRingEvent? {
if (((raw.size != 3) || (u8(raw[0]) != 0xff))) {
return null
}
var type: Int = u8(raw[1])
var param: Int = u8(raw[2])
var detail: String =
"type=0x${type.toString(16).padStart(2, '0')} param=0x${param.toString(16).padStart(2, '0')}"
if (((type == 0x03) && (param == 0x20))) {
return event(BleProtocol.EVENT_RING_LONG_PRESS, "HOLD", detail)
}
if (((type == 0x04) && (param == 0x01))) {
return event(BleProtocol.EVENT_CLICK, "TAP_SINGLE", detail)
}
if (((type == 0x04) && (param == 0x02))) {
return event(BleProtocol.EVENT_DOUBLE_CLICK, "TAP_DOUBLE", detail)
}
if ((type == 0x05)) {
if ((param <= 0x01)) {
return event(BleProtocol.EVENT_SCROLL_BOTTOM, "SWIPE_FORWARD", detail)
}
return event(BleProtocol.EVENT_SCROLL_TOP, "SWIPE_BACKWARD", detail)
}
return null
}
private fun event(eventType: Int, label: String, detail: String): DirectRingEvent {
var event: G2Event =
G2Event("sys-event", "", eventType, BleProtocol.EVENT_SOURCE_RING, 0)
return DirectRingEvent(event, label, detail)
}
private fun chargerCodeName(code: Int): String {
when (code) {
0x00 -> {
return "LONG_PRESS"
}
0x01 -> {
return "TAP"
}
0x02 -> {
return "DOUBLE_TAP"
}
0x04 -> {
return "SWIPE_UP"
}
0x05 -> {
return "SWIPE_DOWN"
}
0x08 -> {
return "LONG_PRESS_RELEASE"
}
else -> {
return "unknown"
}
}
}
private fun u8(value: Byte): Int {
return (value and 0xff)
}
}
constructor() {}
class DirectRingEvent {
@JvmField val event: G2Event
@JvmField val label: String
@JvmField val detail: String
constructor(event: G2Event, label: String, detail: String) {
this.event = event
this.label = label
this.detail = detail
}
}
}
@@ -0,0 +1,141 @@
package com.faceclaw.app
import kotlin.jvm.JvmField
import kotlin.jvm.JvmStatic
class G2Event {
companion object {
@JvmStatic
fun decode(frame: BleProtocol.ParsedFrame): G2Event? {
return decodePayload(frame.sid, BleProtocol.stripTrailingCrc(frame.pb))
}
/** Decode an already validated payload, without the envelope's trailing CRC. */
@JvmStatic
fun decodePayload(sid: Int, pb: ByteArray): G2Event? {
if ((sid == BleProtocol.SID_EVEN_AI)) {
var commandId: Int = BleProtocol.readVarintFieldValue(pb, 1, -1)
if ((commandId != BleProtocol.EVEN_AI_CMD_CTRL)) {
return null
}
var ctrl: ByteArray? = BleProtocol.readFieldBytes(pb, 3)
if ((ctrl == null)) {
return null
}
var status: Int = BleProtocol.readVarintFieldValue(ctrl, 1, -1)
if ((status < 0)) {
return null
}
return G2Event("even-ai", "", status, 0, 0)
}
var deviceEvent: ByteArray? = BleProtocol.readFieldBytes(pb, 13)
if ((deviceEvent == null)) {
return null
}
var listEvent: ByteArray? = BleProtocol.readFieldBytes(deviceEvent, 1)
if ((listEvent != null)) {
return G2Event(
"list-click",
BleProtocol.readStringFieldValue(listEvent, 2),
BleProtocol.readVarintFieldValue(listEvent, 5, BleProtocol.EVENT_CLICK),
0,
0,
)
}
var textEvent: ByteArray? = BleProtocol.readFieldBytes(deviceEvent, 2)
if ((textEvent != null)) {
return G2Event(
"text-click",
BleProtocol.readStringFieldValue(textEvent, 2),
BleProtocol.readVarintFieldValue(textEvent, 3, BleProtocol.EVENT_CLICK),
0,
0,
)
}
var sysEvent: ByteArray? = BleProtocol.readFieldBytes(deviceEvent, 3)
if ((sysEvent != null)) {
var eventType: Int =
BleProtocol.readVarintFieldValue(sysEvent, 1, BleProtocol.EVENT_CLICK)
var eventSource: Int = BleProtocol.readVarintFieldValue(sysEvent, 2, 0)
var exitReason: Int = BleProtocol.readVarintFieldValue(sysEvent, 4, 0)
var imu: ByteArray? = BleProtocol.readFieldBytes(sysEvent, 3)
if ((imu != null)) {
var x: Double = BleProtocol.readFloatFieldValue(imu, 1, 0f).toDouble()
var y: Double = BleProtocol.readFloatFieldValue(imu, 2, 0f).toDouble()
var z: Double = BleProtocol.readFloatFieldValue(imu, 3, 0f).toDouble()
return G2Event(
"sys-event",
"",
eventType,
eventSource,
exitReason,
true,
x,
y,
z,
)
}
return G2Event("sys-event", "", eventType, eventSource, exitReason)
}
return null
}
}
@JvmField val kind: String
@JvmField val containerName: String
@JvmField val eventType: Int
@JvmField val eventSource: Int
@JvmField val systemExitReasonCode: Int
@JvmField val hasImu: Boolean
@JvmField val imuX: Double
@JvmField val imuY: Double
@JvmField val imuZ: Double
constructor(
kind: String,
containerName: String,
eventType: Int,
eventSource: Int,
systemExitReasonCode: Int,
) : this(
kind,
containerName,
eventType,
eventSource,
systemExitReasonCode,
false,
0.0,
0.0,
0.0,
) {}
constructor(
kind: String,
containerName: String,
eventType: Int,
eventSource: Int,
systemExitReasonCode: Int,
hasImu: Boolean,
imuX: Double,
imuY: Double,
imuZ: Double,
) {
this.kind = kind
this.containerName = containerName
this.eventType = eventType
this.eventSource = eventSource
this.systemExitReasonCode = systemExitReasonCode
this.hasImu = hasImu
this.imuX = imuX
this.imuY = imuY
this.imuZ = imuZ
}
}
@@ -0,0 +1,292 @@
package com.faceclaw.app
class MessageBuilder {
companion object {
private const val ACK_TIMEOUT_MS: Int = 3_500
private const val HEARTBEAT_TIMEOUT_MS: Int = 1_500
}
private var magicPool: BleMagicPool
constructor(magicPool: BleMagicPool) {
this.magicPool = magicPool
}
fun prelude(): OutboundMessage {
return OutboundMessage(
"prelude",
"prelude",
BleProtocol.PRELUDE_ACK_SID,
BleProtocol.FLAG_REQUEST,
BleProtocol.PRELUDE_ACK_MAGIC,
BleProtocol.PRELUDE_F5872_PAYLOAD,
ACK_TIMEOUT_MS,
-1,
false,
)
}
/** The per-connection sid-0x80 security-auth request (one per arm). */
fun securityAuth(leftArm: Boolean): OutboundMessage {
var magic: Int = magicPool.allocate()
return OutboundMessage(
"security-auth",
("security auth " + (if (leftArm) "L" else "R")),
BleProtocol.SID_SECURITY_AUTH,
BleProtocol.FLAG_SECURITY_AUTH,
magic,
BleProtocol.buildAuthenticationRequest(magic),
ConnectionOptions.SECURITY_AUTH_SOFT_TIMEOUT_MS,
-1,
leftArm,
)
}
fun shutdown(exitMode: Int): OutboundMessage {
var magic: Int = magicPool.allocate()
return OutboundMessage(
"shutdown",
("shutdown mode=" + exitMode),
BleProtocol.SID_EVENHUB,
BleProtocol.FLAG_REQUEST,
magic,
BleProtocol.buildShutdown(magic, exitMode),
ACK_TIMEOUT_MS,
-1,
false,
)
}
fun imagePayload(
tile: BleProtocol.ImageTileOptions,
sessionId: Int,
payload: ByteArray,
label: String,
leftArm: Boolean,
): OutboundMessage {
return imagePayload("sound", tile, sessionId, payload, label, leftArm)
}
fun imagePayload(
kind: String,
tile: BleProtocol.ImageTileOptions,
sessionId: Int,
payload: ByteArray,
label: String,
leftArm: Boolean,
): OutboundMessage {
return customMessage(kind, payload, label, -1, leftArm)
}
fun customMessage(
kind: String,
payload: ByteArray?,
label: String,
tileIndex: Int,
leftArm: Boolean,
): OutboundMessage {
if (((payload == null) || (payload.size > CfwTransport.MAX_MESSAGE))) {
throw IllegalArgumentException("CFW message exceeds uint16 length")
}
var streamId: Int = magicPool.allocate()
return OutboundMessage(
kind,
label,
CfwTransport.SID,
0,
streamId,
payload,
CfwMessageWindow.ACK_TIMEOUT_MS,
tileIndex,
leftArm,
)
}
fun cfwCleanup(
tile: BleProtocol.ImageTileOptions,
sessionId: Int,
leftArm: Boolean,
): OutboundMessage {
var payload: ByteArray = byteArrayOf((BleProtocol.CFW_IMAGE_MODE_CLEANUP).toByte())
return imagePayload("cfw-cleanup", tile, sessionId, payload, "CFW cleanup", leftArm)
}
fun enableOrDisableMic(enable: Boolean): OutboundMessage {
var magic: Int = magicPool.allocate()
return OutboundMessage(
"audio-control",
(if (enable) "G2 mic enable" else "G2 mic disable"),
BleProtocol.SID_EVENHUB,
BleProtocol.FLAG_REQUEST,
magic,
BleProtocol.buildAudioControl(magic, enable),
ACK_TIMEOUT_MS,
-1,
false,
)
}
fun enableOrDisableImu(enable: Boolean, reportFrq: Int): OutboundMessage {
var magic: Int = magicPool.allocate()
return OutboundMessage(
"imu-control",
(if (enable) "IMU enable" else "IMU disable"),
BleProtocol.SID_EVENHUB,
BleProtocol.FLAG_REQUEST,
magic,
BleProtocol.buildImuControl(magic, enable, reportFrq),
ACK_TIMEOUT_MS,
-1,
false,
)
}
fun setBrightness(autoAdjust: Boolean, brightnessLevel: Int): OutboundMessage {
var magic: Int = magicPool.allocate()
return OutboundMessage(
"brightness-control",
(if (autoAdjust) "brightness auto" else ("brightness level=" + brightnessLevel)),
BleProtocol.SID_UI_SETTING,
BleProtocol.FLAG_REQUEST,
magic,
BleProtocol.buildSetBrightness(magic, autoAdjust, brightnessLevel),
ACK_TIMEOUT_MS,
-1,
false,
)
}
fun setWearDetection(enabled: Boolean): OutboundMessage {
var magic: Int = magicPool.allocate()
return OutboundMessage(
"wear-detection-control",
("wear detection " + (if (enabled) "enable" else "disable")),
BleProtocol.SID_UI_SETTING,
BleProtocol.FLAG_REQUEST,
magic,
BleProtocol.buildSetWearDetection(magic, enabled),
ACK_TIMEOUT_MS,
-1,
false,
)
}
fun createLayout(): OutboundMessage {
var magic: Int = magicPool.allocate()
return OutboundMessage(
"create-layout",
"create-layout",
BleProtocol.SID_EVENHUB,
BleProtocol.FLAG_REQUEST,
magic,
BleProtocol.buildCreateInputPage(magic),
ACK_TIMEOUT_MS,
-1,
false,
)
}
fun startupTextProbe(): OutboundMessage {
var magic: Int = magicPool.allocate()
return OutboundMessage(
"startup-text-probe",
"startup text probe",
BleProtocol.SID_EVENHUB,
BleProtocol.FLAG_REQUEST,
magic,
BleProtocol.buildDashboardTextUpgrade(magic),
ACK_TIMEOUT_MS,
-1,
false,
)
}
fun heartbeat(): OutboundMessage {
var magic: Int = magicPool.allocate()
return OutboundMessage(
"heartbeat",
"heartbeat",
BleProtocol.SID_EVENHUB,
BleProtocol.FLAG_REQUEST,
magic,
BleProtocol.buildHeartbeat(magic),
HEARTBEAT_TIMEOUT_MS,
-1,
false,
)
}
fun batteryQuery(): OutboundMessage {
var magic: Int = magicPool.allocate()
return OutboundMessage(
"battery",
"battery",
BleProtocol.SID_UI_SETTING,
BleProtocol.FLAG_REQUEST,
magic,
BleProtocol.buildSettingsQuery(magic),
ACK_TIMEOUT_MS,
-1,
false,
)
}
fun faceclawWakeControl(operation: Int, nonce: Int, leftArm: Boolean): OutboundMessage {
return OutboundMessage(
"wake-lease-control",
(((("wake lease op=" + operation) + " nonce=") + nonce) +
(if (leftArm) " L" else " R")),
BleProtocol.SID_UI_SETTING,
BleProtocol.FLAG_REQUEST,
0,
BleProtocol.buildFaceclawWakeControl(operation, nonce),
ACK_TIMEOUT_MS,
-1,
leftArm,
)
}
fun faceclawFramebufferControl(operation: Int, leftArm: Boolean): OutboundMessage {
return OutboundMessage(
"framebuffer-lease-control",
(("framebuffer lease op=" + operation) + (if (leftArm) " L" else " R")),
BleProtocol.SID_UI_SETTING,
BleProtocol.FLAG_REQUEST,
0,
BleProtocol.buildFaceclawWakeControl(operation, 0),
ACK_TIMEOUT_MS,
-1,
leftArm,
)
}
/** CFW mic_control write (settings field 103) targeted at one temple. */
fun faceclawMicControl(record: ByteArray, label: String, leftArm: Boolean): OutboundMessage {
return OutboundMessage(
"mic-control",
(("mic control " + label) + (if (leftArm) " L" else " R")),
BleProtocol.SID_UI_SETTING,
BleProtocol.FLAG_REQUEST,
0,
BleProtocol.buildFaceclawMicControl(record),
ACK_TIMEOUT_MS,
-1,
leftArm,
)
}
fun faceclawWearQuery(leftArm: Boolean): OutboundMessage {
return OutboundMessage(
"wear-query-control",
("wear state query" + (if (leftArm) " L" else " R")),
BleProtocol.SID_UI_SETTING,
BleProtocol.FLAG_REQUEST,
0,
BleProtocol.buildFaceclawWakeControl(BleProtocol.FACECLAW_WEAR_OP_QUERY, 0),
ACK_TIMEOUT_MS,
-1,
leftArm,
)
}
}
@@ -0,0 +1,127 @@
package com.faceclaw.app
/** Split/coalesced notifications and protobuf fragments, isolated per BLE link and sequence. */
class MessageReceiver {
class Message(
val sid: Int,
val flag: Int,
val payload: ByteArray,
val command: Int,
val magic: Int,
val packet: ByteArray? = null,
)
private class Partial(val total: Int, var seen: Long) {
val chunks = mutableListOf<ByteArray>()
var size = 0
}
private val lock = protocolPlatform().createLock()
private val streams = mutableMapOf<String, ByteArray>()
private val messages = mutableMapOf<String, Partial>()
fun clear() = lock.withLock {
streams.clear()
messages.clear()
}
fun receive(link: String, data: ByteArray, now: Long): List<Message> = lock.withLock {
messages.entries.removeAll { now - it.value.seen > 5000 }
val stream = (streams[link] ?: ByteArray(0)) + data
val output = mutableListOf<Message>()
var offset = 0
while (stream.size - offset >= 8) {
fun u8(index: Int) = stream[offset + index].toInt() and 255
if (u8(0) != 0xaa || u8(1) !in intArrayOf(0x21, 0x12)) {
offset++
continue
}
val length = u8(3)
if (stream.size - offset < length + 8) break
val count = u8(4)
val index = u8(5)
val sid = u8(6)
val flag = u8(7)
val key = "$link:${u8(2)}:$sid:$flag"
val packet = stream.copyOfRange(offset, offset + length + 8)
val chunk = packet.copyOfRange(8, packet.size)
offset += length + 8
if (sid == CfwTransport.SID) {
output.add(Message(sid, flag, chunk, -1, -1, packet))
continue
}
if (count == 0 || index == 0 || index > count) {
messages.remove(key)
continue
}
if (index == 1) messages[key] = Partial(count, now)
val partial = messages[key]
if (partial == null || partial.total != count || partial.chunks.size + 1 != index) {
messages.remove(key)
continue
}
partial.chunks.add(chunk)
partial.size += chunk.size
partial.seen = now
if (partial.size > 65536) {
messages.remove(key)
continue
}
if (index != count) continue
messages.remove(key)
val bytes = ByteSink(partial.size)
partial.chunks.forEach { bytes.write(it) }
val complete = bytes.toByteArray()
if (complete.size < 2) continue
val payload = complete.copyOf(complete.size - 2)
val checksum =
(complete[complete.size - 2].toInt() and 255) or
((complete.last().toInt() and 255) shl 8)
if (CfwTransport.crc(payload) != checksum || !validProtobuf(payload)) continue
output.add(
Message(
sid,
flag,
payload,
BleProtocol.readVarintFieldValue(payload, 1, -1),
BleProtocol.readVarintFieldValue(payload, 2, -1),
)
)
}
streams[link] = stream.copyOfRange(offset, stream.size)
output
}
private fun validProtobuf(data: ByteArray): Boolean {
var offset = 0
fun varint(): Int? {
var value = 0
for (i in 0..4) {
if (offset == data.size) return null
val byte = data[offset++].toInt() and 255
if (i == 4 && byte > 15) return null
value = value or ((byte and 127) shl (i * 7))
if (byte and 128 == 0) return value
}
return null
}
while (offset < data.size) {
val key = varint() ?: return false
if (key ushr 3 == 0) return false
if (key and 7 == 0) {
varint() ?: return false
continue
}
val length =
when (key and 7) {
1 -> 8
5 -> 4
2 -> varint() ?: return false
else -> return false
}
if (length < 0 || length > data.size - offset) return false
offset += length
}
return true
}
}
@@ -0,0 +1,68 @@
package com.faceclaw.app
import kotlin.jvm.JvmField
/** Mutable attempt state, owned by the communicator and guarded by its lock. */
class OutboundMessage(
@JvmField val kind: String?,
@JvmField val label: String?,
@JvmField val sid: Int,
@JvmField val flag: Int,
@JvmField var magic: Int,
message: ByteArray?,
@JvmField val ackTimeoutMs: Int,
@JvmField val tileIndex: Int,
@JvmField val isLeftArmMessage: Boolean,
) {
@JvmField val message: ByteArray = message?.copyOf() ?: ByteArray(0)
@JvmField
val cfwChecksum: Int = if (sid == CfwTransport.SID) CfwTransport.crc(this.message) else 0
@JvmField var cfwAckLenses = 0
@JvmField var cfwRetries = 0
@JvmField var cfwRetryPending = false
@JvmField var imageUpdateId = 0
@JvmField var imageMessageNumber = 0
@JvmField var imageMessageCount = 0
@JvmField var sentAtMs = 0L
@JvmField var writeStartedAtMs = 0L
@JvmField var ackDeadlineAtMs = 0L
@JvmField var ackPayload = ByteArray(0)
@JvmField var onSent: MessageCallback? = null
@JvmField var onAck: MessageCallback? = null
@JvmField var onTimeout: MessageCallback? = null
/** NACK names this attempt's sequence, even without a decoded size. */
fun acceptCfwAck(ack: CfwTransport.Ack?): Boolean {
if (
ack == null ||
sid != CfwTransport.SID ||
magic != ack.streamId ||
ack.messageId != 0 ||
(ack.lens != 1 && ack.lens != 2)
)
return false
if (ack.nack) {
cfwRetryPending = true
return false
}
if (message.size != ack.size || cfwChecksum != ack.checksum) return false
cfwAckLenses = cfwAckLenses or ack.lens
return !cfwRetryPending && cfwAckLenses == CfwTransport.BOTH
}
/** Replace an attempt atomically under the communicator lock. */
fun prepareCfwReplay(streamId: Int) {
magic = streamId
cfwRetries++
cfwAckLenses = 0
cfwRetryPending = false
ackDeadlineAtMs = 0
ackPayload = ByteArray(0)
}
fun setImageUpdatePosition(updateId: Int, messageNumber: Int, messageCount: Int) {
imageUpdateId = updateId
imageMessageNumber = messageNumber
imageMessageCount = messageCount
}
}
@@ -0,0 +1,46 @@
package com.faceclaw.app
/** OS services used by the protocol state machines; supplied by their owner. */
interface ProtocolPlatform {
/** Monotonic milliseconds since boot, including time spent asleep. */
fun elapsedRealtimeMs(): Long
fun createLock(): ProtocolLock
fun createDeflater(): ProtocolDeflater
}
/** Must be reentrant: transport encoding can reset its history under the lock. */
interface ProtocolLock {
fun lock()
fun unlock()
}
/** Persistent zlib stream. Each write completes a SYNC_FLUSH, not a new stream. */
interface ProtocolDeflater {
fun reset()
fun syncFlush(message: ByteArray): ByteArray
fun close()
}
internal inline fun <T> ProtocolLock.withLock(action: () -> T): T {
lock()
try {
return action()
} finally {
unlock()
}
}
enum class GattWriteMode {
WITH_RESPONSE,
WITHOUT_RESPONSE,
}
/** Keeps message callbacks independent of java.lang.Runnable. */
fun interface MessageCallback {
fun run()
}
@@ -0,0 +1,191 @@
package com.faceclaw.app
/**
* Phone-side model of the CFW's flat 256 KiB texture cache (zlib_glue mode 18). The firmware holds
* only bytes; every structural decision — where font tables and glyph images live, what is
* resident, when to start over — is made here. Layout per font: a 96-entry uint32 offset table
* (chars 32..127) whose entries are written lazily as each glyph is uploaded; glyph images are the
* GlyphAtlas cached bytes ([w][h][4bpp RLE]).
*
* Allocation is a bump pointer with reset-on-full: the realistic working set (a few fonts' tables
* plus ~hundreds of glyphs at tens of bytes each) is far below 256 KiB, so eviction sophistication
* buys nothing. After a reset the firmware cache still holds stale bytes; that is safe because a
* draw only ever references (font table entry, glyph image) pairs uploaded in the current
* generation — stale table entries are never named in a mode-20 string.
*
* Upload entries accumulate as complete [offset u32][len u16][data] records and drain into one or
* more standalone mode-18 payloads, each independently valid, enqueued ahead of the image message
* that references them (the transport is FIFO, so no ack round-trip is needed before use).
*
* Not thread-safe by itself: the communicator calls it under its own lock.
*/
class TextureCacheState {
companion object {
const val CACHE_SIZE: Int = 262144
private val FONT_TABLE_BYTES: Int = (96 * 4)
}
private var allocEnd: Int = 0
private val fontTableOffsets: MutableMap<Int, Int> = HashMap()
private val glyphOffsets: MutableMap<Long, Int> = HashMap()
private val imageOffsets: MutableMap<Int, Int> = HashMap()
private val pendingEntries: MutableList<ByteArray> = ArrayList()
private var pendingBytes: Int = 0
private var generation: Int = 0
/**
* Forget everything: the on-glasses cache is gone (session teardown, lease loss) or no longer
* trustworthy (upload timeout), or we ran out of space. Glyphs and images re-upload lazily on
* next use.
*/
fun reset(): Unit {
allocEnd = 0
fontTableOffsets.clear()
glyphOffsets.clear()
imageOffsets.clear()
pendingEntries.clear()
pendingBytes = 0
generation++
}
/** Bumped by every reset; lets a planner detect a mid-plan reset. */
fun generation(): Int {
return generation
}
/**
* Bytes currently allocated out of CACHE_SIZE, for logging/tuning (a sudden drop in the frame
* log means a reset-on-full re-upload cycle — the signal that the 256 KiB budget is being
* outgrown).
*/
fun usedBytes(): Int {
return allocEnd
}
/**
* The font's 96-entry table offset in the cache, or -1 when it exists but this glyph can't fit.
*/
fun fontTableOffset(fontId: Int): Int {
var existing: Int? = fontTableOffsets.get(fontId)
return (if ((existing == null)) -1 else existing)
}
/**
* Ensure a glyph image (and its font table slot) is resident, queuing the mode-18 upload
* entries for anything newly placed. Returns the glyph's image offset, or -1 when the cache is
* full (caller resets and retries) or the encoding is outside the mode-20 table (32..127).
*/
fun ensureGlyph(fontId: Int, encoding: Int, glyph: GlyphAtlas.Glyph?): Int {
if ((((glyph == null) || (encoding < 32)) || (encoding > 127))) {
return -1
}
var key: Long = (((fontId).toLong() shl 32) or (encoding.toLong() and 0xffffffffL))
var existing: Int? = glyphOffsets.get(key)
if ((existing != null)) {
return existing
}
var table: Int? = fontTableOffsets.get(fontId)
var need: Int = (glyph.cachedBytes.size + (if ((table == null)) FONT_TABLE_BYTES else 0))
if (((allocEnd + need) > CACHE_SIZE)) {
return -1
}
if ((table == null)) {
table = allocEnd
allocEnd += FONT_TABLE_BYTES
fontTableOffsets.put(fontId, table)
}
var offset: Int = allocEnd
allocEnd += glyph.cachedBytes.size
glyphOffsets.put(key, offset)
addEntry(offset, glyph.cachedBytes)
addEntry(
(table + (4 * (encoding - 32))),
byteArrayOf(
(offset).toByte(),
((offset shr 8)).toByte(),
((offset shr 16)).toByte(),
((offset shr 24)).toByte(),
),
)
return offset
}
/**
* Ensure an ImageAtlas entry is resident (mode-19 draws reference it by raw cache offset; no
* table involved). Returns the image offset, or -1 when the cache is full (caller resets and
* retries).
*/
fun ensureImage(imageId: Int, image: ImageAtlas.Entry?): Int {
if ((image == null)) {
return -1
}
var existing: Int? = imageOffsets.get(imageId)
if ((existing != null)) {
return existing
}
if (((allocEnd + image.cachedBytes.size) > CACHE_SIZE)) {
return -1
}
var offset: Int = allocEnd
allocEnd += image.cachedBytes.size
imageOffsets.put(imageId, offset)
addEntry(offset, image.cachedBytes)
return offset
}
private fun addEntry(offset: Int, data: ByteArray): Unit {
var entry: ByteArray = ByteArray((6 + data.size))
entry[0] = ((offset and 0xff)).toByte()
entry[1] = (((offset shr 8) and 0xff)).toByte()
entry[2] = (((offset shr 16) and 0xff)).toByte()
entry[3] = (((offset shr 24) and 0xff)).toByte()
entry[4] = ((data.size and 0xff)).toByte()
entry[5] = (((data.size shr 8) and 0xff)).toByte()
data.copyInto(entry, 6, 0, 0 + data.size)
pendingEntries.add(entry)
pendingBytes += entry.size
}
fun hasPendingUploads(): Boolean {
return !pendingEntries.isEmpty()
}
/** Total queued upload bytes (entry framing included), for logging. */
fun pendingUploadBytes(): Int {
return pendingBytes
}
/**
* Drain the queued entries into complete mode-18 payloads ([18][entries]), each at most
* maxPayloadBytes so a payload fits one BLE image message without fragmentation. A single entry
* larger than the cap still gets its own payload (the transport can fragment it).
*/
fun drainUploadPayloads(maxPayloadBytes: Int): MutableList<ByteArray> {
var payloads: MutableList<ByteArray> = ArrayList()
var current: ByteSink? = null
for (entry in pendingEntries) {
if (((current != null) && ((current.size() + entry.size) > maxPayloadBytes))) {
payloads.add(current.toByteArray())
current = null
}
if ((current == null)) {
current = ByteSink(minOf(maxPayloadBytes, (4 + entry.size)))
current.write(18)
}
current.write(entry, 0, entry.size)
}
if ((current != null)) {
payloads.add(current.toByteArray())
}
pendingEntries.clear()
pendingBytes = 0
return payloads
}
}
@@ -0,0 +1,987 @@
package com.faceclaw.app
import kotlin.jvm.JvmField
import kotlin.jvm.JvmStatic
/**
* Plans a screen update that ships text and icons as on-glasses cached draws instead of pixels (CFW
* modes 18/19/20; see g2flash/patches/zlib_glue.c and texture_cache.c).
*
* Approach: dirty rects come from comparing the fully composited old and new frames, exactly as the
* plain incremental path does. For each deferred draw (glyph or image) whose pixels intersect a
* sent rect, check that every pixel the on-glasses draw would write lands correct — i.e. the new
* composite's 4bpp value at each in-panel nonzero-source pixel equals what the draw produces (this
* one check subsumes plane occlusion, surface occlusion/clipping, and draw-on-draw overlap). A draw
* that passes and is (or can be made) resident in the texture cache is replayed on-glasses: its
* written pixels are punched to 0 in the delta rect content — which is what makes text and icons
* nearly free to compress — and re-emitted as a mode-20 string or mode-19 image sub-message in the
* same atomic mode-8 batch, so the shadow after apply equals the full composite exactly. Everything
* else stays baked.
*
* Fallback shape: when nothing qualifies, plan() returns null and the caller uses the plain
* single-bbox/multi-rect/full-frame paths unchanged.
*/
class TexturePlanner {
companion object {
/** Cap on mode-20 string sub-messages (mode-8 count is a u8). */
private const val MAX_GLYPH_RUNS: Int = 180
/**
* Soft cap on cached-glyph selection per update, before run grouping. Sized above a
* full-screen 6x12 terminal repaint (~4,240 cells) so dense screens draw everything; the
* run budget is the real bound.
*/
private const val MAX_SELECTED_GLYPHS: Int = 4600
/** Cap on mode-19 image sub-messages per update. */
private const val MAX_IMAGE_DRAWS: Int = 60
/** Cap on mode-15 builtin-font string sub-messages per update. */
private const val MAX_FWTEXT_RUNS: Int = 80
/** Mode-18 payload size that fits one BLE image message comfortably. */
private const val UPLOAD_PAYLOAD_MAX: Int = 3600
/** Max x-adjust control bytes between two glyphs before starting a new run. */
private const val MAX_ADJUST_BYTES: Int = 4
/** Options: identity LUT (top 15) + transparent, for mode-19 image draws. */
private const val IMAGE_DRAW_OPTIONS: Int = 0x1f
/**
* Plan a cached-draw update. previous is the delta base (the frame the shadow currently
* holds), or null/mismatched for a full-frame keyframe. Returns null when the plain paths
* should run instead (no replayable draws, or identical frames).
*/
@JvmStatic
fun plan(
previous: ByteArray?,
next: ByteArray?,
width: Int,
height: Int,
draws: Array<SurfaceCompositor.ScreenDraw>?,
cache: TextureCacheState,
fidStart: Int,
allowMultiRect: Boolean,
maxRects: Int,
): Result? {
if (
(((((next == null) || (draws == null)) || (draws.size == 0)) || (width <= 0)) ||
(height <= 0))
) {
return null
}
var stride: Int = ((width + 1) shr 1)
if ((next.size != (stride * height))) {
return null
}
var planStartedAtMs: Long = protocolPlatform().elapsedRealtimeMs()
var fullFrame: Boolean = ((previous == null) || (previous.size != next.size))
var rects: MutableList<IntArray>
if (!fullFrame) {
var box: IntArray? =
BleImageOptimizer.computeChangedBox(previous, next, width, height)
if ((box == null)) {
return null
}
if (((box[2] >= width) && (box[3] >= height))) {
fullFrame = true
rects = mutableListOf(intArrayOf(0, 0, width, height))
} else {
var split: MutableList<IntArray>? =
(if (allowMultiRect)
BleImageOptimizer.computeChangedRects(
previous,
next,
width,
height,
maxRects,
)
else null)
rects = (if ((split != null)) split else mutableListOf(box))
}
} else {
rects = mutableListOf(intArrayOf(0, 0, width, height))
}
var matchStartedAtMs: Long = protocolPlatform().elapsedRealtimeMs()
var selected: MutableList<Selected> = ArrayList()
var bakedCandidates: Int = 0
var selectedImages: Int = 0
var fwSubs: MutableList<ByteArray> = ArrayList()
var fwPunches: MutableList<FwPunch> = ArrayList()
var fwGlyphCount: Int = 0
var fwBaked: Int = 0
for (draw in draws) {
if ((draw.kind != SurfaceCompositor.ScreenDraw.KIND_FWTEXT)) {
continue
}
fwBaked += planFwRun(draw, rects, next, stride, width, height, fwSubs, fwPunches)
}
fwGlyphCount = fwPunches.size
bakedCandidates += fwBaked
for (draw in draws) {
if ((draw.kind == SurfaceCompositor.ScreenDraw.KIND_GLYPH)) {
var atlas: GlyphAtlas.Glyph? = GlyphAtlas.get(draw.fontId, draw.encoding)
if ((atlas == null)) {
continue
}
var gx: Int = (draw.x + atlas.bbxX)
var inkTop: Int = (draw.y + atlas.inkTop)
if (!intersectsAny(rects, gx, inkTop, atlas.width, atlas.inkHeight)) {
continue
}
if (
((((((draw.encoding < 32) || (draw.encoding > 127)) || (gx < 0)) ||
(draw.y < 0)) || (gx > 0xffff)) || (draw.y > 0xffff))
) {
bakedCandidates++
continue
}
var top: Int = BmpUtil.nibbleForGray(draw.value)
if (
!glyphMatchesComposite(next, stride, width, height, atlas, gx, draw.y, top)
) {
bakedCandidates++
continue
}
if ((selected.size >= MAX_SELECTED_GLYPHS)) {
bakedCandidates++
continue
}
selected.add(Selected(draw, atlas, null, gx, top))
} else {
var atlas: ImageAtlas.Entry? = ImageAtlas.get(draw.imageId)
if ((atlas == null)) {
continue
}
if (!intersectsAny(rects, draw.x, draw.y, atlas.width, atlas.height)) {
continue
}
if (
(((((draw.x < 0) || (draw.y < 0)) || (draw.x > 0xffff)) ||
(draw.y > 0xffff)) || (selectedImages >= MAX_IMAGE_DRAWS))
) {
bakedCandidates++
continue
}
if (
!imageMatchesComposite(next, stride, width, height, atlas, draw.x, draw.y)
) {
bakedCandidates++
continue
}
selected.add(Selected(draw, null, atlas, draw.x, 0))
selectedImages++
}
}
if ((selected.isEmpty() && fwSubs.isEmpty())) {
return null
}
var cacheStartedAtMs: Long = protocolPlatform().elapsedRealtimeMs()
var drawable: MutableList<Selected> =
(if (selected.isEmpty()) mutableListOf() else ensureResident(cache, selected))
bakedCandidates += (selected.size - drawable.size)
var glyphDrawable: MutableList<Selected> = ArrayList()
var imageDrawable: MutableList<Selected> = ArrayList()
for (sel in drawable) {
if ((sel.glyphAtlas != null)) {
glyphDrawable.add(sel)
} else {
imageDrawable.add(sel)
}
}
var runs: MutableList<ByteArray> = ArrayList()
var drawnGlyphs: MutableList<Selected> = buildRuns(cache, glyphDrawable, runs)
bakedCandidates += (glyphDrawable.size - drawnGlyphs.size)
var drawn: MutableList<Selected> = ArrayList(drawnGlyphs)
drawn.addAll(imageDrawable)
if (
((drawn.isEmpty() && fwSubs.isEmpty()) ||
((((rects.size + runs.size) + imageDrawable.size) + fwSubs.size) > 255))
) {
if (!selected.isEmpty()) {
cache.reset()
}
return null
}
var punchStartedAtMs: Long = protocolPlatform().elapsedRealtimeMs()
var punched: ByteArray = next.copyOf()
for (sel in drawn) {
punch(punched, stride, width, height, sel)
}
for (fw in fwPunches) {
punchFw(punched, stride, width, height, fw)
}
var encodeStartedAtMs: Long = protocolPlatform().elapsedRealtimeMs()
var subs: MutableList<ByteArray> = ArrayList()
var fid: Int = fidStart
if (fullFrame) {
subs.add(BleImageOptimizer.maybeCompress(punched, width, height))
} else {
for (r in rects) {
subs.add(
BleImageOptimizer.encodeMode3Rect(
punched,
stride,
r[0],
r[1],
r[2],
r[3],
fid,
)
)
fid = (if ((fid >= 0xfffe)) 1 else (fid + 1))
}
}
for (sel in imageDrawable) {
subs.add(encodeImageDraw(cache, sel))
}
subs.addAll(runs)
subs.addAll(fwSubs)
var payload: ByteArray = assembleMode8(subs)
var uploadBytes: Int = cache.pendingUploadBytes()
var uploads: MutableList<ByteArray> = cache.drainUploadPayloads(UPLOAD_PAYLOAD_MAX)
var result: Result =
Result(
payload,
uploads,
fid,
rects.size,
drawnGlyphs.size,
runs.size,
imageDrawable.size,
fwGlyphCount,
fwSubs.size,
bakedCandidates,
uploadBytes,
fullFrame,
)
var doneAtMs: Long = protocolPlatform().elapsedRealtimeMs()
result.rectsMs = ((matchStartedAtMs - planStartedAtMs)).toInt()
result.matchMs = ((cacheStartedAtMs - matchStartedAtMs)).toInt()
result.cacheMs = ((punchStartedAtMs - cacheStartedAtMs)).toInt()
result.punchMs = ((encodeStartedAtMs - punchStartedAtMs)).toInt()
result.encodeMs = ((doneAtMs - encodeStartedAtMs)).toInt()
return result
}
/**
* Plan one firmware-font run (mode 15): classify every member, and when any correct ink
* member intersects a sent rect, emit sub-messages over the maximal correct stretches. A
* stretch never spans an incorrect member — the sub-message boundary restarts the
* firmware's advance walk at an explicit pen x, so occluded or unknown glyphs simply stay
* baked without disturbing their neighbors' positions. Emitted ink members are queued for
* punching. Returns how many ink members stay baked.
*/
private fun planFwRun(
draw: SurfaceCompositor.ScreenDraw,
rects: MutableList<IntArray>,
next: ByteArray,
stride: Int,
width: Int,
height: Int,
outSubs: MutableList<ByteArray>,
outPunches: MutableList<FwPunch>,
): Int {
var n: Int = draw.fwCps!!.size
if ((((n == 0) || (draw.y < 0)) || (draw.y > 0xffff))) {
return countInk(draw)
}
var top: Int = BmpUtil.nibbleForGray(draw.value)
var ok: BooleanArray = BooleanArray(n)
var entries: Array<FwGlyphAtlas.Entry?> = arrayOfNulls<FwGlyphAtlas.Entry>(n)
var relevant: Boolean = false
var baked: Int = 0
run {
var i: Int = 0
while ((i < n)) {
if (!draw.fwInk!![i]) {
ok[i] = true
i++
continue
}
var entry: FwGlyphAtlas.Entry? = FwGlyphAtlas.get(draw.fwCps!![i])
if ((entry == null)) {
baked++
i++
continue
}
var gx: Int = ((draw.x + draw.fwDx!![i]) + entry.ofsX)
var gy: Int = (draw.y + entry.inkTop)
if (!fwGlyphMatchesComposite(next, stride, width, height, entry, gx, gy, top)) {
baked++
i++
continue
}
ok[i] = true
entries[i] = entry
if (intersectsAny(rects, gx, gy, entry.boxW, entry.boxH)) {
relevant = true
}
i++
}
}
if (!relevant) {
return 0
}
var i: Int = 0
while (((i < n) && (outSubs.size < MAX_FWTEXT_RUNS))) {
while (((i < n) && !(ok[i] && draw.fwInk!![i]))) {
i++
}
if ((i >= n)) {
break
}
var startX: Int = (draw.x + draw.fwDx!![i])
if (((startX < 0) || (startX > 0xffff))) {
baked++
i++
continue
}
var j: Int = i
var lastInk: Int = -1
var bytesToLastInk: Int = 0
var bytesSoFar: Int = 0
while (((j < n) && ok[j])) {
var encodedLength: Int = utf8Length(draw.fwCps!![j])
if (((bytesSoFar + encodedLength) > 255)) {
break
}
bytesSoFar += encodedLength
if (draw.fwInk!![j]) {
lastInk = j
bytesToLastInk = bytesSoFar
}
j++
}
if ((lastInk < i)) {
baked++
i++
continue
}
var sub: ByteArray = ByteArray((7 + bytesToLastInk))
sub[0] = 15
sub[1] = ((startX and 0xff)).toByte()
sub[2] = (((startX shr 8) and 0xff)).toByte()
sub[3] = ((draw.y and 0xff)).toByte()
sub[4] = (((draw.y shr 8) and 0xff)).toByte()
sub[5] = ((top or 0x10)).toByte()
sub[6] = (bytesToLastInk).toByte()
var pos: Int = 7
run {
var k: Int = i
while ((k <= lastInk)) {
var encoded: ByteArray = codePointUtf8(draw.fwCps!![k])
encoded.copyInto(sub, pos, 0, 0 + encoded.size)
pos += encoded.size
k++
}
}
outSubs.add(sub)
run {
var k: Int = i
while ((k <= lastInk)) {
if (draw.fwInk!![k]) {
outPunches.add(
FwPunch(
entries[k]!!,
((draw.x + draw.fwDx!![k]) + entries[k]!!.ofsX),
(draw.y + entries[k]!!.inkTop),
)
)
}
k++
}
}
i = (lastInk + 1)
}
return baked
}
private fun utf8Length(cp: Int): Int {
if ((cp < 0x80)) {
return 1
}
if ((cp < 0x800)) {
return 2
}
if ((cp < 0x10000)) {
return 3
}
return 4
}
private fun countInk(draw: SurfaceCompositor.ScreenDraw): Int {
var count: Int = 0
for (ink in draw.fwInk!!) {
if (ink) {
count++
}
}
return count
}
/**
* Whether every in-panel nonzero pixel of the builtin-font glyph lands correct: the
* composite's 4bpp value must equal the firmware's LUT output src*top/15 (integer division
* — a general LUT check, since these glyphs are anti-aliased).
*/
private fun fwGlyphMatchesComposite(
packed: ByteArray,
stride: Int,
width: Int,
height: Int,
entry: FwGlyphAtlas.Entry,
gx: Int,
gy: Int,
top: Int,
): Boolean {
run {
var row: Int = 0
while ((row < entry.boxH)) {
var y: Int = (gy + row)
if (((y < 0) || (y >= height))) {
row++
continue
}
run {
var col: Int = 0
while ((col < entry.boxW)) {
var source: Int = entry.nibbleAt(col, row)
if ((source == 0)) {
col++
continue
}
var x: Int = (gx + col)
if (((x < 0) || (x >= width))) {
col++
continue
}
if ((nibbleAt(packed, stride, x, y) != ((source * top) / 15))) {
return false
}
col++
}
}
row++
}
}
return true
}
private fun punchFw(
packed: ByteArray,
stride: Int,
width: Int,
height: Int,
fw: FwPunch,
): Unit {
run {
var row: Int = 0
while ((row < fw.entry.boxH)) {
var y: Int = (fw.gy + row)
if (((y < 0) || (y >= height))) {
row++
continue
}
run {
var col: Int = 0
while ((col < fw.entry.boxW)) {
if ((fw.entry.nibbleAt(col, row) == 0)) {
col++
continue
}
punchPixel(packed, stride, width, (fw.gx + col), y)
col++
}
}
row++
}
}
}
private fun intersectsAny(
rects: MutableList<IntArray>,
x: Int,
y: Int,
w: Int,
h: Int,
): Boolean {
if (((w <= 0) || (h <= 0))) {
return false
}
for (r in rects) {
if (
((((x < (r[0] + r[2])) && ((x + w) > r[0])) && (y < (r[1] + r[3]))) &&
((y + h) > r[1]))
) {
return true
}
}
return false
}
/**
* Whether every in-panel ink pixel of the glyph lands correct in the packed new frame: the
* composite's 4bpp value must equal the firmware's LUT output source*top/15 (integer). For
* 1bpp glyphs (source 15) this is exactly `== top`; AA glyphs get the general check. Ink
* outside the panel is clipped by the firmware and irrelevant.
*/
private fun glyphMatchesComposite(
packed: ByteArray,
stride: Int,
width: Int,
height: Int,
atlas: GlyphAtlas.Glyph,
gx: Int,
lineY: Int,
top: Int,
): Boolean {
run {
var row: Int = atlas.inkTop
while ((row < (atlas.inkTop + atlas.inkHeight))) {
var y: Int = (lineY + row)
if (((y < 0) || (y >= height))) {
row++
continue
}
run {
var col: Int = 0
while ((col < atlas.width)) {
var source: Int = atlas.nibbleAt(col, row)
if ((source == 0)) {
col++
continue
}
var x: Int = (gx + col)
if (((x < 0) || (x >= width))) {
col++
continue
}
if ((nibbleAt(packed, stride, x, y) != ((source * top) / 15))) {
return false
}
col++
}
}
row++
}
}
return true
}
/**
* Whether every in-panel nonzero pixel of the cached image equals the packed new frame —
* the mode-19 transparent draw writes exactly those.
*/
private fun imageMatchesComposite(
packed: ByteArray,
stride: Int,
width: Int,
height: Int,
atlas: ImageAtlas.Entry,
left: Int,
top: Int,
): Boolean {
run {
var row: Int = 0
while ((row < atlas.height)) {
var y: Int = (top + row)
if (((y < 0) || (y >= height))) {
row++
continue
}
run {
var col: Int = 0
while ((col < atlas.width)) {
var source: Int = atlas.nibbleAt(col, row)
if ((source == 0)) {
col++
continue
}
var x: Int = (left + col)
if (((x < 0) || (x >= width))) {
col++
continue
}
if ((nibbleAt(packed, stride, x, y) != source)) {
return false
}
col++
}
}
row++
}
}
return true
}
private fun nibbleAt(packed: ByteArray, stride: Int, x: Int, y: Int): Int {
var b: Int = (packed[((y * stride) + (x shr 1))] and 0xff)
return (if (((x and 1) != 0)) (b and 0x0f) else (b shr 4))
}
private fun punch(
packed: ByteArray,
stride: Int,
width: Int,
height: Int,
sel: Selected,
): Unit {
if ((sel.glyphAtlas != null)) {
var atlas: GlyphAtlas.Glyph = sel.glyphAtlas
run {
var row: Int = atlas.inkTop
while ((row < (atlas.inkTop + atlas.inkHeight))) {
var y: Int = (sel.draw.y + row)
if (((y < 0) || (y >= height))) {
row++
continue
}
run {
var col: Int = 0
while ((col < atlas.width)) {
if (!atlas.inkAt(col, row)) {
col++
continue
}
punchPixel(packed, stride, width, (sel.gx + col), y)
col++
}
}
row++
}
}
} else {
var atlas: ImageAtlas.Entry = sel.imageAtlas!!
run {
var row: Int = 0
while ((row < atlas.height)) {
var y: Int = (sel.draw.y + row)
if (((y < 0) || (y >= height))) {
row++
continue
}
run {
var col: Int = 0
while ((col < atlas.width)) {
if ((atlas.nibbleAt(col, row) == 0)) {
col++
continue
}
punchPixel(packed, stride, width, (sel.gx + col), y)
col++
}
}
row++
}
}
}
}
private fun punchPixel(packed: ByteArray, stride: Int, width: Int, x: Int, y: Int): Unit {
if (((x < 0) || (x >= width))) {
return
}
var index: Int = ((y * stride) + (x shr 1))
if (((x and 1) != 0)) {
packed[index] = ((packed[index] and 0xf0)).toByte()
} else {
packed[index] = ((packed[index] and 0x0f)).toByte()
}
}
/**
* Ensure residency for all of `selected`, resetting the cache once if the allocator fills
* mid-plan (a reset invalidates offsets handed out earlier in the same pass, so the whole
* pass reruns). Returns the draws that are resident afterwards.
*/
private fun ensureResident(
cache: TextureCacheState,
selected: MutableList<Selected>,
): MutableList<Selected> {
run {
var attempt: Int = 0
while ((attempt < 2)) {
var resident: MutableList<Selected> = ArrayList(selected.size)
var full: Boolean = false
for (sel in selected) {
var offset: Int =
(if ((sel.glyphAtlas != null))
cache.ensureGlyph(
sel.draw.fontId,
sel.draw.encoding,
sel.glyphAtlas,
)
else cache.ensureImage(sel.draw.imageId, sel.imageAtlas))
if ((offset >= 0)) {
resident.add(sel)
} else {
full = true
}
}
if ((!full || (attempt == 1))) {
return resident
}
cache.reset()
attempt++
}
}
return mutableListOf()
}
/** Mode-19 cached-image draw: [19][offset u32][x u16][y u16][options u8]. */
@JvmStatic
private fun encodeImageDraw(cache: TextureCacheState, sel: Selected): ByteArray {
var offset: Int = cache.ensureImage(sel.draw.imageId, sel.imageAtlas)
var sub: ByteArray = ByteArray(10)
sub[0] = 19
sub[1] = ((offset and 0xff)).toByte()
sub[2] = (((offset shr 8) and 0xff)).toByte()
sub[3] = (((offset shr 16) and 0xff)).toByte()
sub[4] = (((offset shr 24) and 0xff)).toByte()
sub[5] = ((sel.draw.x and 0xff)).toByte()
sub[6] = (((sel.draw.x shr 8) and 0xff)).toByte()
sub[7] = ((sel.draw.y and 0xff)).toByte()
sub[8] = (((sel.draw.y shr 8) and 0xff)).toByte()
sub[9] = (IMAGE_DRAW_OPTIONS).toByte()
return sub
}
/**
* Group selected glyphs into mode-20 string sub-messages: [20][fontTable u32][x u16][y
* u16][options u8][strlen u8][string] One run per (font, line y, top color) span; within a
* run, control bytes 1..31 adjust x by -10..+20 to hit each glyph's exact position, and
* each glyph advances x by its cached width. Order across runs is free: every drawn glyph's
* ink equals the composite, so overlaps write equal values. Returns the glyphs actually
* emitted (run budget can drop stragglers).
*/
@JvmStatic
private fun buildRuns(
cache: TextureCacheState,
drawable: MutableList<Selected>,
outRuns: MutableList<ByteArray>,
): MutableList<Selected> {
var sorted: MutableList<Selected> = ArrayList(drawable)
sorted.sortWith(
compareBy<Selected> { it.draw.fontId }
.thenBy { it.draw.y }
.thenBy { it.top }
.thenBy { it.gx }
)
var drawn: MutableList<Selected> = ArrayList(sorted.size)
var i: Int = 0
while (((i < sorted.size) && (outRuns.size < MAX_GLYPH_RUNS))) {
var first: Selected = sorted.get(i)
var fontTable: Int = cache.fontTableOffset(first.draw.fontId)
if ((fontTable < 0)) {
i++
continue
}
var string: ByteSink = ByteSink()
var runGlyphs: MutableList<Selected> = ArrayList()
var cursor: Int = first.gx
var j: Int = i
while ((j < sorted.size)) {
var sel: Selected = sorted.get(j)
if (
(((sel.draw.fontId != first.draw.fontId) || (sel.draw.y != first.draw.y)) ||
(sel.top != first.top))
) {
break
}
var delta: Int = (sel.gx - cursor)
var adjustBytes: Int = adjustByteCount(delta)
if (((adjustBytes > MAX_ADJUST_BYTES) && (j > i))) {
break
}
if ((((string.size() + adjustBytes) + 1) > 255)) {
break
}
emitAdjust(string, delta)
string.write(sel.draw.encoding)
runGlyphs.add(sel)
cursor = (sel.gx + sel.glyphAtlas!!.width)
j++
}
if (!runGlyphs.isEmpty()) {
var stringBytes: ByteArray = string.toByteArray()
var run: ByteArray = ByteArray((11 + stringBytes.size))
run[0] = 20
run[1] = ((fontTable and 0xff)).toByte()
run[2] = (((fontTable shr 8) and 0xff)).toByte()
run[3] = (((fontTable shr 16) and 0xff)).toByte()
run[4] = (((fontTable shr 24) and 0xff)).toByte()
run[5] = ((first.gx and 0xff)).toByte()
run[6] = (((first.gx shr 8) and 0xff)).toByte()
run[7] = ((first.draw.y and 0xff)).toByte()
run[8] = (((first.draw.y shr 8) and 0xff)).toByte()
run[9] = ((first.top or 0x10)).toByte()
run[10] = (stringBytes.size).toByte()
stringBytes.copyInto(run, 11, 0, 0 + stringBytes.size)
outRuns.add(run)
drawn.addAll(runGlyphs)
i = j
} else {
i++
}
}
return drawn
}
/** How many 1..31 control bytes are needed to move the cursor by delta. */
private fun adjustByteCount(delta: Int): Int {
if ((delta == 0)) {
return 0
}
return (if ((delta > 0)) ((delta + 19) / 20) else ((-delta + 9) / 10))
}
private fun emitAdjust(out: ByteSink, delta: Int): Unit {
var delta = delta
while ((delta != 0)) {
var step: Int = (if ((delta > 0)) minOf(delta, 20) else maxOf(delta, -10))
out.write((step + 11))
delta -= step
}
}
/** [8][count]([len u16][sub])* — one atomic present for deltas + draws. */
private fun assembleMode8(subs: MutableList<ByteArray>): ByteArray {
var total: Int = 2
for (sub in subs) {
total += (2 + sub.size)
}
var out: ByteArray = ByteArray(total)
out[0] = 8
out[1] = (subs.size).toByte()
var pos: Int = 2
for (sub in subs) {
out[pos] = ((sub.size and 0xff)).toByte()
out[(pos + 1)] = (((sub.size shr 8) and 0xff)).toByte()
pos += 2
sub.copyInto(out, pos, 0, 0 + sub.size)
pos += sub.size
}
return out
}
}
constructor() {}
class Result {
/** Complete image payload: a mode-8 batch of rect deltas + cached draws. */
@JvmField val payload: ByteArray
/** Mode-18 upload payloads to enqueue BEFORE the image message. */
@JvmField val uploads: MutableList<ByteArray>
/** Next mode-3 frame id (deltas consumed some). */
@JvmField val nextFid: Int
@JvmField val rectCount: Int
@JvmField val drawnGlyphs: Int
@JvmField val runCount: Int
@JvmField val drawnImages: Int
@JvmField val fwGlyphs: Int
@JvmField val fwRuns: Int
@JvmField val bakedCandidates: Int
@JvmField val uploadBytes: Int
@JvmField val fullFrame: Boolean
/**
* Where the planning time went, in ms. This runs on the send thread, directly in the
* input-to-display path, so the split is reported in the frame log: rects = finding the
* changed region, match = checking each draw against the composite, cache =
* residency/uploads plus run building, punch = clearing replayed draws out of a copy of the
* frame, encode = compressing the leftover pixels.
*/
@JvmField var rectsMs: Int = 0
@JvmField var matchMs: Int = 0
@JvmField var cacheMs: Int = 0
@JvmField var punchMs: Int = 0
@JvmField var encodeMs: Int = 0
constructor(
payload: ByteArray,
uploads: MutableList<ByteArray>,
nextFid: Int,
rectCount: Int,
drawnGlyphs: Int,
runCount: Int,
drawnImages: Int,
fwGlyphs: Int,
fwRuns: Int,
bakedCandidates: Int,
uploadBytes: Int,
fullFrame: Boolean,
) {
this.payload = payload
this.uploads = uploads
this.nextFid = nextFid
this.rectCount = rectCount
this.drawnGlyphs = drawnGlyphs
this.runCount = runCount
this.drawnImages = drawnImages
this.fwGlyphs = fwGlyphs
this.fwRuns = fwRuns
this.bakedCandidates = bakedCandidates
this.uploadBytes = uploadBytes
this.fullFrame = fullFrame
}
}
/** One draw selected for on-glasses replay. */
private class Selected {
@JvmField val draw: SurfaceCompositor.ScreenDraw
@JvmField val glyphAtlas: GlyphAtlas.Glyph?
@JvmField val imageAtlas: ImageAtlas.Entry?
@JvmField val gx: Int
@JvmField val top: Int
constructor(
draw: SurfaceCompositor.ScreenDraw,
glyphAtlas: GlyphAtlas.Glyph?,
imageAtlas: ImageAtlas.Entry?,
gx: Int,
top: Int,
) {
this.draw = draw
this.glyphAtlas = glyphAtlas
this.imageAtlas = imageAtlas
this.gx = gx
this.top = top
}
}
/** One punch task for an emitted firmware-font glyph. */
private class FwPunch {
@JvmField val entry: FwGlyphAtlas.Entry
@JvmField val gx: Int
@JvmField val gy: Int
constructor(entry: FwGlyphAtlas.Entry, gx: Int, gy: Int) {
this.entry = entry
this.gx = gx
this.gy = gy
}
}
}
@@ -0,0 +1,89 @@
package com.faceclaw.app
import kotlin.jvm.JvmField
import kotlin.jvm.JvmStatic
/**
* Process-wide registry of the glasses' builtin 20 px font's glyph rasters, fed from the TS side
* (EvenHubFont's extracted firmware font data). Unlike GlyphAtlas/ImageAtlas entries, these are
* never uploaded: the firmware already owns the font, and mode 15 draws it directly. The planner
* needs the rasters purely phone-side — for the "would this draw land correctly" check against the
* composite and for punching drawn ink out of delta rects.
*
* Placement model matches the firmware's mode-15 math: a glyph draws at (penX + ofsX, lineY +
* inkTop) where inkTop already folds in the root font's line height and baseline. The TS side only
* registers glyphs whose on-glasses rendering it predicts exactly (native-table glyphs; baseline-
* shifted fallback symbols stay baked).
*/
class FwGlyphAtlas {
companion object {
private val lock = protocolPlatform().createLock()
private val entries: MutableMap<Int, Entry> = HashMap()
/**
* Register glyph rasters. Little-endian buffer of records: [cp u32][ofsX s16][inkTop
* s16][boxW u8][boxH u8] [boxH x ceil(boxW/2) nibble bytes] Re-registration of a codepoint
* is ignored (the builtin font is fixed).
*/
@JvmStatic
fun register(buffer: ByteReader?): Unit {
if ((buffer == null)) {
return
}
val cursor: ByteReader = buffer
lock.withLock {
while ((cursor.remaining() >= 10)) {
var cp: Int = cursor.getInt()
var ofsX: Int = cursor.getShort().toInt()
var inkTop: Int = cursor.getShort().toInt()
var boxW: Int = (cursor.get() and 0xff)
var boxH: Int = (cursor.get() and 0xff)
var nibbles: ByteArray = ByteArray((((boxW + 1) shr 1) * boxH))
cursor.get(nibbles)
if ((((boxW > 0) && (boxH > 0)) && !entries.containsKey(cp))) {
entries.put(cp, Entry(ofsX, inkTop, boxW, boxH, nibbles))
}
}
}
}
@JvmStatic
fun get(cp: Int): Entry? {
lock.withLock {
return entries.get(cp)
}
}
}
constructor() {}
class Entry {
/** Draw x = pen x + ofsX. */
@JvmField val ofsX: Int
/** First raster row relative to the run's line-top y. */
@JvmField val inkTop: Int
@JvmField val boxW: Int
@JvmField val boxH: Int
/** 4bpp rows, stride ceil(boxW/2), high nibble = left pixel. */
@JvmField val nibbles: ByteArray
constructor(ofsX: Int, inkTop: Int, boxW: Int, boxH: Int, nibbles: ByteArray) {
this.ofsX = ofsX
this.inkTop = inkTop
this.boxW = boxW
this.boxH = boxH
this.nibbles = nibbles
}
/** The 4bpp source value at (col, row); the draw skips 0 (transparent). */
fun nibbleAt(col: Int, row: Int): Int {
var b: Int = (nibbles[((row * ((boxW + 1) shr 1)) + (col shr 1))] and 0xff)
return (if (((col and 1) != 0)) (b and 0x0f) else (b shr 4))
}
}
}
@@ -0,0 +1,325 @@
package com.faceclaw.app
import kotlin.jvm.JvmField
import kotlin.jvm.JvmStatic
/**
* Process-wide registry of glyph rasters, fed from the TS side so glyph identity survives to the
* BLE encoder (see notes/texture-cache-display-list- design.md). A frame's glyph list references
* entries here by (fontId, encoding); the texture-cache planner uses the raster three ways:
*
* - as the ink mask for the "would this draw land correctly" check against the composited frame,
* - to punch the ink pixels out of the baked delta rect it replaces,
* - pre-encoded as the CFW cached-image bytes ([w][h][4bpp RLE]) uploaded via mode 12 and drawn via
* mode 14 with a top-color LUT. 1bpp (BDF) glyphs store ink at 15 so the LUT maps them to exactly
* the requested level; AA (TTF) glyphs store true coverage nibbles the LUT scales.
*
* Fonts are identified by a stable string key (the embedded font name), NOT a per-JS-context
* counter: worker threads register independently and must agree on ids. Glyph rasters are immutable
* once registered; re-registration of the same (fontId, encoding) is ignored.
*
* Cell model: a cached glyph image is bbxWidth x cellHeight (the font's line height), with the ink
* rows at inkTop — so every glyph of a line is drawn at the same y (the line top), which is what
* lets one mode-14 string carry a whole run. The horizontal bearing (bbxX) is applied phone-side
* when computing the draw x. Vertical padding is nearly free: it RLE-encodes to a couple of tokens.
*/
class GlyphAtlas {
companion object {
private val lock = protocolPlatform().createLock()
private val fontIds: MutableMap<String, Int> = HashMap()
private val glyphs: MutableMap<Long, Glyph> = HashMap()
private var nextFontId: Int = 1
/** Stable id for a font key; assigns one on first use. */
@JvmStatic
fun fontId(key: String): Int {
lock.withLock {
var id: Int? = fontIds.get(key)
if ((id == null)) {
id = nextFontId++
fontIds.put(key, id)
}
return id
}
}
@JvmStatic
fun get(fontId: Int, encoding: Int): Glyph? {
lock.withLock {
return glyphs.get(key(fontId, encoding))
}
}
private fun key(fontId: Int, encoding: Int): Long {
return (((fontId).toLong() shl 32) or (encoding.toLong() and 0xffffffffL))
}
/**
* Register a batch of glyph rasters. Little-endian buffer, a sequence of font groups:
* [keyLen u8][key utf8][cellHeight u8][count u16] each followed by count glyph records:
* [encoding u32][bbxX s8][inkTop u8][width u8][inkHeight u8] [inkHeight x rows u32] (bit
* (ceil(width/8)*8 - 1 - col) = ink) Records for an already-registered (font, encoding) are
* skipped (glyph rasters are immutable for a given font key). Malformed buffers throw: they
* indicate a phone-side marshalling bug, never device state.
*/
@JvmStatic
fun register(buffer: ByteReader?): Unit {
if ((buffer == null)) {
return
}
val cursor: ByteReader = buffer
lock.withLock {
while ((cursor.remaining() > 0)) {
var keyLen: Int = (cursor.get() and 0xff)
var keyBytes: ByteArray = ByteArray(keyLen)
cursor.get(keyBytes)
var fontKey: String = keyBytes.decodeToString()
var cellHeight: Int = (cursor.get() and 0xff)
var count: Int = (cursor.getShort().toInt() and 0xffff)
var idBoxed: Int? = fontIds.get(fontKey)
var id: Int = 0
if ((idBoxed == null)) {
id = nextFontId++
fontIds.put(fontKey, id)
} else {
id = idBoxed
}
run {
var g: Int = 0
while ((g < count)) {
var encoding: Int = cursor.getInt()
val bbxX: Int = cursor.get().toInt()
var inkTop: Int = (cursor.get() and 0xff)
var width: Int = (cursor.get() and 0xff)
var inkHeight: Int = (cursor.get() and 0xff)
val rows: IntArray = IntArray(inkHeight)
run {
var r: Int = 0
while ((r < inkHeight)) {
rows[r] = cursor.getInt()
r++
}
}
var k: Long = key(id, encoding)
if (
(((!glyphs.containsKey(k) && (width > 0)) && (cellHeight > 0)) &&
((inkTop + inkHeight) <= cellHeight))
) {
glyphs.put(
k,
Glyph(width, cellHeight, inkTop, inkHeight, bbxX, rows, null),
)
}
g++
}
}
}
}
}
/**
* Register a batch of antialiased (4bpp) glyph rasters, e.g. TTF renders. Same font-group
* framing as register(), but each glyph record carries packed coverage instead of 1bpp
* rows: [encoding u32][bbxX s8][inkTop u8][width u8][inkHeight u8]
* [inkHeight x ceil(width/2) packed bytes, high nibble = left pixel] Nibble values are the
* final 4bpp source levels (the draw-time LUT scales them by the top color).
* Already-registered (font, encoding) records are skipped, as in register().
*/
@JvmStatic
fun registerAa(buffer: ByteReader?): Unit {
if ((buffer == null)) {
return
}
val cursor: ByteReader = buffer
lock.withLock {
while ((cursor.remaining() > 0)) {
var keyLen: Int = (cursor.get() and 0xff)
var keyBytes: ByteArray = ByteArray(keyLen)
cursor.get(keyBytes)
var fontKey: String = keyBytes.decodeToString()
var cellHeight: Int = (cursor.get() and 0xff)
var count: Int = (cursor.getShort().toInt() and 0xffff)
var idBoxed: Int? = fontIds.get(fontKey)
var id: Int = 0
if ((idBoxed == null)) {
id = nextFontId++
fontIds.put(fontKey, id)
} else {
id = idBoxed
}
run {
var g: Int = 0
while ((g < count)) {
var encoding: Int = cursor.getInt()
val bbxX: Int = cursor.get().toInt()
var inkTop: Int = (cursor.get() and 0xff)
var width: Int = (cursor.get() and 0xff)
var inkHeight: Int = (cursor.get() and 0xff)
var stride: Int = ((width + 1) shr 1)
var coverage: ByteArray = ByteArray((width * inkHeight))
run {
var r: Int = 0
while ((r < inkHeight)) {
run {
var b: Int = 0
while ((b < stride)) {
var packed: Int = (cursor.get() and 0xff)
var col: Int = (b * 2)
coverage[((r * width) + col)] =
((packed shr 4)).toByte()
if (((col + 1) < width)) {
coverage[(((r * width) + col) + 1)] =
((packed and 0x0f)).toByte()
}
b++
}
}
r++
}
}
var k: Long = key(id, encoding)
if (
(((!glyphs.containsKey(k) && (width > 0)) && (cellHeight > 0)) &&
((inkTop + inkHeight) <= cellHeight))
) {
glyphs.put(
k,
Glyph(
width,
cellHeight,
inkTop,
inkHeight,
bbxX,
null,
coverage,
),
)
}
g++
}
}
}
}
}
}
constructor() {}
class Glyph {
/** Cached image width in pixels (the glyph's tight bbox width). */
@JvmField val width: Int
/** Cached image height in pixels (the font's line height). */
@JvmField val cellHeight: Int
/** First row of ink within the cell. */
@JvmField val inkTop: Int
/** Number of ink rows. */
@JvmField val inkHeight: Int
/** Horizontal bearing: draw x = pen x + bbxX. */
@JvmField val bbxX: Int
/** 1bpp ink rows (inkHeight entries), bit (rowBitWidth-1-col) = ink; null for AA glyphs. */
@JvmField val rows: IntArray?
@JvmField val rowBitWidth: Int
/** AA coverage, width*inkHeight nibble values 0..15; null for 1bpp glyphs. */
@JvmField val coverage: ByteArray?
/** CFW cached-image bytes: [w][cellHeight][RLE(w*cellHeight px)]. */
@JvmField val cachedBytes: ByteArray
constructor(
width: Int,
cellHeight: Int,
inkTop: Int,
inkHeight: Int,
bbxX: Int,
rows: IntArray?,
coverage: ByteArray?,
) {
this.width = width
this.cellHeight = cellHeight
this.inkTop = inkTop
this.inkHeight = inkHeight
this.bbxX = bbxX
this.rows = rows
this.coverage = coverage
this.rowBitWidth = (((width + 7) shr 3) shl 3)
this.cachedBytes = encodeCachedImage()
}
/**
* The cell pixel's 4bpp source value at (col, row), row cell-relative. 1bpp glyphs are
* stored at 15 so the draw-time LUT (source*top/15) maps them to exactly the requested top
* color; AA glyphs carry their true coverage level.
*/
fun nibbleAt(col: Int, row: Int): Int {
if (((col < 0) || (col >= width))) {
return 0
}
var inkRow: Int = (row - inkTop)
if (((inkRow < 0) || (inkRow >= inkHeight))) {
return 0
}
if ((coverage != null)) {
return coverage[((inkRow * width) + col)].toInt()
}
return (if ((((rows!![inkRow] ushr ((rowBitWidth - 1) - col)) and 1) != 0)) 15 else 0)
}
/** Whether the cell pixel at (col, row) has any ink (row is cell-relative). */
fun inkAt(col: Int, row: Int): Boolean {
return (nibbleAt(col, row) != 0)
}
/**
* The firmware cached-image encoding: [width][height][RLE tokens] covering exactly
* width*cellHeight pixels (no row padding), each pixel's stored color being its nibbleAt
* value.
*/
private fun encodeCachedImage(): ByteArray {
var total: Int = (width * cellHeight)
var out: ByteArray = ByteArray((2 + total))
out[0] = (width).toByte()
out[1] = (cellHeight).toByte()
var o: Int = 2
var i: Int = 0
while ((i < total)) {
var color: Int = nibbleAt((i % width), (i / width))
var j: Int = (i + 1)
while (((j < total) && (nibbleAt((j % width), (j / width)) == color))) {
j++
}
var run: Int = (j - i)
while ((run > 0)) {
var c: Int = minOf(run, 0xffff)
if ((c <= 15)) {
out[o++] = (((c shl 4) or color)).toByte()
} else {
if ((c <= 255)) {
out[o++] = (color).toByte()
out[o++] = (c).toByte()
} else {
out[o++] = (color).toByte()
out[o++] = 0
out[o++] = ((c and 0xff)).toByte()
out[o++] = ((c shr 8)).toByte()
}
}
run -= c
}
i = j
}
var trimmed: ByteArray = ByteArray(o)
out.copyInto(trimmed, 0, 0, 0 + o)
return trimmed
}
}
}
@@ -0,0 +1,134 @@
package com.faceclaw.app
import kotlin.jvm.JvmField
import kotlin.jvm.JvmStatic
/**
* Process-wide registry of icon/image rasters for the texture-cache pipeline, the image counterpart
* of GlyphAtlas. Entries are content-addressed: the TS side keys each image by a hash of its
* dimensions and pixels, so the same icon registered from any thread or rendered at any time
* dedupes to one entry, and an icon whose content changes is simply a new entry.
*
* Unlike glyphs (1-bit ink recolored at draw time), images keep their exact 4bpp values: the cached
* bytes quantize the registered 8bpp pixels with the same GRAY_TO_NIBBLE table the composite is
* packed with, and the planner draws them via mode 13 with an identity LUT (top color 15) and the
* transparent bit — the draw writes exactly the nonzero-nibble pixels, which is what the
* eligibility check and hole punching are defined over.
*/
class ImageAtlas {
companion object {
private val lock = protocolPlatform().createLock()
private val ids: MutableMap<String, Int> = HashMap()
private val entries: MutableList<Entry> = ArrayList()
/**
* Register an image (8bpp grayscale pixels, width*height bytes, row-major) under a content
* key, returning its id — or the existing id when the key is already registered (the pixels
* are then ignored: same key means same content). Ids are positive and stable for the
* process lifetime.
*/
@JvmStatic
fun ensure(key: String?, width: Int, height: Int, pixels8bpp: ByteReader?): Int {
if (
(((((((key == null) || (width <= 0)) || (width > 255)) || (height <= 0)) ||
(height > 255)) || (pixels8bpp == null)) ||
(pixels8bpp.remaining() < (width * height)))
) {
throw IllegalArgumentException(
((("bad image registration " + width) + "x") + height)
)
}
lock.withLock {
var existing: Int? = ids.get(key)
if ((existing != null)) {
return existing
}
var nibbles: ByteArray = ByteArray((width * height))
run {
var i: Int = 0
while ((i < nibbles.size)) {
nibbles[i] = (BmpUtil.nibbleForGray((pixels8bpp.get(i) and 0xff))).toByte()
i++
}
}
entries.add(Entry(width, height, nibbles))
var id: Int = entries.size
ids.put(key, id)
return id
}
}
@JvmStatic
fun get(id: Int): Entry? {
lock.withLock {
return (if (((id >= 1) && (id <= entries.size))) entries.get((id - 1)) else null)
}
}
}
constructor() {}
class Entry {
@JvmField val width: Int
@JvmField val height: Int
/** One 4bpp value per pixel, row-major. */
@JvmField val nibbles: ByteArray
/** CFW cached-image bytes: [w][h][RLE(w*h pixels)]. */
@JvmField val cachedBytes: ByteArray
constructor(width: Int, height: Int, nibbles: ByteArray) {
this.width = width
this.height = height
this.nibbles = nibbles
this.cachedBytes = encodeCachedImage()
}
/** The 4bpp value the mode-13 draw would write at (col, row); 0 = skipped. */
fun nibbleAt(col: Int, row: Int): Int {
return (nibbles[((row * width) + col)] and 0xff)
}
private fun encodeCachedImage(): ByteArray {
var total: Int = (width * height)
var out: ByteArray = ByteArray((2 + total))
out[0] = (width).toByte()
out[1] = (height).toByte()
var o: Int = 2
var i: Int = 0
while ((i < total)) {
var color: Int = (nibbles[i] and 0xff)
var j: Int = (i + 1)
while (((j < total) && ((nibbles[j] and 0xff) == color))) {
j++
}
var run: Int = (j - i)
while ((run > 0)) {
var c: Int = minOf(run, 0xffff)
if ((c <= 15)) {
out[o++] = (((c shl 4) or color)).toByte()
} else {
if ((c <= 255)) {
out[o++] = (color).toByte()
out[o++] = (c).toByte()
} else {
out[o++] = (color).toByte()
out[o++] = 0
out[o++] = ((c and 0xff)).toByte()
out[o++] = ((c shr 8)).toByte()
}
}
run -= c
}
i = j
}
var trimmed: ByteArray = ByteArray(o)
out.copyInto(trimmed, 0, 0, 0 + o)
return trimmed
}
}
}
@@ -0,0 +1,250 @@
package com.faceclaw.app
import kotlin.jvm.JvmField
import kotlin.jvm.JvmStatic
/**
* Reader for the serialized LVGL font image in the G2's CJK flash partition. Faceclaw bundles that
* font because it is Source Han Sans SC Light under the SIL OFL. Proprietary fonts embedded in the
* main firmware never pass through this class and are instead extracted on the user's phone during
* CFW install.
*/
class LvglFontFile {
companion object {
private val lock = protocolPlatform().createLock()
private fun load(path: String?): FontData? = lock.withLock {
if (path.isNullOrEmpty()) return null
val cached = fontCache.remove(path)
if (cached != null) {
fontCache[path] = cached
return cached
}
try {
val font = FontData(readFileData(path))
fontCache[path] = font
while (fontCache.size > FONT_CACHE_SIZE) fontCache.remove(fontCache.keys.first())
font
} catch (_: Exception) {
null
}
}
private const val TAG: String = "LvglFontFile"
private const val MAPPED_BASE: Int = -2146435072
private const val FONT_CACHE_SIZE: Int = 2
private val fontCache: MutableMap<String, FontData> = LinkedHashMap<String, FontData>()
/**
* Returns [lineHeight, baseline] or an empty array when the font is invalid. The baseline
* is measured down from the top of the line.
*/
@JvmStatic
fun getMetrics(path: String): ByteArray {
var font: FontData? = load(path)
if ((font == null)) {
return ByteArray(0)
}
return byteArrayOf((font.lineHeight).toByte(), (font.baseline).toByte())
}
/**
* Returns one packed glyph: u16 boxW, u16 boxH, i16 ofsX, i16 ofsY, then aligned 4bpp
* bitmap bytes. Returns an empty array when the code point is not mapped.
*/
@JvmStatic
fun getGlyph(path: String, codePoint: Int): ByteArray {
var font: FontData? = load(path)
if ((font == null)) {
return ByteArray(0)
}
var glyphId: Int? = font.codePointToGlyph.get(codePoint)
if (((glyphId == null) || (glyphId <= 0))) {
return ByteArray(0)
}
try {
var descriptor: Int = (font.glyphDescriptors + (glyphId * 16))
var bitmapIndex: Int = font.u32(descriptor)
var boxWidth: Int = font.u16((descriptor + 8))
var boxHeight: Int = font.u16((descriptor + 10))
var offsetX: Int = font.i16((descriptor + 12))
var offsetY: Int = font.i16((descriptor + 14))
var stride: Int = ((boxWidth / 2) + 1)
var bitmapLength: Int =
(if (((boxWidth > 0) && (boxHeight > 0))) (stride * boxHeight) else 0)
font.check((font.glyphBitmaps + bitmapIndex), bitmapLength)
var result: ByteArray = ByteArray((8 + bitmapLength))
putU16(result, 0, boxWidth)
putU16(result, 2, boxHeight)
putU16(result, 4, offsetX)
putU16(result, 6, offsetY)
font.bytes.copyInto(
result,
8,
(font.glyphBitmaps + bitmapIndex),
(font.glyphBitmaps + bitmapIndex) + bitmapLength,
)
return result
} catch (e: RuntimeException) {
return ByteArray(0)
}
}
private fun putU16(destination: ByteArray, offset: Int, value: Int): Unit {
destination[offset] = ((value and 0xff)).toByte()
destination[(offset + 1)] = (((value ushr 8) and 0xff)).toByte()
}
}
constructor() {}
private class FontData {
@JvmField val bytes: ByteArray
@JvmField val lineHeight: Int
@JvmField val baseline: Int
@JvmField val glyphBitmaps: Int
@JvmField val glyphDescriptors: Int
@JvmField val codePointToGlyph: MutableMap<Int, Int>
constructor(bytes: ByteArray) {
this.bytes = bytes
if (
(((((bytes.size < 0x40) || (bytes[0].toInt() != 90)) || (bytes[1].toInt() != 90)) ||
(bytes[2].toInt() != 90)) || (bytes[3].toInt() != 90))
) {
throw IllegalArgumentException("missing ZZZZ font header")
}
lineHeight = u16(0x38)
var baseLine: Int = u16(0x3a)
baseline = (lineHeight - baseLine)
var descriptor: Int = pointerOffset(u32(0x34))
glyphBitmaps = pointerOffset(u32(descriptor))
glyphDescriptors = pointerOffset(u32((descriptor + 4)))
var cmaps: Int = pointerOffset(u32((descriptor + 8)))
var packed: Int = u16((descriptor + 18))
var cmapCount: Int = (packed and 0x1ff)
var bitsPerPixel: Int = ((packed ushr 9) and 0xf)
var bitmapFormat: Int = ((packed ushr 14) and 0x3)
if (
(((((lineHeight <= 0) || (baseline < 0)) || (cmapCount <= 0)) ||
(bitsPerPixel != 4)) || (bitmapFormat != 3))
) {
throw IllegalArgumentException("unsupported LVGL font metadata")
}
codePointToGlyph = HashMap()
run {
var index: Int = 0
while ((index < cmapCount)) {
readCmap((cmaps + (index * 20)))
index++
}
}
}
fun readCmap(address: Int): Unit {
var rangeStart: Int = u32(address)
var rangeLength: Int = u16((address + 4))
var glyphIdStart: Int = u16((address + 6))
var unicodeListPointer: Int = u32((address + 8))
var glyphIdOffsetsPointer: Int = u32((address + 12))
var listLength: Int = u16((address + 16))
var type: Int = u8((address + 18))
if ((type == 2)) {
run {
var relative: Int = 0
while ((relative < rangeLength)) {
codePointToGlyph.put((rangeStart + relative), (glyphIdStart + relative))
relative++
}
}
return
}
if ((type == 0)) {
var offsets: Int = pointerOffset(glyphIdOffsetsPointer)
run {
var relative: Int = 0
while ((relative < rangeLength)) {
var offset: Int = u8((offsets + relative))
if (((relative != 0) && (offset == 0))) {
relative++
continue
}
codePointToGlyph.put((rangeStart + relative), (glyphIdStart + offset))
relative++
}
}
return
}
if (((type == 3) || (type == 1))) {
var unicodeList: Int = pointerOffset(unicodeListPointer)
var offsets: Int =
(if ((glyphIdOffsetsPointer == 0)) -1 else pointerOffset(glyphIdOffsetsPointer))
run {
var index: Int = 0
while ((index < listLength)) {
var relative: Int = u16((unicodeList + (index * 2)))
var offset: Int = index
if ((offsets >= 0)) {
offset =
(if ((type == 1)) u16((offsets + (index * 2)))
else u8((offsets + index)))
if (((index != 0) && (offset == 0))) {
index++
continue
}
}
codePointToGlyph.put((rangeStart + relative), (glyphIdStart + offset))
index++
}
}
return
}
throw IllegalArgumentException(("unknown cmap type " + type))
}
fun u8(offset: Int): Int {
check(offset, 1)
return (bytes[offset] and 0xff)
}
fun u16(offset: Int): Int {
check(offset, 2)
return ((bytes[offset].toInt() and 255) or ((bytes[offset + 1].toInt() and 255) shl 8))
}
fun i16(offset: Int): Int {
check(offset, 2)
return u16(offset).toShort().toInt()
}
fun u32(offset: Int): Int {
check(offset, 4)
return u16(offset) or (u16(offset + 2) shl 16)
}
fun pointerOffset(pointer: Int): Int {
var unsignedPointer: Long = (pointer.toLong() and 0xffffffffL)
var offset: Long = (unsignedPointer - (MAPPED_BASE.toLong() and 0xffffffffL))
if (((offset < 0) || (offset > Int.MAX_VALUE))) {
throw IllegalArgumentException("font pointer is outside the mapped partition")
}
check((offset).toInt(), 1)
return (offset).toInt()
}
fun check(offset: Int, length: Int): Unit {
if ((((offset < 0) || (length < 0)) || (((offset).toLong() + length) > bytes.size))) {
throw IllegalArgumentException("font range is out of bounds")
}
}
}
}
@@ -0,0 +1,270 @@
package com.faceclaw.app
import kotlin.jvm.JvmField
import kotlin.math.roundToLong
/**
* Accumulates composited screen frames and saves them as a looping animated GIF with a 16-entry
* grayscale palette. Stored frames are deflate-compressed (a typical UI frame shrinks from 166KB of
* indices to a few KB) so long recordings stay affordable in memory; consecutive identical frames
* are deduplicated, with the elapsed time folded into the next frame's delay.
*/
class SharedGifScreenRecorder {
private val lock = protocolPlatform().createLock()
fun encode(): ByteArray = lock.withLock {
if (frames.isEmpty()) return ByteArray(0)
val out = ByteSink()
writeGif(out)
out.toByteArray()
}
companion object {
/** Backstop against unbounded growth if a recording is left running. */
private const val MAX_FRAMES: Int = 3600
/** Displayed duration of the final frame, which has no successor. */
private const val LAST_FRAME_DELAY_CS: Int = 100
/** Below this, GIF renderers historically clamp or ignore the delay. */
private const val MIN_FRAME_DELAY_CS: Int = 2
/** GIF LZW compression of a 4-bit index stream (minimum code size 4). */
private fun writeLzwImageData(out: ByteSink, indices: ByteArray): Unit {
val minCodeSize: Int = 4
out.write(minCodeSize)
var writer: SubBlockBitWriter = SubBlockBitWriter(out)
val clearCode: Int = (1 shl minCodeSize)
val eoiCode: Int = (clearCode + 1)
var codeSize: Int = (minCodeSize + 1)
var nextCode: Int = (eoiCode + 1)
var table: HashMap<Int, Int> = HashMap()
writer.writeCode(clearCode, codeSize)
var prefix: Int = (indices[0] and 0xff)
run {
var i: Int = 1
while ((i < indices.size)) {
var k: Int = (indices[i] and 0xff)
var key: Int = ((prefix shl 8) or k)
var code: Int? = table.get(key)
if ((code != null)) {
prefix = code
i++
continue
}
writer.writeCode(prefix, codeSize)
if ((nextCode == 0x1000)) {
writer.writeCode(clearCode, codeSize)
table.clear()
codeSize = (minCodeSize + 1)
nextCode = (eoiCode + 1)
} else {
if ((nextCode >= (1 shl codeSize))) {
codeSize++
}
table.put(key, nextCode++)
}
prefix = k
i++
}
}
writer.writeCode(prefix, codeSize)
writer.writeCode(eoiCode, codeSize)
writer.finish()
}
private fun writeShortLE(out: ByteSink, value: Int): Unit {
out.write((value and 0xff))
out.write(((value ushr 8) and 0xff))
}
}
private class Frame {
@JvmField val deflatedIndices: ByteArray
@JvmField val timestampMs: Long
constructor(deflatedIndices: ByteArray, timestampMs: Long) {
this.deflatedIndices = deflatedIndices
this.timestampMs = timestampMs
}
}
private val frames: MutableList<Frame> = ArrayList()
/** Undeflated copy of the newest frame, kept for duplicate detection. */
private var lastIndices: ByteArray? = null
private var width: Int = 0
private var height: Int = 0
private var overflowed: Boolean = false
/**
* Append the current screen. gray is 8bpp; only the top nibble (the display's native depth, and
* the palette index) is kept.
*/
fun addFrame(gray: ByteArray?, width: Int, height: Int, timestampMs: Long): Unit {
lock.withLock {
if (
((((gray == null) || (width <= 0)) || (height <= 0)) ||
(gray.size < (width * height)))
) {
return
}
if (frames.isEmpty()) {
this.width = width
this.height = height
} else {
if (((width != this.width) || (height != this.height))) {
return
}
}
var indices: ByteArray = ByteArray((width * height))
run {
var i: Int = 0
while ((i < indices.size)) {
indices[i] = (((gray[i] and 0xff) ushr 4)).toByte()
i++
}
}
if (((lastIndices != null) && indices.contentEquals(lastIndices))) {
return
}
if ((frames.size >= MAX_FRAMES)) {
overflowed = true
return
}
frames.add(Frame(deflateData(indices), timestampMs))
lastIndices = indices
}
}
/** True if frames were dropped because the recording hit MAX_FRAMES. */
fun isOverflowed(): Boolean {
lock.withLock {
return overflowed
}
}
private fun writeGif(out: ByteSink): Unit {
out.write('G')
out.write('I')
out.write('F')
out.write('8')
out.write('9')
out.write('a')
writeShortLE(out, width)
writeShortLE(out, height)
out.write(((0x80 or 0x70) or 0x03))
out.write(0)
out.write(0)
run {
var i: Int = 0
while ((i < 16)) {
var v: Int = (i * 17)
out.write(v)
out.write(v)
out.write(v)
i++
}
}
out.write(0x21)
out.write(0xff)
out.write(11)
out.write("NETSCAPE2.0".encodeToByteArray())
out.write(3)
out.write(1)
writeShortLE(out, 0)
out.write(0)
run {
var i: Int = 0
while ((i < frames.size)) {
val frame: Frame = frames.get(i)
var delayCs: Int = LAST_FRAME_DELAY_CS
if (((i + 1) < frames.size)) {
var deltaMs: Long = (frames.get((i + 1)).timestampMs - frame.timestampMs)
delayCs =
(maxOf(
MIN_FRAME_DELAY_CS.toLong(),
minOf(0xffffL, (deltaMs / 10.0).roundToLong()),
))
.toInt()
}
out.write(0x21)
out.write(0xf9)
out.write(4)
out.write(0x04)
writeShortLE(out, delayCs)
out.write(0)
out.write(0)
out.write(0x2c)
writeShortLE(out, 0)
writeShortLE(out, 0)
writeShortLE(out, width)
writeShortLE(out, height)
out.write(0)
writeLzwImageData(out, inflateData(frame.deflatedIndices, (width * height)))
i++
}
}
out.write(0x3b)
}
/**
* Packs LZW codes LSB-first into bytes and the bytes into the 255-byte data sub-blocks GIF
* image data is carried in.
*/
private class SubBlockBitWriter {
private val out: ByteSink
private val block: ByteArray = ByteArray(255)
private var blockLength: Int = 0
private var bitBuffer: Int = 0
private var bitCount: Int = 0
constructor(out: ByteSink) {
this.out = out
}
fun writeCode(code: Int, codeSize: Int): Unit {
bitBuffer = (bitBuffer or (code shl bitCount))
bitCount += codeSize
while ((bitCount >= 8)) {
writeByte((bitBuffer and 0xff))
bitBuffer = (bitBuffer ushr 8)
bitCount -= 8
}
}
fun finish(): Unit {
if ((bitCount > 0)) {
writeByte((bitBuffer and 0xff))
bitBuffer = 0
bitCount = 0
}
flushBlock()
out.write(0)
}
private fun writeByte(b: Int): Unit {
block[blockLength++] = (b).toByte()
if ((blockLength == 255)) {
flushBlock()
}
}
private fun flushBlock(): Unit {
if ((blockLength == 0)) {
return
}
out.write(blockLength)
out.write(block, 0, blockLength)
blockLength = 0
}
}
}
@@ -0,0 +1,55 @@
package com.faceclaw.app
import kotlin.jvm.JvmStatic
/** Screen export codecs; filesystem destinations belong to platform adapters. */
class SharedScreenshots private constructor() {
companion object {
@JvmStatic
fun encode4BitGrayPng(gray: ByteArray, width: Int, height: Int): ByteArray {
require(width > 0 && height > 0 && gray.size >= width * height)
val rowBytes = (width + 1) / 2
val raw = ByteArray(height * (1 + rowBytes))
var offset = 0
for (y in 0 until height) {
raw[offset++] = 0 // PNG filter: None
for (x in 0 until width step 2) {
val hi = (gray[y * width + x].toInt() and 255) ushr 4
val lo =
if (x + 1 < width) (gray[y * width + x + 1].toInt() and 255) ushr 4 else 0
raw[offset++] = ((hi shl 4) or lo).toByte()
}
}
val out = ByteSink()
out.write(byteArrayOf(0x89.toByte(), 80, 78, 71, 13, 10, 26, 10))
val header = ByteSink()
writeInt(header, width)
writeInt(header, height)
header.write(byteArrayOf(4, 0, 0, 0, 0))
writeChunk(out, "IHDR", header.toByteArray())
writeChunk(out, "IDAT", deflateData(raw))
writeChunk(out, "IEND", ByteArray(0))
return out.toByteArray()
}
private fun writeChunk(out: ByteSink, type: String, data: ByteArray) {
writeInt(out, data.size)
val name = type.encodeToByteArray()
out.write(name)
out.write(data)
var crc = -1
for (bytes in arrayOf(name, data)) for (byte in bytes) {
crc = crc xor (byte.toInt() and 255)
repeat(8) { crc = (crc ushr 1) xor (if (crc and 1 != 0) 0xedb88320.toInt() else 0) }
}
writeInt(out, crc.inv())
}
private fun writeInt(out: ByteSink, value: Int) {
out.write(value ushr 24)
out.write(value ushr 16)
out.write(value ushr 8)
out.write(value)
}
}
}
@@ -0,0 +1,702 @@
package com.faceclaw.app
import kotlin.jvm.JvmField
import kotlin.jvm.JvmStatic
/**
* Retained-surface compositor: the phone-side model of what is on the glasses screen.
*
* Each surface is an 8bpp grayscale buffer retained at its last-submitted contents, positioned on
* the screen with a z-order and a transparency mode. Sources on the TS side (today the dashboard;
* later the shell and per-app worker threads) submit updates to their surface, and the compositor
* recombines the retained surfaces into a full-screen frame that feeds the existing
* dedupe/compress/transmit pipeline. Because every surface is retained, a full screen frame can be
* regenerated at any time (frame drops, reconnects, previews, screenshots) without asking sources
* to repaint.
*
* Surface format contract (mirrored by the TS side):
* - Pixels are 8bpp grayscale, row-major, one byte per pixel. The wire pipeline quantizes to 4bpp
* downstream, so the low nibble is not visible.
* - TRANSPARENCY_COLOR_KEY surfaces treat pixel value 0 as fully transparent and value 1 as black.
* Quantization collapses 1 into the same 4bpp level as 0, so reserving 0 costs no visible shade;
* painters of color-key surfaces must clamp intentional black to 1.
* - An update may cover any rect of its surface; the rest is retained. The update buffer holds
* exactly rectWidth*rectHeight bytes, row-major.
* - Surfaces composite in ascending z-order onto a black (0) background.
* - Surface geometry changes take effect when the next frame composites; they do not trigger a
* recomposite by themselves.
*
* Thread safety: all methods are safe to call from any thread. Apply and composite happen
* atomically under an internal lock, and each composite carries a monotonic sequence number so
* callers can detect when a composite was superseded by a concurrent one before being acted on.
*/
class SurfaceCompositor {
companion object {
const val TRANSPARENCY_OPAQUE: Int = 0
const val TRANSPARENCY_COLOR_KEY: Int = 1
private val NO_DRAWS: Array<ScreenDraw> = emptyArray<ScreenDraw>()
private fun dimValue(value: Int, dim: Int): Int {
return maxOf(1, (((value * dim) + 128) shr 8))
}
private fun parseDraws(draws: ByteReader?): Array<ScreenDraw> {
if (((draws == null) || (draws.remaining() < 1))) {
return NO_DRAWS
}
val cursor: ByteReader = draws
var out: MutableList<ScreenDraw> = ArrayList()
while ((cursor.remaining() >= 1)) {
var kind: Int = (cursor.get() and 0xff)
if (((kind == ScreenDraw.KIND_GLYPH) && (cursor.remaining() >= 11))) {
var fontId: Int = (cursor.getShort().toInt() and 0xffff)
var encoding: Int = cursor.getInt()
var penX: Int = cursor.getShort().toInt()
var lineY: Int = cursor.getShort().toInt()
var value: Int = (cursor.get() and 0xff)
out.add(ScreenDraw.glyph(fontId, encoding, penX, lineY, value))
} else {
if (((kind == ScreenDraw.KIND_IMAGE) && (cursor.remaining() >= 8))) {
var imageId: Int = cursor.getInt()
var x: Int = cursor.getShort().toInt()
var y: Int = cursor.getShort().toInt()
out.add(ScreenDraw.image(imageId, x, y))
} else {
if (((kind == ScreenDraw.KIND_FWTEXT) && (cursor.remaining() >= 6))) {
var x: Int = cursor.getShort().toInt()
var y: Int = cursor.getShort().toInt()
var value: Int = (cursor.get() and 0xff)
var count: Int = (cursor.get() and 0xff)
if ((cursor.remaining() < (count * 7))) {
break
}
var cps: IntArray = IntArray(count)
var dx: IntArray = IntArray(count)
var ink: BooleanArray = BooleanArray(count)
run {
var i: Int = 0
while ((i < count)) {
cps[i] = cursor.getInt()
dx[i] = cursor.getShort().toInt()
ink[i] = (cursor.get().toInt() != 0)
i++
}
}
out.add(ScreenDraw.fwText(x, y, value, cps, dx, ink))
} else {
break
}
}
}
}
return out.toTypedArray()
}
}
/**
* One deferred draw (a text glyph or an icon image) within a frame, in screen coordinates. The
* draw's pixels are already baked into the composited gray buffer (the TS side bakes before
* submitting); this record preserves the draw's identity so the texture-cache planner can
* replay it as an on-glasses cached draw instead of image bytes.
*
* Glyphs: x/y are the pen position and line top; the raster (and its bearing/cell placement)
* comes from GlyphAtlas under (fontId, encoding). Images: x/y are the blit's top-left; the
* raster comes from ImageAtlas under imageId.
*/
class ScreenDraw {
companion object {
const val KIND_GLYPH: Int = 0
const val KIND_IMAGE: Int = 1
/** A firmware-builtin-font text run (CFW mode 15). */
const val KIND_FWTEXT: Int = 2
@JvmStatic
fun glyph(fontId: Int, encoding: Int, penX: Int, lineY: Int, value: Int): ScreenDraw {
return ScreenDraw(
KIND_GLYPH,
fontId,
encoding,
0,
penX,
lineY,
value,
null,
null,
null,
)
}
@JvmStatic
fun image(imageId: Int, x: Int, y: Int): ScreenDraw {
return ScreenDraw(KIND_IMAGE, 0, 0, imageId, x, y, 0, null, null, null)
}
@JvmStatic
fun fwText(
x: Int,
y: Int,
value: Int,
cps: IntArray,
dx: IntArray,
ink: BooleanArray,
): ScreenDraw {
return ScreenDraw(KIND_FWTEXT, 0, 0, 0, x, y, value, cps, dx, ink)
}
}
@JvmField val kind: Int
/** Glyph draws only. */
@JvmField val fontId: Int
/** Glyph draws only. */
@JvmField val encoding: Int
/** Image draws only. */
@JvmField val imageId: Int
@JvmField val x: Int
@JvmField val y: Int
/** Glyph and fw-text draws: 8-bit brightness. */
@JvmField val value: Int
/** Fw-text runs only: member codepoints, in text order. */
@JvmField val fwCps: IntArray?
/** Fw-text runs only: member pen offsets relative to x. */
@JvmField val fwDx: IntArray?
/** Fw-text runs only: whether each member has visible pixels. */
@JvmField val fwInk: BooleanArray?
constructor(
kind: Int,
fontId: Int,
encoding: Int,
imageId: Int,
x: Int,
y: Int,
value: Int,
fwCps: IntArray?,
fwDx: IntArray?,
fwInk: BooleanArray?,
) {
this.kind = kind
this.fontId = fontId
this.encoding = encoding
this.imageId = imageId
this.x = x
this.y = y
this.value = value
this.fwCps = fwCps
this.fwDx = fwDx
this.fwInk = fwInk
}
}
/** One composited full-screen frame plus the metadata the pipeline needs. */
class Composite {
/** Full-screen 8bpp grayscale pixels, screenWidth*screenHeight bytes. */
@JvmField val gray: ByteArray
@JvmField val width: Int
@JvmField val height: Int
/**
* Stable identifier of screen content, combining every surface's geometry and content
* fingerprint; equal fingerprints mean equal composited pixels.
*/
@JvmField val fingerprint: String
/** Monotonic: a Composite with a higher seq contains strictly newer state. */
@JvmField val seq: Long
/**
* Screen-space deferred draws of every visible surface, in composite order (surface z
* ascending, then each surface's draw order). Their pixels are already baked into gray.
*/
@JvmField val draws: Array<ScreenDraw>?
constructor(
gray: ByteArray,
width: Int,
height: Int,
fingerprint: String,
seq: Long,
draws: Array<ScreenDraw>?,
) {
this.gray = gray
this.width = width
this.height = height
this.fingerprint = fingerprint
this.seq = seq
this.draws = (if ((draws == null)) NO_DRAWS else draws)
}
}
private class Surface {
@JvmField val id: String
@JvmField var x: Int = 0
@JvmField var y: Int = 0
@JvmField var width: Int = 0
@JvmField var height: Int = 0
@JvmField var zOrder: Int = 0
@JvmField var transparency: Int = 0
@JvmField var visible: Boolean = true
@JvmField var pixels: ByteArray = ByteArray(0)
@JvmField var fingerprint: String = ""
/** Surface-local deferred draws of the retained content (already baked into pixels). */
@JvmField var draws: Array<ScreenDraw> = NO_DRAWS
constructor(id: String) {
this.id = id
}
}
private val lock = protocolPlatform().createLock()
private var screenWidth: Int = 0
private var screenHeight: Int = 0
private var blanked: Boolean = false
/** See setUnderlayDim: surfaces with zOrder below this are dimmed by underlayDim/256. */
private var underlayDimBelowZOrder: Int = Int.MIN_VALUE
private var underlayDim: Int = 256
private val surfaces: MutableMap<String, Surface> = HashMap()
private var nextCompositeSeq: Long = 1
/** Set the output frame size. Must be called before any surface work. */
fun configureScreen(width: Int, height: Int): Unit {
if (((width <= 0) || (height <= 0))) {
throw IllegalArgumentException(((("bad screen size " + width) + "x") + height))
}
lock.withLock {
this.screenWidth = width
this.screenHeight = height
}
}
/**
* Create a surface or update an existing one's geometry. A resize discards the surface's
* retained pixels (reset to 0).
*/
fun configureSurface(
id: String?,
x: Int,
y: Int,
width: Int,
height: Int,
zOrder: Int,
transparency: Int,
): Unit {
if (((id == null) || id.isEmpty())) {
throw IllegalArgumentException("surface id must be non-empty")
}
if (((width <= 0) || (height <= 0))) {
throw IllegalArgumentException(
((((("bad surface size " + width) + "x") + height) + " for ") + id)
)
}
if (((transparency != TRANSPARENCY_OPAQUE) && (transparency != TRANSPARENCY_COLOR_KEY))) {
throw IllegalArgumentException(
((("bad transparency mode " + transparency) + " for ") + id)
)
}
lock.withLock {
requireScreenConfiguredLocked()
var surface: Surface? = surfaces.get(id)
if ((surface == null)) {
surface = Surface(id)
surfaces.put(id, surface)
}
if (
(((surface.pixels == null) || (surface.width != width)) ||
(surface.height != height))
) {
surface.pixels = ByteArray((width * height))
surface.fingerprint = ""
surface.draws = NO_DRAWS
}
surface.x = x
surface.y = y
surface.width = width
surface.height = height
surface.zOrder = zOrder
surface.transparency = transparency
}
}
fun removeSurface(id: String): Unit {
lock.withLock {
surfaces.remove(id)
}
}
/**
* Dim every surface whose zOrder is below belowZOrder to factor256/256 of its brightness (256 =
* no dimming): how a shell overlay that dims what it covers (the TS Layer.dimUnderneath)
* reaches the window surfaces beneath the shell surface, which the shell's own layer stack
* cannot paint. Visible pixels stay at least 1 (the color-key black); glyph and firmware-text
* draws keep their cached-draw form with a dimmed value; image draws leave the composite's draw
* list (their pixels, already baked into the surface, dim as raster). Takes effect when the
* next frame composites.
*/
fun setUnderlayDim(belowZOrder: Int, factor256: Int): Unit {
lock.withLock {
underlayDimBelowZOrder = belowZOrder
underlayDim = maxOf(0, minOf(256, factor256))
}
}
/** The dim factor (256 = none) that applies to a surface. */
private fun dimForLocked(surface: Surface): Int {
return (if ((surface.zOrder < underlayDimBelowZOrder)) underlayDim else 256)
}
/**
* Hidden surfaces keep their retained pixels and accept updates, but are excluded from the
* composite and its fingerprint (used for background windows). Takes effect when the next frame
* composites.
*/
fun setSurfaceVisible(id: String, visible: Boolean): Unit {
lock.withLock {
var surface: Surface? = surfaces.get(id)
if ((surface == null)) {
throw IllegalArgumentException(("unknown surface " + id))
}
surface.visible = visible
}
}
/**
* While blanked (screen off), composites are all-zero regardless of surface content; retained
* state is untouched, so unblanking restores the screen without asking sources to repaint.
*/
fun setBlanked(blanked: Boolean): Unit {
lock.withLock {
this.blanked = blanked
}
}
/**
* Apply an update to one surface and composite the whole screen, as one atomic step. The update
* covers the rect (rectX, rectY, rectWidth, rectHeight) in surface-local coordinates; pixels
* must hold exactly rectWidth*rectHeight bytes. contentFingerprint identifies the surface's
* full content after this update.
*/
fun applyAndComposite(
surfaceId: String,
pixels: ByteReader,
rectX: Int,
rectY: Int,
rectWidth: Int,
rectHeight: Int,
contentFingerprint: String,
): Composite {
return applyAndComposite(
surfaceId,
pixels,
rectX,
rectY,
rectWidth,
rectHeight,
contentFingerprint,
null,
)
}
/**
* As above, with the frame's deferred draws: a little-endian buffer of tagged records
* [0][fontId u16][encoding u32][penX s16][lineY s16][value u8] (glyph) [1][imageId u32][x
* s16][y s16] (image) in surface-local coordinates and draw order. The list describes the
* surface's FULL retained content and replaces the previous list, so it is only meaningful for
* full-surface updates (which is what every submitter sends); pass null to clear. Draw pixels
* must already be baked into pixels.
*/
fun applyAndComposite(
surfaceId: String,
pixels: ByteReader?,
rectX: Int,
rectY: Int,
rectWidth: Int,
rectHeight: Int,
contentFingerprint: String?,
glyphs: ByteReader?,
): Composite =
updateSurface(
surfaceId,
pixels,
rectX,
rectY,
rectWidth,
rectHeight,
contentFingerprint,
glyphs,
true,
)!!
/** Retain pixels without recompositing; iOS coalesces updates before its frame callback. */
fun submitSurface(
surfaceId: String,
pixels: ByteReader,
rectX: Int,
rectY: Int,
rectWidth: Int,
rectHeight: Int,
fingerprint: String,
) {
updateSurface(
surfaceId,
pixels,
rectX,
rectY,
rectWidth,
rectHeight,
fingerprint,
null,
false,
)
}
private fun updateSurface(
surfaceId: String,
pixels: ByteReader?,
rectX: Int,
rectY: Int,
rectWidth: Int,
rectHeight: Int,
contentFingerprint: String?,
glyphs: ByteReader?,
composeAfter: Boolean,
): Composite? {
var parsed: Array<ScreenDraw> = parseDraws(glyphs)
lock.withLock {
requireScreenConfiguredLocked()
var surface: Surface? = surfaces.get(surfaceId)
if ((surface == null)) {
throw IllegalArgumentException(("unknown surface " + surfaceId))
}
if (
((((((rectX < 0) || (rectY < 0)) || (rectWidth <= 0)) || (rectHeight <= 0)) ||
((rectX + rectWidth) > surface.width)) ||
((rectY + rectHeight) > surface.height))
) {
throw IllegalArgumentException(
(((((((((((((("update rect " + rectWidth) + "x") + rectHeight) + "+") + rectX) +
"+") + rectY) + " outside surface ") + surfaceId) + " (") + surface.width) +
"x") + surface.height) + ")")
)
}
var expectedBytes: Int = (rectWidth * rectHeight)
if (((pixels == null) || (pixels.remaining() != expectedBytes))) {
throw IllegalArgumentException(
((((("update buffer for " + surfaceId) + " has ") +
(if ((pixels == null)) 0 else pixels.remaining())) + " bytes, expected ") +
expectedBytes)
)
}
run {
var row: Int = 0
while ((row < rectHeight)) {
var dstOffset: Int = (((rectY + row) * surface.width) + rectX)
pixels.get(surface.pixels, dstOffset, rectWidth)
row++
}
}
surface.fingerprint = (if ((contentFingerprint == null)) "" else contentFingerprint)
surface.draws = parsed
return if (composeAfter) compositeLocked() else null
}
}
/** Composite the current retained state without applying an update. */
fun composite(): Composite {
lock.withLock {
requireScreenConfiguredLocked()
return compositeLocked()
}
}
/**
* Current composited pixels for the phone-side preview / screenshot, or null before the screen
* is configured. Does not consume a sequence number (it is never stored as the desired frame).
*/
fun previewComposite(): Composite? {
lock.withLock {
if (((screenWidth <= 0) || (screenHeight <= 0))) {
return null
}
var gray: ByteArray = buildGrayLocked()
return Composite(gray, screenWidth, screenHeight, "preview", 0, NO_DRAWS)
}
}
private fun buildGrayLocked(): ByteArray {
var gray: ByteArray = ByteArray((screenWidth * screenHeight))
if (blanked) {
return gray
}
var ordered: MutableList<Surface> = ArrayList(surfaces.values)
ordered.sortWith(compareBy<Surface> { it.zOrder }.thenBy { it.id })
for (surface in ordered) {
if (surface.visible) {
blendLocked(gray, surface)
}
}
return gray
}
private fun compositeLocked(): Composite {
var gray: ByteArray = ByteArray((screenWidth * screenHeight))
if (blanked) {
return Composite(
gray,
screenWidth,
screenHeight,
((("blanked:" + screenWidth) + "x") + screenHeight),
nextCompositeSeq++,
NO_DRAWS,
)
}
var ordered: MutableList<Surface> = ArrayList(surfaces.values)
ordered.sortWith(compareBy<Surface> { it.zOrder }.thenBy { it.id })
var fingerprint: StringBuilder = StringBuilder()
fingerprint.append(screenWidth).append('x').append(screenHeight)
var draws: MutableList<ScreenDraw> = ArrayList()
for (surface in ordered) {
if (!surface.visible) {
continue
}
blendLocked(gray, surface)
var dim: Int = dimForLocked(surface)
for (draw in surface.draws) {
if (((dim < 256) && (draw.kind == ScreenDraw.KIND_IMAGE))) {
continue
}
var value: Int = (if ((dim < 256)) dimValue(draw.value, dim) else draw.value)
draws.add(
ScreenDraw(
draw.kind,
draw.fontId,
draw.encoding,
draw.imageId,
(draw.x + surface.x),
(draw.y + surface.y),
value,
draw.fwCps,
draw.fwDx,
draw.fwInk,
)
)
}
fingerprint
.append('|')
.append(surface.id)
.append('@')
.append(surface.x)
.append(',')
.append(surface.y)
.append('+')
.append(surface.width)
.append('x')
.append(surface.height)
.append('#')
.append(surface.zOrder)
.append(':')
.append(surface.transparency)
.append(':')
.append(surface.fingerprint)
if ((dim < 256)) {
fingerprint.append(":dim").append(dim)
}
}
return Composite(
gray,
screenWidth,
screenHeight,
fingerprint.toString(),
nextCompositeSeq++,
draws.toTypedArray(),
)
}
private fun blendLocked(gray: ByteArray, surface: Surface): Unit {
var srcX: Int = maxOf(0, -surface.x)
var srcY: Int = maxOf(0, -surface.y)
var dstX: Int = maxOf(0, surface.x)
var dstY: Int = maxOf(0, surface.y)
var copyWidth: Int = minOf((surface.width - srcX), (screenWidth - dstX))
var copyHeight: Int = minOf((surface.height - srcY), (screenHeight - dstY))
if (((copyWidth <= 0) || (copyHeight <= 0))) {
return
}
var dim: Int = dimForLocked(surface)
run {
var row: Int = 0
while ((row < copyHeight)) {
var srcOffset: Int = (((srcY + row) * surface.width) + srcX)
var dstOffset: Int = (((dstY + row) * screenWidth) + dstX)
if ((dim < 256)) {
run {
var col: Int = 0
while ((col < copyWidth)) {
var value: Int = (surface.pixels[(srcOffset + col)] and 0xff)
if ((value.toInt() != 0)) {
gray[(dstOffset + col)] = (dimValue(value, dim)).toByte()
} else {
if ((surface.transparency == TRANSPARENCY_OPAQUE)) {
gray[(dstOffset + col)] = 0
}
}
col++
}
}
} else {
if ((surface.transparency == TRANSPARENCY_OPAQUE)) {
surface.pixels.copyInto(gray, dstOffset, srcOffset, srcOffset + copyWidth)
} else {
run {
var col: Int = 0
while ((col < copyWidth)) {
var value: Byte = surface.pixels[(srcOffset + col)]
if ((value.toInt() != 0)) {
gray[(dstOffset + col)] = value
}
col++
}
}
}
}
row++
}
}
}
private fun requireScreenConfiguredLocked(): Unit {
if (((screenWidth <= 0) || (screenHeight <= 0))) {
throw IllegalStateException("configureScreen must be called first")
}
}
}
@@ -0,0 +1,45 @@
package com.faceclaw.app
import kotlin.jvm.JvmStatic
/** Binary encodings shared by firmware downloads and audio recordings. */
object BinaryEncoding {
@JvmStatic
fun bytesToHex(bytes: ByteArray?): String {
if (bytes == null) return ""
val digits = "0123456789abcdef"
return buildString(bytes.size * 2) {
for (byte in bytes) {
val value = byte.toInt() and 255
append(digits[value ushr 4])
append(digits[value and 15])
}
}
}
@JvmStatic
fun wavHeader(pcmBytes: Int, sampleRate: Int, channels: Int): ByteArray {
val out = ByteSink(44)
fun u16(value: Int) {
out.write(value)
out.write(value ushr 8)
}
fun u32(value: Int) {
u16(value)
u16(value ushr 16)
}
out.write("RIFF".encodeToByteArray())
u32(36 + pcmBytes)
out.write("WAVEfmt ".encodeToByteArray())
u32(16)
u16(1)
u16(channels)
u32(sampleRate)
u32(sampleRate * channels * 2)
u16(channels * 2)
u16(16)
out.write("data".encodeToByteArray())
u32(pcmBytes)
return out.toByteArray()
}
}
@@ -0,0 +1,150 @@
package com.faceclaw.app
import kotlin.jvm.JvmStatic
class BmpUtil {
companion object {
@JvmStatic
fun copyTileBmp(bmp: ByteArray?): ByteArray {
if (((bmp == null) || (bmp.size == 0))) {
return ByteArray(0)
}
return bmp.copyOf(bmp.size)
}
private val GRAY_TO_NIBBLE =
ByteArray(256) { v -> if (v == 0) 0 else minOf(15, (v + 8) shr 4).toByte() }
/**
* The 4bpp level an 8-bit gray value packs to. Public so the texture-cache planner computes
* a glyph draw's top color with the exact quantization the composited frame was packed
* with.
*/
@JvmStatic
fun nibbleForGray(gray: Int): Int {
return (GRAY_TO_NIBBLE[(gray and 0xff)] and 0xff)
}
/**
* Pack a full-resolution 8bpp grayscale buffer (row-major, top-to-bottom, one byte per
* pixel) into the headerless 4bpp wire format used by CFW load_image_z modes 3/6/8:
* top-down rows, stride ceil(width/2), high nibble = left pixel. This is the canonical
* in-memory frame format; BMP framing (needed only when an uncompressed full frame goes on
* the wire) is added on demand by build4bppBmpFromPacked.
*/
@JvmStatic
fun pack4bppFromGray8(gray8: ByteArray?, width: Int, height: Int): ByteArray {
if (((width <= 0) || (height <= 0))) {
return ByteArray(0)
}
var stride: Int = ((width + 1) shr 1)
var out: ByteArray = ByteArray((stride * height))
if (((gray8 == null) || (gray8.size < (width * height)))) {
return out
}
var pairs: Int = (width shr 1)
var src: Int = 0
var dst: Int = 0
run {
var y: Int = 0
while ((y < height)) {
run {
var i: Int = 0
while ((i < pairs)) {
out[dst++] =
(((GRAY_TO_NIBBLE[(gray8[src] and 0xff)] shl 4) or
(GRAY_TO_NIBBLE[(gray8[(src + 1)] and 0xff)] and 0xff)))
.toByte()
src += 2
i++
}
}
if (((width and 1) != 0)) {
out[dst++] = ((GRAY_TO_NIBBLE[(gray8[src++] and 0xff)] shl 4)).toByte()
}
y++
}
}
return out
}
/**
* Wrap a packed 4bpp frame in BMP framing (BITMAPINFOHEADER + 16-entry gray palette,
* bottom-up rows padded to a 4-byte stride). Only the full-frame fallback used when mode-6
* compression does not shrink the frame still speaks BMP on the wire; everything else uses
* the headerless packed format directly.
*/
@JvmStatic
fun build4bppBmpFromPacked(packed: ByteArray?, width: Int, height: Int): ByteArray {
var packedStride: Int = ((width + 1) shr 1)
if (
((((width <= 0) || (height <= 0)) || (packed == null)) ||
(packed.size < (packedStride * height)))
) {
return ByteArray(0)
}
var rowStride: Int = ((packedStride + 3) and 3.inv())
var pixelDataSize: Int = (rowStride * height)
var fileHeaderSize: Int = 14
var dibHeaderSize: Int = 40
var paletteSize: Int = (16 * 4)
var pixelOffset: Int = ((fileHeaderSize + dibHeaderSize) + paletteSize)
var fileSize: Int = (pixelOffset + pixelDataSize)
var buf: ByteArray = ByteArray(fileSize)
buf[0] = 0x42
buf[1] = 0x4
putUint32Le(buf, 2, fileSize)
putUint32Le(buf, 10, pixelOffset)
putUint32Le(buf, 14, dibHeaderSize)
putInt32Le(buf, 18, width)
putInt32Le(buf, 22, height)
putUint16Le(buf, 26, 1)
putUint16Le(buf, 28, 4)
putUint32Le(buf, 30, 0)
putUint32Le(buf, 34, pixelDataSize)
putUint32Le(buf, 46, 16)
run {
var i: Int = 0
while ((i < 16)) {
var v: Int = (i * 17)
var base: Int = ((fileHeaderSize + dibHeaderSize) + (i * 4))
buf[base] = (v).toByte()
buf[(base + 1)] = (v).toByte()
buf[(base + 2)] = (v).toByte()
buf[(base + 3)] = 0
i++
}
}
run {
var bmpRow: Int = 0
while ((bmpRow < height)) {
var srcY: Int = ((height - 1) - bmpRow)
packed.copyInto(
buf,
(pixelOffset + (bmpRow * rowStride)),
(srcY * packedStride),
(srcY * packedStride) + packedStride,
)
bmpRow++
}
}
return buf
}
private fun putUint16Le(buf: ByteArray, offset: Int, value: Int): Unit {
buf[offset] = ((value and 0xff)).toByte()
buf[(offset + 1)] = (((value ushr 8) and 0xff)).toByte()
}
private fun putUint32Le(buf: ByteArray, offset: Int, value: Int): Unit {
buf[offset] = ((value and 0xff)).toByte()
buf[(offset + 1)] = (((value ushr 8) and 0xff)).toByte()
buf[(offset + 2)] = (((value ushr 16) and 0xff)).toByte()
buf[(offset + 3)] = (((value ushr 24) and 0xff)).toByte()
}
private fun putInt32Le(buf: ByteArray, offset: Int, value: Int): Unit {
putUint32Le(buf, offset, value)
}
}
}
@@ -0,0 +1,43 @@
package com.faceclaw.app
/** Sequential little-endian input. Platform adapters can read native buffers without a copy. */
abstract class ByteReader {
abstract fun remaining(): Int
abstract fun get(): Byte
abstract fun get(index: Int): Byte
open fun get(bytes: ByteArray): ByteReader = get(bytes, 0, bytes.size)
open fun get(bytes: ByteArray, offset: Int, length: Int): ByteReader {
require(
length >= 0 && offset >= 0 && offset <= bytes.size - length && remaining() >= length
)
for (i in offset until offset + length) bytes[i] = get()
return this
}
fun getShort(): Short = ((get().toInt() and 255) or ((get().toInt() and 255) shl 8)).toShort()
fun getInt(): Int = (getShort().toInt() and 65535) or (getShort().toInt() shl 16)
}
class ArrayByteReader(private val bytes: ByteArray) : ByteReader() {
private var position = 0
override fun remaining(): Int = bytes.size - position
override fun get(): Byte = bytes[position++]
override fun get(index: Int): Byte = bytes[index]
override fun get(bytes: ByteArray, offset: Int, length: Int): ByteReader {
require(
length >= 0 && offset >= 0 && offset <= bytes.size - length && remaining() >= length
)
this.bytes.copyInto(bytes, offset, position, position + length)
position += length
return this
}
}
@@ -0,0 +1,44 @@
package com.faceclaw.app
/** Growable byte buffer for wire encoders; no streams or platform objects escape. */
internal class ByteSink(capacity: Int = 32) {
private var bytes = ByteArray(capacity.coerceAtLeast(1))
private var count = 0
fun size(): Int = count
fun write(value: Char) = write(value.code)
fun write(value: ByteArray) = write(value, 0, value.size)
fun write(value: Int) {
reserve(1)
bytes[count++] = value.toByte()
}
fun write(value: ByteArray, offset: Int, length: Int) {
require(offset >= 0 && length >= 0 && offset <= value.size - length)
reserve(length)
value.copyInto(bytes, count, offset, offset + length)
count += length
}
fun toByteArray(): ByteArray = bytes.copyOf(count)
private fun reserve(extra: Int) {
if (count + extra > bytes.size) bytes = bytes.copyOf(maxOf(count + extra, bytes.size * 2))
}
}
internal infix fun Byte.and(mask: Int): Int = toInt() and mask
internal infix fun Byte.shl(bits: Int): Int = toInt() shl bits
internal fun codePointUtf8(codePoint: Int): ByteArray {
require(codePoint in 0..0x10ffff)
if (codePoint <= 0xffff) return codePoint.toChar().toString().encodeToByteArray()
val value = codePoint - 0x10000
return charArrayOf((0xd800 + (value ushr 10)).toChar(), (0xdc00 + (value and 1023)).toChar())
.concatToString()
.encodeToByteArray()
}
@@ -0,0 +1,31 @@
package com.faceclaw.app
import kotlin.jvm.JvmStatic
class CollectionUtils {
companion object {
@JvmStatic
fun singletonList(value: ByteArray): MutableList<ByteArray> {
var list: MutableList<ByteArray> = ArrayList(1)
list.add(value)
return list
}
@JvmStatic
fun listOf(a: ByteArray, b: ByteArray): MutableList<ByteArray> {
var list: MutableList<ByteArray> = ArrayList(2)
list.add(a)
list.add(b)
return list
}
@JvmStatic
fun listOf(a: ByteArray, b: ByteArray, c: ByteArray): MutableList<ByteArray> {
var list: MutableList<ByteArray> = ArrayList(3)
list.add(a)
list.add(b)
list.add(c)
return list
}
}
}
@@ -6,11 +6,17 @@ interface KotlinBridgeListener {
}
/**
* Portable Kotlin at the same commonMain source path used by ios-port.
* Calls and callbacks are synchronous on the caller's thread. The listener is
* used only during roundTrip and is never retained.
* Portable Kotlin at the same commonMain source path used by ios-port. Calls and callbacks are
* synchronous on the caller's thread. The listener is used only during roundTrip and is never
* retained.
*/
class KotlinBridge {
fun platform(): String = PlatformInfo.name
fun threadName(): String = PlatformInfo.threadName()
fun isMainThread(): Boolean = PlatformInfo.isMainThread()
fun greet(name: String): String = "Hello, $name, from Kotlin"
fun roundTrip(message: String, listener: KotlinBridgeListener): String {
@@ -19,3 +25,11 @@ class KotlinBridge {
return "Kotlin received callback: $reply"
}
}
internal expect object PlatformInfo {
val name: String
fun threadName(): String
fun isMainThread(): Boolean
}
@@ -0,0 +1,264 @@
@file:OptIn(kotlinx.cinterop.ExperimentalForeignApi::class)
package com.faceclaw.app
import kotlinx.cinterop.*
import platform.Foundation.NSData
import platform.Foundation.create
import platform.posix.memcpy
/** NSData keeps large binary crossings out of per-element JavaScript bridge loops. */
internal fun NSData.byteArray(): ByteArray {
require(length <= Int.MAX_VALUE.toULong())
val result = ByteArray(length.toInt())
if (result.isNotEmpty()) result.usePinned { memcpy(it.addressOf(0), bytes, length) }
return result
}
internal fun ByteArray.data(): NSData =
if (isEmpty()) NSData()
else
usePinned {
NSData.create(bytes = it.addressOf(0), length = size.toULong())
}
class IosByteReader(private val data: NSData) : ByteReader() {
private var position = 0
override fun remaining(): Int = data.length.toInt() - position
override fun get(): Byte = get(position++)
override fun get(index: Int): Byte {
require(index >= 0 && index < data.length.toInt())
return data.bytes!!.reinterpret<ByteVar>()[index]
}
override fun get(bytes: ByteArray, offset: Int, length: Int): ByteReader {
require(
offset >= 0 && length >= 0 && offset <= bytes.size - length && remaining() >= length
)
if (length > 0)
bytes.usePinned {
memcpy(
it.addressOf(offset),
data.bytes!!.reinterpret<ByteVar>() + position,
length.toULong(),
)
}
position += length
return this
}
}
class IosCfwTransport {
private val transport = CfwTransport(IosProtocolPlatform)
fun reset() = transport.reset()
fun close() = transport.close()
fun encode(data: NSData, streamId: Int, lenses: Int, maxWrite: Int): List<NSData> =
transport.encode(data.byteArray(), streamId, lenses, maxWrite + 3).map { it.data() }
}
class IosProtocolMessage(message: MessageReceiver.Message) {
val sid = message.sid
val flag = message.flag
val payload = message.payload.data()
val command = message.command
val magic = message.magic
val packet = message.packet?.data()
}
class IosMessageReceiver {
private val receiver = MessageReceiver()
fun clear() = receiver.clear()
fun receive(link: String, data: NSData, now: Long): List<IosProtocolMessage> =
receiver.receive(link, data.byteArray(), now).map { IosProtocolMessage(it) }
}
/** Stable binary facade over the shared wire algorithms and font parser. */
class IosProtocol {
fun authentication(magic: Int): NSData = BleProtocol.buildAuthenticationRequest(magic).data()
fun prelude(): NSData = BleProtocol.PRELUDE_F5872_PAYLOAD.data()
fun heartbeat(magic: Int): NSData = BleProtocol.buildHeartbeat(magic).data()
fun audioControl(magic: Int, enabled: Boolean): NSData =
BleProtocol.buildAudioControl(magic, enabled).data()
fun settingsQuery(magic: Int): NSData = BleProtocol.buildSettingsQuery(magic).data()
fun shutdown(magic: Int): NSData = BleProtocol.buildShutdown(magic, 1).data()
fun framebufferLease(acquire: Boolean): NSData =
BleProtocol.buildFaceclawWakeControl(
if (acquire) BleProtocol.FACECLAW_FB_OP_ACQUIRE
else BleProtocol.FACECLAW_FB_OP_RELEASE,
0,
)
.data()
fun createLayout(magic: Int): NSData = BleProtocol.buildCreateInputPage(magic).data()
fun integer(field: Int, value: Int): NSData = BleProtocol.encodeVarintField(field, value).data()
fun bytes(field: Int, data: NSData): NSData =
BleProtocol.encodeBytesField(field, data.byteArray()).data()
fun crc(data: NSData): Int = CfwTransport.crc(data.byteArray())
fun frame(data: NSData, sid: Int, flag: Int, sequence: Int, maxWrite: Int): List<NSData> =
BleProtocol.framePb(data.byteArray(), sid, flag, sequence, maxWrite).map { it.data() }
fun acks(data: NSData): List<CfwTransport.Ack>? =
CfwTransport.parseAcks(data.byteArray())?.toList()
fun readInteger(data: NSData, field: Int, fallback: Int): Int =
BleProtocol.readVarintFieldValue(data.byteArray(), field, fallback)
fun readBytes(data: NSData, field: Int): NSData? =
BleProtocol.readFieldBytes(data.byteArray(), field)?.data()
fun readString(data: NSData, field: Int): String =
BleProtocol.readStringFieldValue(data.byteArray(), field)
fun glassesInput(data: NSData, sid: Int, flag: Int): G2Event? =
if (sid == BleProtocol.SID_EVENHUB && flag in intArrayOf(1, 6))
G2Event.decodePayload(sid, data.byteArray())
else null
fun ringInput(data: NSData): G2Event? = FaceclawRingEventDecoder.decode(data.byteArray())?.event
fun pack(data: NSData, width: Int, height: Int): NSData {
require(width > 0 && height > 0 && data.length == width.toULong() * height.toULong())
return BmpUtil.pack4bppFromGray8(data.byteArray(), width, height).data()
}
fun rle(data: NSData): NSData = BleImageOptimizer.rleEncode(data.byteArray()).data()
fun boundingBox(
previous: NSData?,
next: NSData,
width: Int,
height: Int,
frameId: Int,
): NSData? =
BleImageOptimizer.buildIncrementalImagePayload(
previous?.byteArray(),
next.byteArray(),
width,
height,
frameId,
)
?.payload
?.data()
fun fullFrameBands(data: NSData, width: Int, height: Int, firstId: Int): List<NSData> {
require(
width in 1..640 &&
width % 4 == 0 &&
height in 1..480 &&
height % 2 == 0 &&
data.length == (width * height / 2).toULong()
)
val packed = data.byteArray()
var id = firstId
return (0 until height step 64).map { top ->
val result =
BleImageOptimizer.encodeMode3Rect(
packed,
width / 2,
0,
top,
width,
minOf(64, height - top),
id,
)
.data()
id = if (id >= 0xfffe) 1 else id + 1
result
}
}
fun fontMetrics(path: String): NSData = LvglFontFile.getMetrics(path).data()
fun fontGlyph(path: String, codePoint: Int): NSData =
LvglFontFile.getGlyph(path, codePoint).data()
fun png(data: NSData, width: Int, height: Int): NSData =
SharedScreenshots.encode4BitGrayPng(data.byteArray(), width, height).data()
}
/** iOS retains/coalesces surface updates in the same compositor Android uses. */
class IosSurfaceCompositor(width: Int, height: Int) {
private val compositor = SurfaceCompositor().also { it.configureScreen(width, height) }
private val sizes = mutableMapOf<String, Pair<Int, Int>>()
private var sequence = 0L
fun configure(
id: String,
x: Int,
y: Int,
width: Int,
height: Int,
zOrder: Int,
transparent: Boolean,
) {
compositor.configureSurface(id, x, y, width, height, zOrder, if (transparent) 1 else 0)
sizes[id] = width to height
}
fun remove(id: String) {
sizes.remove(id)
compositor.removeSurface(id)
}
fun visible(id: String, visible: Boolean) = compositor.setSurfaceVisible(id, visible)
fun dim(below: Int, factor: Double) {
require(factor.isFinite())
compositor.setUnderlayDim(below, (factor.coerceIn(0.0, 1.0) * 256).toInt())
}
fun blank(blanked: Boolean) = compositor.setBlanked(blanked)
fun submit(id: String, data: NSData, x: Int, y: Int, width: Int, height: Int) {
val (sw, sh) = sizes[id] ?: error("Unknown surface: $id")
require(width > 0 && height > 0 && data.length == width.toULong() * height.toULong())
val left = maxOf(0, x)
val right = minOf(sw, x + width)
val top = maxOf(0, y)
val bottom = minOf(sh, y + height)
if (right <= left || bottom <= top) return
val reader =
if (left == x && top == y && right - left == width && bottom - top == height)
IosByteReader(data)
else {
val source = data.byteArray()
val clipped = ByteArray((right - left) * (bottom - top))
for (row in top until bottom) source.copyInto(
clipped,
(row - top) * (right - left),
(row - y) * width + left - x,
(row - y) * width + right - x,
)
ArrayByteReader(clipped)
}
compositor.submitSurface(
id,
reader,
left,
top,
right - left,
bottom - top,
(++sequence).toString(),
)
}
fun composite(): NSData = compositor.composite().gray.data()
}
@@ -0,0 +1,72 @@
package com.faceclaw.app
import kotlinx.cinterop.*
internal actual fun protocolPlatform(): ProtocolPlatform = IosProtocolPlatform
@OptIn(kotlinx.cinterop.ExperimentalForeignApi::class)
internal actual fun readFileData(path: String): ByteArray {
val file = platform.posix.fopen(path, "rb") ?: error("Cannot read $path")
try {
check(platform.posix.fseek(file, 0, platform.posix.SEEK_END) == 0)
val size = platform.posix.ftell(file)
require(size in 0..Int.MAX_VALUE.toLong())
platform.posix.rewind(file)
val bytes = ByteArray(size.toInt())
if (bytes.isNotEmpty())
bytes.usePinned {
check(
platform.posix.fread(it.addressOf(0), 1u, bytes.size.toULong(), file) ==
bytes.size.toULong()
)
}
return bytes
} finally {
platform.posix.fclose(file)
}
}
@OptIn(kotlinx.cinterop.ExperimentalForeignApi::class)
internal actual fun deflateData(bytes: ByteArray): ByteArray =
kotlinx.cinterop.memScoped {
val capacity = platform.zlib.compressBound(bytes.size.toULong())
val result = ByteArray(capacity.toInt())
val length = alloc<ULongVar>()
length.value = capacity
bytes.usePinned { input ->
result.usePinned { output ->
check(
platform.zlib.compress2(
output.addressOf(0).reinterpret(),
length.ptr,
if (bytes.isEmpty()) null else input.addressOf(0).reinterpret(),
bytes.size.toULong(),
platform.zlib.Z_DEFAULT_COMPRESSION,
) == platform.zlib.Z_OK
)
}
}
result.copyOf(length.value.toInt())
}
@OptIn(kotlinx.cinterop.ExperimentalForeignApi::class)
internal actual fun inflateData(bytes: ByteArray, size: Int): ByteArray =
kotlinx.cinterop.memScoped {
val result = ByteArray(size)
val length = alloc<ULongVar>()
length.value = size.toULong()
bytes.usePinned { input ->
result.usePinned { output ->
check(
platform.zlib.uncompress(
output.addressOf(0).reinterpret(),
length.ptr,
input.addressOf(0).reinterpret(),
bytes.size.toULong(),
) == platform.zlib.Z_OK
)
}
}
check(length.value.toInt() == size) { "recorded frame truncated" }
result
}
@@ -0,0 +1,129 @@
@file:OptIn(
kotlinx.cinterop.ExperimentalForeignApi::class,
kotlin.experimental.ExperimentalNativeApi::class,
)
package com.faceclaw.app
import kotlin.native.ref.createCleaner
import kotlinx.cinterop.*
import platform.CoreBluetooth.CBCharacteristicWriteType
import platform.CoreBluetooth.CBCharacteristicWriteWithResponse
import platform.CoreBluetooth.CBCharacteristicWriteWithoutResponse
import platform.Foundation.NSRecursiveLock
import platform.posix.CLOCK_MONOTONIC_RAW
import platform.posix.clock_gettime_nsec_np
import platform.posix.memset
import platform.zlib.*
object IosProtocolPlatform : ProtocolPlatform {
// Unlike wall time, this cannot jump; unlike uptime, it includes sleep.
override fun elapsedRealtimeMs(): Long =
(clock_gettime_nsec_np(CLOCK_MONOTONIC_RAW.convert()) / 1_000_000uL).toLong()
override fun createLock(): ProtocolLock =
object : ProtocolLock {
private val lock = NSRecursiveLock()
override fun lock() = lock.lock()
override fun unlock() = lock.unlock()
}
override fun createDeflater(): ProtocolDeflater = IosDeflater()
fun writeType(mode: GattWriteMode): CBCharacteristicWriteType =
when (mode) {
GattWriteMode.WITH_RESPONSE -> CBCharacteristicWriteWithResponse
GattWriteMode.WITHOUT_RESPONSE -> CBCharacteristicWriteWithoutResponse
}
}
/** The cleaner holds only the allocation state, never the owning deflater. */
private class ZlibState {
private val lock = NSRecursiveLock()
var stream: CPointer<z_stream>? = nativeHeap.alloc<z_stream>().ptr
private set
init {
memset(stream, 0, sizeOf<z_stream>().convert())
val status = deflateInit(stream, Z_DEFAULT_COMPRESSION)
if (status != Z_OK) {
nativeHeap.free(stream!!)
stream = null
error("zlib initialization failed: $status")
}
}
fun <T> use(action: (CPointer<z_stream>) -> T): T {
lock.lock()
try {
return action(checkNotNull(stream) { "zlib stream is closed" })
} finally {
lock.unlock()
}
}
fun close() {
lock.lock()
try {
val value = stream ?: return
deflateEnd(value)
nativeHeap.free(value)
stream = null
} finally {
lock.unlock()
}
}
}
private class IosDeflater : ProtocolDeflater {
private val state = ZlibState()
private val cleaner = createCleaner(state) { it.close() }
override fun reset() = state.use { check(deflateReset(it) == Z_OK) }
override fun syncFlush(message: ByteArray): ByteArray = state.use { stream ->
val chunks = mutableListOf<ByteArray>()
var size = 0
val buffer = ByteArray(4096)
message.usePinned { input ->
stream.pointed.next_in =
if (message.isEmpty()) null else input.addressOf(0).reinterpret()
stream.pointed.avail_in = message.size.toUInt()
try {
buffer.usePinned { output ->
do {
stream.pointed.next_out = output.addressOf(0).reinterpret()
stream.pointed.avail_out = buffer.size.toUInt()
val status = deflate(stream, Z_SYNC_FLUSH)
// A repeated empty SYNC_FLUSH is a valid no-op.
check(status == Z_OK || status == Z_BUF_ERROR) {
"zlib deflate failed: $status"
}
val count = buffer.size - stream.pointed.avail_out.toInt()
if (count > 0) {
chunks.add(buffer.copyOf(count))
size += count
}
} while (stream.pointed.avail_out == 0u)
}
} finally {
// Do not leave pointers to unpinned Kotlin arrays in native memory.
stream.pointed.next_in = null
stream.pointed.next_out = null
stream.pointed.avail_in = 0u
stream.pointed.avail_out = 0u
}
}
ByteArray(size).also { result ->
var offset = 0
for (chunk in chunks) {
chunk.copyInto(result, offset)
offset += chunk.size
}
}
}
override fun close() = state.close()
}
@@ -0,0 +1,13 @@
package com.faceclaw.shared
import platform.Foundation.NSThread
internal actual object PlatformInfo {
actual val name: String = "iOS"
actual fun threadName(): String =
NSThread.currentThread.name?.takeIf { it.isNotEmpty() }
?: if (NSThread.isMainThread) "main" else "background"
actual fun isMainThread(): Boolean = NSThread.isMainThread
}