Pocket computers and first-PCB build stories are attracting attention today. For an AI-enabled edge device, a demo while plugged into Wi-Fi proves very little.
Define a test envelope before comparing models:
power: [100-percent, 20-percent, thermal-throttle]
network: [wifi, high-latency, offline, reconnect]
permission: [granted, revoked-during-task]
lifecycle: [foreground, screen-off, restart]
model: [bundled, downloaded, corrupt-update]
For every combination that represents a real user journey, record task success, p50/p95 latency, energy consumed, peak temperature, memory, bytes transferred, and recovery outcome. Averages hide the failure a person remembers.
The device must make placement visible: which operations stay local, which require cloud access, what is retained, and how queued work is cancelled. Test permission revocation mid-recording and connectivity loss after an upload begins. On restart, the UI should distinguish resumed, safely failed, and unknown state.
Use a power monitor when possible; battery percentage is too coarse for short tasks. Pin firmware, model, quantization, ambient temperature, and screen brightness so another tester can reproduce the envelope.
An edge-AI release is credible when it describes the operating boundary, not when it shows one fast happy-path demo.