Lumen
1. Motivation: Forcing System State Onto the Desk
Lumen started from a simple failure mode: system and application state stayed trapped behind windows and logs. Nothing
on the desk moved, blinked, or sounded different when builds broke, power spiked, or motion changed. The workspace
stayed visually flat even when the machine was on fire.
The goal for Lumen was to clamp on that gap and build a desk-first, programmable hardware presence:
- Link software and system state to physical feedback: lights, motion, sound, display.
- Keep the device useful even when the host PC is detached.
- Avoid custom programmers and opaque tools so users don’t get blocked before the first boot.
That forced three constraints:
Fully open source, firmware-level programmable.
No “vendor firmware” blob that can’t be rebuilt.Straightforward build and flash flow.
A user with a browser should be able to flash CI-produced binaries through WebUSB without installing a toolchain.Long-term desktop value over peak performance.
If a small performance win broke reproducibility or required special jigs, it got dropped.
These constraints shaped everything: the stack, the drivers, and how the CI pipeline was wired.
2. Project Framework Design
2.1 Hardware Platform and System Layout
The hardware (v2) locks in the capabilities and therefore the firmware structure:
- MCU: ESP32‑C3
- Sensors and IO:
- LSM6DSO IMU
- INA226 current sensor
- Buzzer
- Rotary encoder
- Display:
- 240×240 panel, ST7789‑like, driven via
esp_lcd
- 240×240 panel, ST7789‑like, driven via
- Power and connectivity:
- USB‑C
- Power switch
This stack forced a design where display throughput, motion sampling, power monitoring, and host I/O all run
concurrently without clobbering each other, under FreeRTOS.
2.2 Firmware Stack: FreeRTOS Core, Layered Drivers
At the firmware level, Lumen runs on ESP‑IDF v5.5 with FreeRTOS. The project is structured in layers rather than one
monolithic main.c:
Hardware drivers (C/C++):
- Display panel via
esp_lcd - I2C plumbing for LSM6DSO and INA226
- Rotary encoder and buzzer
- USB serial‑JTAG interface
- Display panel via
System integration and logic:
- FreeRTOS task graph
- Motion task
- UI rendering pipeline
- Host serial protocol
Rust no‑std component (main_app):
- Sits only in the system integration layer.
- Does not replace the C/C++ hardware drivers.
- Avoids a split stack where half the drivers live in one language and half in another.
Rust was pulled in specifically where it helped structure higher‑level integration logic, not to rewrite vendor SDK
surfaces. That constraint prevented a steady stream of build and tooling failures from dual ecosystems fighting each
other.
2.3 UI Stack: Vision‑UI on u8g2 on ST7789
The display stack uses a layered approach to keep UI changes from breaking lower layers:
- Vision‑UI: a small embedded UI library.
- u8g2: provides a mono framebuffer and drawing primitives.
- Panel driver: ST7789‑like display, driven via
esp_lcdAPIs.
Implementation lives in main/src/ui_hardware_driver.cpp:
- A global
PANELhandle ties intoesp_lcd. - The UI code only sees a framebuffer and a driver callback to “send buffer,” not the exact panel timing.
This wiring means:
- UI layout changes do not force display driver changes.
- Display driver swaps stay local to the
esp_lcdlayer. - The top‑level logic doesn’t need to know about DMA allocations or RGB565 conversions.
2.4 Build and Release: CI‑First, Web Flash Second
The build system is pinned down in .github/workflows/build.yml:
- GitHub Actions installs:
- ESP‑IDF v5.5
- A minimal Rust toolchain
- The RISC‑V target for ESP32‑C3
- CI runs
idf.py buildand produces firmware artifacts.
Those artifacts feed a web flashing flow:
- Browser + WebUSB grabs the CI artifact and flashes the board.
- No local ESP‑IDF or Rust installation is required for basic users.
This flow looked heavier than a one‑off local build, but it removed a constant failure mode: users blocked by toolchain
setup. All complexity is clamped into CI instead of onto every developer’s machine.
3. Overhead Balance and Tradeoffs
3.1 Reproducibility vs. Performance
Several design choices pulled away from “fastest possible” and toward “reproducible and survivable”:
Using ESP‑IDF v5.5 and
esp_lcdadds abstraction and some overhead vs. hand‑tuned register banging, but:- It keeps code aligned with upstream.
- It avoids a fragile, undocumented display driver that breaks when the next board spin rearranges pins.
Driving the display through a u8g2 mono framebuffer and then converting to RGB565 is not the fastest route:
- A direct color framebuffer could shave steps.
- But it would lock UI code into a specific panel format.
- The chosen route makes assets and widgets less brittle when the panel or bus changes.
The README explicitly notes that squeezing the last bit of performance did not win over long‑term desktop usefulness and
reproducibility.
3.2 Independence from the Host PC
Another hard constraint: the device must hold value even when unplugged from any PC.
Two consequences:
Core logic cannot depend on serial I/O:
- The serial pack protocol (host interaction) is optional.
- Motion processing, display updates, and protection logic run without any host.
UI decoupled from top‑level logic:
- Input handling and UI rendering can change without rewriting the control loop.
- Firmware doesn’t stall if the serial channel drops or a host app misbehaves.
This avoids a failure mode where an absent or crashed host application bricks device usefulness.
3.3 Web Flashing Limitations
The web flashing path trades one class of problems for another:
Dependency on WebUSB support:
- Some browsers and platforms simply do not support it.
- In those cases, the user is forced back to local tooling.
No CI caching yet:
- The current CI build workflow does not cache IDF or build artifacts.
- Every build re‑does configuration and compilation.
- Builds therefore take longer than necessary.
The README calls these out instead of hiding them. Build times can be improved later by inserting caching or diff
builds, but correctness and repeatability were prioritized first.
4. Display Pipeline: Double‑Buffered DMA Under Pressure
4.1 Design Constraint
The display needed to refresh a 240×240 panel without stalling the MCU or blocking other tasks. Direct full‑frame copies
would overflow time budgets and starve motion and serial tasks.
To avoid that, the implementation uses line‑based double buffering with DMA‑capable memory:
- Defined in
main/src/ui_hardware_driver.cpp. - Globals and constants:
S_LINES[2]: two DMA‑capable line buffers.S_BUF_BUSY[2]: flags to track which buffer is in-flight.PARALLEL_LINES: number of lines per chunk; set to 128.PANEL:esp_lcd_panel_handle_t.
4.2 Pipeline Behavior
Initialization (
displayInit):- Allocates
S_LINES[0]andS_LINES[1]withheap_caps_mallocusingMALLOC_CAP_DMA | MALLOC_CAP_INTERNAL. - Sets up the panel via
esp_lcd, including a callback:onColorTransDone: fires when DMA completes for a chunk.
- Allocates
Rendering (
vision_ui_driver_buffer_send):- UI code provides a mono u8g2 framebuffer.
- The driver:
- Splits it into RGB565 blocks of
PARALLEL_LINESheight. - For each block:
- Finds a non‑busy buffer (
S_BUF_BUSY[i] == false). - Converts mono data into RGB565 into that buffer.
- Calls
esp_lcd_panel_draw_bitmaponPANELfor that chunk. - Marks buffer busy.
- Finds a non‑busy buffer (
- Splits it into RGB565 blocks of
DMA completion (
onColorTransDone):- Clears the corresponding
S_BUF_BUSY[i]. - Signals that chunk buffer is ready to be reused.
- Clears the corresponding
This design keeps the CPU from sitting on long blocking writes. Conversion and DMA transfer are overlapped: while one
buffer is in flight, the other can be filled.
4.3 Tradeoffs
Pros:
- Reduced stalls on the display path.
- Predictable memory footprint via
PARALLEL_LINEScontrol. - Easier tuning for future panels.
Cons:
- Extra copy step: mono → RGB565.
- More state to track (
S_BUF_BUSY, callbacks). - Harder debugging when transfers misalign.
The double‑buffering logic is deliberately localized in ui_hardware_driver.cpp so that higher layers don’t need to
know about DMA or panel quirks.
5. Motion Processing: IMU Task and Filters
5.1 Motion Task Setup
Motion handling sits in its own FreeRTOS task to avoid blocking UI or serial operations.
Implementation: main/src/motion.cpp.
Key pieces:
- IMU: LSM6DSO
- I2C access: shared handle, with wrappers:
imuWriteimuReadimuWriteThenRead
- Task:
motionInitsets up IMU and spawnsmotionTaskwithxTaskCreate.- Target rate: roughly 25 Hz.
5.2 Orientation Estimation
Two filter paths run on the sensor data:
2‑state Kalman filter (
espp::KalmanFilter<2>,S_KF):- Tracks roll and pitch.
- Smooths noisy accel/gyro inputs.
Madgwick filter (
MADGWICK):- Provides full orientation (quaternion).
- Fuses accelerometer and gyroscope data.
5.3 Mounting Fix and Accel Scaling
The physical mounting ended up upside‑down relative to the IMU’s default orientation. That broke orientation output
early on. The fix:
- Axis flip:
- Y and Z axes are flipped.
- Gyro axes are also flipped accordingly.
Without this correction, roll/pitch estimates were mirrored and drifted in unexpected ways.
To keep the acceleration magnitude sane:
- Adaptive accel scaling:
- Normalizes magnitude to approximately 9.80665 m/s².
- Prevents gravity from drifting away due to calibration offsets.
This scaling avoids overflow in the filters and reduces long‑term drift.
5.4 Runtime Behavior
The motion task:
- Samples the IMU at ~25 Hz.
- Runs the Kalman and Madgwick updates.
- Exposes motion/pose data internally (and via host protocol).
By isolating this in one task, any stalls in host communication or display updates do not directly block sensor reads.
If CPU load increases, sampling may drop somewhat, but the rest of the system keeps running.
6. Serial “Pack” Protocol: Host Integration Without Entanglement
6.1 Requirements
Host connectivity needed to:
- Support structured commands and payloads.
- Avoid a tangled binary protocol that blocks easy debugging.
- Stay optional: no hard dependency for core firmware behavior.
6.2 Protocol Structure
The serial pack protocol lives in main/src/serial_pack.cpp and runs over the USB serial‑JTAG interface.
Basic flow for each message:
Path:
- ASCII string.
- Terminated by newline (
\n). - Example:
motion/data\n.
Length:
- 4‑byte little‑endian integer.
- Specifies payload size.
Payload:
- Binary blob of that length.
A handler table routes messages based on the path.
6.3 Implementation Details
Constants:
K_MAX_HANDLERS = 2
Small fixed table; keeps memory fixed and simple.K_MAX_DATA_LEN = 2048
Hard clamp to prevent payloads from blowing up buffers.K_RX_TIMEOUT_US = 3 * 1000 * 1000
3‑second receive timeout.
Main functions:
serialPackInit
Sets up state.serialPackAttachHandler
Registers handler callbacks:- Up to
K_MAX_HANDLERS. - Each handler keyed by path string.
- Up to
serialPackStart/serialPackTask
Runs the read loop:- Reads the path until newline.
- Parses the 4‑byte length.
- If
length > K_MAX_DATA_LEN, the message is effectively rejected. - Reads the payload in chunks:
- Dispatches chunks incrementally to handlers.
- Applies timeout via
K_RX_TIMEOUT_US.
- Optional logging/preview to help debugging.
6.4 Tradeoffs and Failure Modes
Low handler count (
K_MAX_HANDLERS = 2):- Forces consolidation of endpoints.
- Prevents unbounded growth of handler registrations.
Payload clamp (
K_MAX_DATA_LEN = 2048):- Protects against memory blow‑ups if a host goes rogue.
- Blocks large streaming use cases unless the protocol is extended.
Timeout (
K_RX_TIMEOUT_US = 3s):- Prevents the parser from hanging forever on partial messages.
- Under poor USB conditions, packets might get dropped or truncated when the timeout triggers.
By keeping the protocol text‑framed on the path and length‑delimited on the payload, debugging is still manageable with
simple serial tools.
7. Features and Observed Results
7.1 Hardware Feature Set (v2)
From the README and hardware directory, v2 boards include:
- Display: 240×240 ST7789‑like panel.
- Motion: LSM6DSO IMU.
- Power: INA226 current sensor on USB‑C path.
- Interaction:
- Rotary encoder.
- Buzzer.
- Power control: physical power switch.
This combination gives enough axes—visual, motion, and sound—to map varied system state into desk‑visible behavior.
7.2 Implemented Firmware Capabilities
Firmware features visible in the repository:
USB‑C power monitoring and protection:
- INA226 sensor used for current/voltage measurements.
- Protection logic clamps down on unsafe conditions (details wired into the system logic).
Motion tracking:
- IMU readout with Kalman and Madgwick filters.
- Mounting fix applied for orientation correctness.
UI rendering via Vision‑UI:
- Framebuffer driven UI.
- Display assets embedded in firmware:
ui_assets_provider.cppprovides embedded images/fonts for Vision‑UI.
Host communication:
- Serial pack protocol over USB serial‑JTAG.
- Structured path + length framing.
Web quick flashing:
- CI‑produced artifacts used directly by the web flasher.
- No local toolchain needed for basic flashing.
7.3 Runtime Behavior and Observations
In combined operation, the system:
- Refreshes the 240×240 panel with double‑buffered DMA without starving FreeRTOS tasks.
- Samples motion at ~25 Hz while filtering orientation in real time.
- Streams motion and state over the serial pack protocol when a host is present.
- Continues to function (UI, motion, basic behavior) without any serial host connected.
Overhead is clearly present: layered UI, filters, serial protocol, CI scaffolding. But each layer was added in response
to a concrete failure mode—tooling blockage, display stalls, incorrect orientation, unsafe power behavior—rather than as
abstract design.
Lumen ends up as a reproducible, firmware‑level programmable desk companion that ties system state into light, motion,
and sound without depending on a constant host tether.