From 8d78af12465eb96842cbb4c9b82b402fac082c8d Mon Sep 17 00:00:00 2001 From: jimmy Date: Tue, 27 Oct 2026 22:30:26 +1300 Subject: [PATCH] docs(v2): bench guide results for the loss, beat, USB and C3 tests Adds the audio endpoints, the Makefile pointer, the first-run results of the lossy-link and beat tests, the USB and analyzer latency methods and the ESP32-C3 comparison (delivery, latency, frame rate, Wi-Fi start-up order). Co-Authored-By: Claude Sonnet 5.5 --- v2/docs/BENCH_TEST.md | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/v2/docs/BENCH_TEST.md b/v2/docs/BENCH_TEST.md index b5741d0..24d01fa 100644 --- a/v2/docs/BENCH_TEST.md +++ b/v2/docs/BENCH_TEST.md @@ -19,6 +19,8 @@ then *ESP-NOW config → Wi-Fi channel*. Alternatively set `CONFIG_ESPNOW_CHANNE ## 1. Build and flash the controller +The repository root has a Makefile that wraps all of this: `make build`, `make flash`, `make monitor` (Ctrl-] to quit), `make log` (console for 10 s, no terminal needed), `make test`, and `make help` for the rest; `PORT=/dev/ttyACM1` selects another port. The manual commands below do the same thing. + ```bash cd v2 tools/build_web.sh @@ -54,6 +56,8 @@ Expect JSON with `uptime_s`, `bpm`, an `espnow` block and a `link` block (`ver` curl -s -D - -o /dev/null -H 'Accept-Encoding: gzip' http://$S3/static/presets.js | grep -i -E 'content-encoding|content-type' ``` +**Audio and beats.** `http://$S3/audio` is the mic page: live levels, tempo and bar position, a tap button to correct beat 1, and a Listen button for the 16 kHz mono stream. The underlying endpoints are `/audio.wav` (one listener at a time), `/beats` (server-sent events: `beat`, `onset`, `tempo`), `POST /tap` and `/time`. `/status` carries the mic levels (`mic_rms_dbfs`, `mic_spl_db`) and the beat state (`bpm`, `beat_locked`, `meter`, `bar_beat`, `bar_sure`). + ## 3. Load data and add a test zone Import the Pi database (the files hold your real zones and presets, so use a copy if you want a smaller test): @@ -232,8 +236,45 @@ Reading it: the double send plus free-run gave the driver a beat every time, eve Latency over USB (200 beats at 120 BPM, one driver, 119 LEDs): delay beyond the 30 ms lead min 0.49, mean 2.07, median 1.77, p95 4.41, max 7.95 ms (std 1.3 ms; accuracy about 1 ms). 197 of 200 beats matched. Both clocks drifted by 60 to 80 ppm against the PC, and individual USB lines arrived up to 54 ms late, which is why the script uses the fastest deliveries. This is the time from the controller sending a beat to the driver firing, beyond the built-in lead. The logic analyzer run has not been done. +### ESP32-C3 driver compared with the S3 driver + +Driver: XIAO ESP32-C3 (MAC 58:8c:81:a3:74:58), MicroPython v1.22.1 stock (no ws2812-dma patch), 160 MHz, about 128 KB free heap, 119 LEDs on pin 10. Same tests as above, same controller. + +| | S3 driver, no loss | C3 driver, no loss | +|---|---|---| +| echoes the controller received | 162 of 162 | 160 of 160 | +| round trip minus the 30 ms lead, min / avg / max | 9.8 / 15.0 / 25.7 ms | 2.6 / 4.5 / 17.8 ms | + +C3 with 30% loss (160 beats): every beat produced exactly one pattern step, 15 of the 160 (9%, theory 9%) came from free-run, and 68 second copies were dropped as duplicates; the controller got 99 of 160 echoes. That matches the S3 result in the table above. + +Reading the free-run counter: after the generator stops, the driver keeps stepping for up to 4 more beats on its last period. A stats window that runs past the end of the test therefore shows 4 extra "free-run" fires even with no loss. They are not missed beats. (My first read of the C3's no-loss run said otherwise; that was wrong.) + +Latency over USB, same settings on both boards (`usb_log` only), 200 beats at 120 BPM, delay beyond the 30 ms lead, accuracy about 1 ms: + +| | matched | min | mean | median | p95 | max | std | +|---|---|---|---|---|---|---|---| +| S3 driver (240 MHz) | 200 / 200 | -2.4 | 2.15 | 2.03 | 6.03 | 12.3 | 2.5 ms | +| C3 driver (160 MHz) | 200 / 200 | 0.2 | 3.36 | 3.06 | 7.47 | 19.7 | 2.3 ms | + +The C3 is about 1.2 ms slower on average and has a longer tail (maximum 19.7 ms against 12.3 ms). With the method's 1 ms accuracy, the mean difference is small and the tail difference is the more telling one. Both are well inside a 120 BPM beat. The echo round trip disagrees with this (S3 15 ms mean, C3 4.5 ms). The S3 echo runs had `usb_log` and a marker pin switched on and the C3 runs did not, and I did not rerun the S3 echo test with identical settings, so treat the echo figures as unexplained and prefer the USB ones. + +Pattern frame rate (strip writes per second, 119 LEDs, delay 1 ms, 2 s each; only patterns that wrote anything are shown): + +| pattern | C3 (160 MHz, 115 KB free) | S3 (240 MHz, 8 MB free) | +|---|---|---| +| flame | 22 | 33 | +| palette_morph | 21 | 32 | +| circle | 150 | 170 | + +The CPU-heavy patterns run about 1.5 times faster on the S3, which is the clock ratio; a light pattern is limited by the strip write and differs by about 13%. `aurora` and `chase` wrote nothing with the parameters I used, so they are not measured. + +Groups: the controller sends a driver its group list when the driver announces itself, but now only if the driver is in at least one zone. A driver in no zone keeps the groups it already has (an empty list from the controller used to wipe them). An explicit zone or device edit still re-sends every list, including empty ones. + +**Wi-Fi start-up order on the C3.** With the extra link code loaded before Wi-Fi start-up, the C3 failed with `Wifi Unknown Error 0x0101` on about every other cold boot (2 of 4). Starting ESP-NOW before the pattern library and the LED driver claim memory fixed it (6 of 6 boots). `led-driver/src/main.py` now does this. + ## 8. Troubleshooting +- **A board on a USB-serial/JTAG port (C3, and the S3 controller) resets whenever a program opens its port,** and a C3 can land in ROM download mode (`wait usb download`). The old apt `mpremote` 0.4.0 hangs on this. Use the newer one in `~/projects/micropython/micropython/tools/mpremote/mpremote.py`, or `led-tool`, and note that `fs cp -r` fails on drivers that print timestamps (copy named files instead). - **Running a long `mpremote exec` on the driver resets it:** the driver's watchdog (10 s) keeps running after `mpremote` interrupts `main.py`, so keep REPL experiments under about 8 s and reset the driver afterwards. - **Driver never reacts, no acks:** channel mismatch. Check `CONFIG_ESPNOW_CHANNEL` against the driver's `wifi_channel`. Neighbouring channels can still work at bench range, so a working link does not prove they match.