Code for MobiCom'22 paper 'TinyML-CAM: 80 FPS Image Recognition in 1 Kb RAM'

Go to file

Bharath Sudharsan 1bacaba9d1 added code		2022-07-23 04:08:11 +01:00
ESP32-image-object-classification-live-demo.mp4	Added demo video	2022-07-18 05:57:39 +01:00
LICENSE	Initial commit	2022-07-18 05:31:50 +01:00
README.md	added code	2022-07-23 04:08:11 +01:00
[h]-HOG-plus-RandomForest-classifier.h	added code	2022-07-23 04:08:11 +01:00
[ino]-arduino-ESP32-code.ino	added code	2022-07-23 04:08:11 +01:00
[ipynb]-TinyML-CAM-full-code-with-markdown.ipynb	added code	2022-07-23 04:08:11 +01:00

TinyML-CAM - Image Recognition System that Runs at 80 FPS in 1 Kb of RAM

ESP32 classifying Raspberry Pi Pico, Portenta H7, Wio Terminal from image frames

Following can be observed from the video:

Time For image frames, the digital signal processing (DSP) based features extraction time is ≈ 12 ms, while classification time is ≈ < 20 𝜇𝑠 (1/1000^th of DSP).
FPS It is 1000/12 ms = 83.3 FPS, which is the time taken by the TinyML-CAM image recognition system to process (DSP) plus classify using a single image frame. Since the ESP32 has a 30 FPS frame rate, just to capture frames, it takes 1000/30 = 33 ms. So the entire frame rate is 1000/(33+12) = 22 FPS.
Accuracy As expected during Pairplot analysis, Portenta and Pi (features overlapped) are mislabelled quite often, which can be rectified by improving dataset quality.
Memory Consumes only 1 kB of RAM - difference between the RAM calculated by Arduino IDE before and after adding the TinyML-CAM image recognition system.