Skip to content

TODO.md

Latest commit

 

History

History
101 lines (73 loc) · 3.06 KB

TODO.md

File metadata and controls

101 lines (73 loc) · 3.06 KB
Raw

To-Dos:

NOTE:

  • Project has been determined to not be deployable to MCU in current form and I have to step away.
  • I am leaving these here just for archival purposes.
  • Check out the plan.md file to see the goals at a larger bird's eye view.

Organization:

  • Each set of Todos that I leave myself will be loosely tied to a goal from plan.md.
  • There is likely to be some bleed between these goals, as things are rarely built completely linearly. I will try to place these smaller to-dos as tasks of sub-goals where it makes sense. However, I want to also keep a trail of my process that is honest.

List of Goals

Project Setup

  • Get UV setup for python and workflow scripts
  • Setup pre-commit hooks
  • Search for models to work off of (GTCRN)
  • setup datasets for offline evaluation
  • Test PyTorch Converters
  • Setup datasets for training
  • Setup basic .sh scripts/makefile
  • Scope out datasheets for $I^{2}S$ Protocol
  • Setup offline evaluation metrics in /python

ESP32

Model Design

  • Reimplement model replacing GRUs and for TFLM
  • Make model adjustments to comply with Torch $\rightarrow$ ONNX $\rightarrow$ TFLite $\rightarrow$ TFLM
  • Fix parameter replacement for JSON
  • Train new model
  • Setup Streaming converter for chosen architecture
  • Test and report performance for streaming model in PyTorch and ONNX format
  • Create QAT training setup to run on HPC
  • Add examples of how to run the model in main README

Export and Quantization

  • Convert PyTorch Adjusted model with TFLite Micro using PTQ
  • Report results of streaming TFLite model
  • Report measure of # of params and MMACs, ensure can fit on-device or adjust
  • Profile and measure accuracy of quanitzed model

Deployment

NOTE: Subject to change to only MCU dependent on time

  • Setup build toolchain for ESP32-S3 deployment
  • Setup $I^2S$ for ESP32
  • Write SE inference program in C for MCU using .tflite model
  • Get performance for both models in terms of Latency, Power Consumption, Accuracy

STM32

Model Design

  • Reimplement model for Cube AI supported Ops
  • Test out STM32 Cube AI conversion
  • Train new model (once shapes are confirmed)
  • Setup Streaming converter for chosen architecture
  • Create QAT training setup to run on HPC
  • Add examples of how to run the model in main README

Export and Quantization

  • Export trained model with CubeAI
  • [] Report measure of # of params and MMACs, ensure can fit on-device or adjust
  • Profile and measure accuracy of quanitzed model

Deployment

  • Setup build for STM32
  • Setup $I^2S$ for STM32
  • Write SE inference program in C for MCU
  • Get performance for both models in terms of Latency, Power Consumption, Accuracy