arXiv preprint ·
te-comp: A Complete MLIR Compiler Generating Self-Contained C Inference Engines for MCUs
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Deploying deep learning on microcontrollers (MCUs) requires fitting both the model weights and runtime activations within tens to hundreds of kilobytes of SRAM and Flash. Existing approaches either use an interpreter with a pre-written kernel library or combine a Python code generator with hand-written C kernels. This incurs dispatch overhead and requires framework dependencies. None produces self-contained, dependency-free C code specialized to a given model. In this paper, we present te-comp, the first complete MLIR-based compiler that takes a standard model in .tflite or .safetensors format and emits all C source files needed for deployment: the inference engine (kernels, weight arrays, memory layout) and the RTOS integration layer with zero runtime dependencies. On 10 MCUNet CNN benchmarks, te-comp outperforms MCUNet TinyEngine by using 38% less SRAM (up to 5.9$\times$) and 15% less Flash, while generating complete C code rather than linking a pre-written kernel library. To demonstrate, we use te-comp to compile an 8.3M-parameter GPT-Neo transformer (TinyStories-v2) and deploy it on a single-core Cortex-M7 at 160MHz. The full firmware uses 249KB of 512KB SRAM and 3.62MB of 4MB Flash and achieves approximately 1 token/s interactive text generation via Q4_0 weight quantization, a stateful int4 KV cache, and vocabulary pruning -- a capability no existing MCU inference framework supports.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Programming Languages
XLA from the Trenches: Correctness on TPU Workloads at Scale
Extensible Programmatic Specification of Finite Element Methods
WFDB-x: Conformance-Tested Implementations of the WFDB Software Package in Seven Programming Languages
MBUndo: Sound, Automatic, and Efficient Measurement-Based Uncomputation for Quantum Circuits
Reasoning about Shared-Qubit Concurrency in Quantum Computing
Get the few papers worth your time in Programming Languages every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.