Edge AI Lab Notes: The Data Between Training and Device

Most edge AI guides are about the model: how to shrink it, convert it, load it onto a device and make it run faster. They rarely cover what happens around the model. That includes the recorded sensor data, the code that cuts it into windows, the file it is stored in, and the point where a model trained in Python meets firmware written in C and the two quietly disagree about what “a window of data” means.

That part is where most of my time on these projects has actually gone, and its failures do not crash. They produce answers that look reasonable and are wrong, and it is easy to spend weeks blaming the model. This series shows how to set up a project so that these mistakes make a build fail, instead of slipping through unnoticed.

Nothing here depends on a particular product. It applies whether you are building a fall detector, a presence sensor, a machine that predicts equipment failure or a voice command detector. You do not need to be an expert to follow it: each post explains the terms it uses.

The companion code runs on an ordinary laptop, with no sensor board and no dataset to download. It uses a generator that produces synthetic data with the same properties as real sensor recordings. Every benchmark result was measured on a Raspberry Pi 5 with 8 GB of memory, running Raspberry Pi OS Bookworm from an SD card, and each run records the processor temperature so that a slowed-down, overheated run cannot pass for a real result. The raw results are in the repository alongside the code.

The series is written to be read in order. Each part builds on the one before it: the contract defines the data, the file format preserves it, and the query engine checks it. If you already have a working pipeline and only want the measurements, start with Part 2, where the benchmarks begin.

Part 4, comparing Parquet and Arrow, is in progress.