Dart vector_kit cuts semantic search from 15.9ms to 1.4ms using SIMD and packed memory

A Flutter developer building an on-device semantic search feature over 20,000 documents found the naive List-of-lists approach took 15.9ms per query, too slow for real-time keystroke re-ranking. The bottleneck was not the linear scan algorithm but the fragmented heap memory layout, where each embedding was a separate object with boxed floats requiring pointer chasing. Switching to the vector_kit package, which stores all embeddings in a single contiguous Float32List and uses SIMD dot products, reduced query time to 1.4ms — an 11x speedup with identical results. For memory-constrained mobile devices, the package also offers an int8 QuantizedMatrix that shrinks a 29.3MB float32 index to 7.6MB, with tests on the demo dataset showing 100% recall@10 against the full-precision ranking. The author notes recall quality is data-dependent and may vary when embeddings are less well-separated.