Developer Claims 9.9x Lower TTFT on Android by Reusing llama.cpp KV State

A developer has reported achieving a 9.9x reduction in time-to-first-token (TTFT) for local large language model inference on a real Android device. The improvement was achieved by reusing KV cache state in llama.cpp, a popular framework for running LLMs on consumer hardware. The project, called EdgeSync-LLM, is publicly available on GitHub for others to test and reproduce. The developer is specifically inviting engineers working on llama.cpp, KV cache management, and edge AI to attempt to reproduce or challenge the benchmark results. They have stated that identifying flaws in the methodology would be more valuable to them than simple endorsements of the work.