Training LLMs on GitHub Commit History Could Teach AI Real Engineering Reasoning

·1 views

Current large language models are trained on billions of lines of finished code from GitHub, but they only learn the final output rather than the decision-making process behind it. A developer argues this is a fundamental gap, comparing it to teaching an architect solely through photos of completed buildings without explaining the reasoning behind design choices. The proposed solution is to train models on full commit histories of major open-source projects like Linux, PostgreSQL, and Rust, so they can learn how and why code evolved over time. Such training could help AI understand refactoring, error handling evolution, and architectural trade-offs rather than simply mimicking clean code patterns. The author acknowledges that most commit data is noisy but suggests focusing on high-quality, well-maintained repositories to extract the most meaningful engineering lessons.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Separating LLM Prefill and Decode Phases Cuts Token Latency by 66%

Large language model inference involves two distinct workloads: a compute-heavy prefill phase that processes the entire prompt, and a latency-sensitive decode phase that generates tokens one at a time. When both phases run on the same hardware, a long-context prefill can stall all active decode streams on that engine, a problem known as head-of-line blocking. Prefill/decode disaggregation addresses this by routing each phase to dedicated server pools, preventing prefills from ever queuing ahead of decodes. A Go-based simulation demonstrated that splitting these pools reduced p99 inter-token latency from 88ms to 30ms, a 66% improvement, at the cost of a modest increase in time-to-first-token. This architectural pattern is already adopted in production serving systems such as DistServe, Splitwise, and vLLM with Mooncake.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Deterministic Simulation Testing Can Reproduce and Shrink Elusive Agent Bugs

A class of hard-to-catch software bugs in AI agents only surfaces when faults occur in a specific sequence — for example, a retry firing after a side effect causes a customer to be charged twice. Deterministic simulation testing (DST) addresses this by routing all sources of nondeterminism, such as faults, timing, and randomness, through a single seed, making any failure perfectly reproducible. The technique, used by systems like FoundationDB and TigerBeetle, also supports shrinking, which strips a complex failing scenario down to its minimal root cause. A Python demonstration showed that standard happy-path tests missed a double-charge bug, while seeded fuzzing caught it, replayed it identically, and reduced a four-fault sequence to the single fault responsible. DST is gaining broader attention as a practical method for testing agents that operate in unpredictable, fault-prone environments.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Speculative Tool Execution Cuts AI Agent Latency by Running Tools in Parallel

AI agents follow a strictly serial loop of reasoning, calling a tool, waiting for results, then reasoning again — leaving GPUs idle for up to 61% of wall-clock time in tool-heavy workloads. A technique called speculative tool execution addresses this by predicting the next tool call and running it in parallel while the model is still reasoning. If the prediction matches the model's actual call, the result is already available and latency is eliminated; if not, the speculative result is discarded and the correct tool runs instead. A Go-based demo using a learned pattern predictor achieved a 58% hit rate, reducing wall-clock time by 1.3x with no impact on output correctness. Four research papers published in 2026 — including Speculative Actions, SPORK, PASTE, and Speculate While You Reason — report real-world speedups of 20 to 48% using similar approaches.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Sleep-Time Compute Cuts AI Agent Latency by Pre-Answering Queries While Idle

Researchers at Letta (Lin et al., 2025) have proposed a technique called sleep-time compute, which shifts AI inference work to idle periods between user sessions rather than processing everything on demand. A background worker pre-answers queries likely to be asked again and compresses standing context into dense summaries, so the system can serve warm, instant responses when users return. Each pre-computed answer is tagged with the source version it was derived from, and a freshness check ensures stale answers are discarded and recomputed rather than served to users. In a demonstration across 400 queries — 70% predictable and 30% novel — foreground latency dropped by 57% and foreground cost fell by over half, while novel queries were still handled live. The approach is particularly suited to workflows where users repeatedly query the same documents or codebases, since the underlying context changes infrequently between sessions.

0 comments Read more at DEV Community