Dynamic Tracing Beats Static Analysis for Linking Tests to Code in RAG Search

A developer building a RAG-based search tool for unfamiliar codebases found that static analysis alone could not reliably identify which functions a given test actually executes. Name-based matching failed entirely on a sample of 109 tests, while file-level import matching reached only 77.9% accuracy — too coarse to be useful. A custom dynamic tracing plugin using Python's sys.settrace was run across 1,727 tests, revealing that each test executes an average of 10.1 source functions, debunking the assumption of a one-to-one test-to-function relationship. The Tarantula heuristic for ranking primary targets was tested but refuted, covering only 22.6% of tests at rank three or better, though it proved useful as a confidence annotation for roughly 16% of cases. The final pipeline produced 16,172 test-to-function edges integrated into a property graph, with test signals now surfaced during search at a reduced relevance weight to complement standard code definitions.