Two lines of Python caching cut a 1M-row script runtime from 3.71s to 1.20s
A developer optimized a Python script processing one million sales rows by identifying date-string parsing as the primary bottleneck, consuming 36% of total runtime. Profiling revealed that despite one million calls to Python's strptime function, only 365 distinct date strings existed in the dataset. Adding just two lines using functools lru_cache reduced strptime executions from one million to 365, achieving a 3.1x speedup while producing byte-for-byte identical output. Benchmarks showed the technique delivered up to 2.77x gains when input values repeated frequently, but imposed a slight 6% overhead when all one million dates were unique. The experiment highlights that caching pure functions is effective only when inputs repeat — profiling first to confirm repetition is essential before applying the fix.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in