Did the model fail, or did the test environment? Memory limits can flip results

A program that totals costs from a CSV returned the correct sum under a 256 MiB memory limit but stopped with MemoryError under 64 MiB, even though the input and code were unchanged. The author argues that a simple pass or fail in an evaluation table cannot reveal whether the solution broke a requirement or the test harness failed to provide the promised resources. To test this, the author wrote three controlled Python programs, two that correctly sum a 300,000-row file and one that deliberately skips a row to serve as a wrong-answer control. Each condition was run three times with the order reversed, and the bulk-loading version peaked near 97 MiB resident memory while the streaming versions used about 21 MiB. The results show that resource limits can determine outcomes independently of correctness, and that a wrong program does not become acceptable simply because more memory is available. The author cautions that these are observations from one Linux and Python setup rather than general memory requirements.