Developer Tests Local LLM for Offline Incident Response, Finds It Fabricates Answers

A solo developer behind RoamSwitch, a Mac and Linux network security tool, conducted a simulated incident response drill to test whether a locally running LLM could assist during a network lockdown. RoamSwitch's Air-Gap containment feature severs all network traffic when ransomware-like behavior is detected, cutting off cloud-based AI tools at the exact moment they are most needed. To address this gap, the developer wired RoamSwitch's read-only MCP server to a local LLM running via Ollama, allowing the model to query diagnostics such as port anomalies, eBPF incidents, and ransomware canary status without any internet connection. A staged attack scenario — involving a suspicious process on a known Metasploit port followed by encrypted writes to a decoy file — was injected into RoamSwitch's state files and fed to the Qwen 27B model for analysis. The drill revealed that while the local LLM could access real diagnostic data, it also generated fabricated details, highlighting a reliability concern for offline AI-assisted incident response.