Developer builds browser-based LLM that runs offline with no install or GPU

·1 views

A developer has created 'ghost', a single HTML file that runs a quantized large language model entirely within a browser using WebAssembly, requiring no server, API key, or installation. The tool uses wllama, a WebAssembly binding for llama.cpp, to perform inference locally at roughly 3–8 tokens per second on a standard CPU. On the first visit, the chosen model is downloaded from HuggingFace and stored via the browser's Cache API, enabling fully offline use on all subsequent visits. Three models are supported: Qwen2.5 1.5B, Qwen2 0.5B, and TinyLlama, catering to different performance needs. The project is open source and publicly accessible on GitHub, with a live demo available online.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Poor Retrieval, Not the LLM, Is Usually Behind RAG Hallucinations

A software developer observed that a customer's RAG application consistently produced confident but incorrect answers despite months of development and multiple LLM swaps. The root cause was traced not to the language model itself, but to the retrieval layer supplying it with flawed context. Document chunking — commonly treated as a routine technical step — was identified as a key culprit, as mechanical token-based splits often sever related content and degrade answer quality. The author advocates for semantic chunking strategies that respect document structure and detect natural topic boundaries using cosine similarity between sentence windows. This approach, now integrated into tools like LangChain and LlamaIndex, helps ensure the model receives coherent, relevant context before any generation occurs.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

AI Adoption Is Fast, But Companies Risk Discarding the Knowledge That Makes It Work

A consultant with experience helping European enterprises integrate AI into business workflows warns that rapid AI adoption often sidelines employees whose deep process knowledge is critical to operations. Drawing on a reported incident of rare books being shredded after digitization, the author uses it as a metaphor for how organizations discard institutional knowledge without acknowledging the trade-off. The consultant argues that the speed of AI rollouts frequently obscures deliberate decisions about what human expertise is being replaced or lost. The core concern is that companies rarely pause to decide intentionally which tasks should remain human-led, slow, or owned by experienced staff. The author urges organizations to recognize that choosing speed in AI adoption is itself a consequential decision, whether or not it is consciously made.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How a Wine Cellar App Explains What MCP Actually Does for AI Models

A developer behind Cellarion, a free open-source wine cellar app, used wine management as an analogy to explain the real purpose of Model Context Protocol (MCP). While AI models like Claude hold broad domain knowledge — such as vintage quality and grape characteristics — they have no access to a user's personal data, like what bottles are currently in their cellar. MCP serves as the connective layer that bridges this gap, allowing models to query private or real-time data sources rather than relying solely on general training knowledge. Cellarion's MCP server exposes around 60 tools, enabling an AI to chain multiple queries — checking drinking windows, open bottles, and recent consumption history — to answer nuanced personal questions. The developer argues this pattern applies far beyond wine, extending to codebases, customer data, and other domains where personal context determines the usefulness of any AI response.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer shares 15-stage framework to fix AI coding's hidden pitfalls beyond the prototype

A software developer with over a decade of experience has outlined the systemic challenges that emerge when using AI tools to build production-grade software. While AI can generate working code quickly, problems such as context drift, security gaps, redundant abstractions, and lack of consistency tend to surface as codebases grow larger. After completing three major AI-assisted projects, the developer identified that a structured process — not better prompts or tools — was the missing ingredient. This led to a 15-stage development framework covering planning, architecture, testing, security, and automation, designed to impose engineering discipline on AI-driven workflows. The developer has compiled these insights into a handbook titled 'Vibe Coding with Confidence' to help others avoid repeating common mistakes.

0 comments Read more at DEV Community