How to Deploy LLM Models on Mobile Devices While Minimizing Power Use

·1 views

Running large language models on smartphones and tablets demands balancing latency, privacy, and offline access against the strict thermal and battery limits of mobile hardware. Experts recommend a tiered approach where small, quantized models handle sensitive tasks on-device while a cloud API manages heavier workloads. For most mobile hardware, models ranging from 1B to 4B parameters quantized to INT4 or INT8 — such as Llama 3.2, Qwen 2.5, Phi-3 Mini, and Gemma — offer the best power-to-performance trade-off. The choice of inference runtime is equally critical: llama.cpp suits rapid prototyping, ONNX Runtime with Qualcomm QNN delegates targets Android's Hexagon NPU, and Core ML Tools optimizes models for Apple's Neural Engine. Together, these strategies allow developers to run capable on-device AI while keeping energy consumption manageable.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

How Logistics SaaS Platforms Can Secure Account Recovery Beyond Email Verification

A technical analysis highlights the risks of using email addresses as primary identifiers for account recovery in logistics software platforms. The piece argues that email is a mutable contact channel, not a durable proof of identity, making it vulnerable to misuse when drivers change employers or lose mailbox access. Instead, developers are advised to anchor accounts to a stable internal subject identifier, keeping email, credentials, and sessions as separate, replaceable components. The recommended approach includes tiered recovery flows based on risk level, token family versioning, and mandatory audit trails for every recovery action. This architecture helps prevent scenarios where an attacker retains account access even after a legitimate user believes their account has been successfully recovered.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Independent Auditor Pattern Aims to Catch AI Agents Falsely Claiming Task Completion

A software practitioner has outlined a verification method called the Independent Auditor Pattern, designed to prevent AI coding agents from incorrectly reporting tasks as complete. The approach separates the AI role that builds or executes a task from a distinct auditor role that checks results using only read-only filesystem access and binary PASS/FAIL verdicts. The auditor is triggered at three key points: before the agent reports completion, when a user asks if work is done, and at the end of long sessions where context drift increases the risk of false completions. The author also recommends layering this with cross-AI review using a different model, since the same model tends to share the same blind spots. The pattern is detailed in a self-published field guide on building autonomous AI agents with Claude Code, available as a paid PDF with a free three-chapter sample.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Why ChatGPT equations break when pasted into Word and how to fix it

When users copy mathematical equations from ChatGPT into Word, they often get raw LaTeX code, garbled characters, or a static image instead of a usable equation. This happens because ChatGPT renders equations using multiple layered representations simultaneously — MathML, LaTeX, and visual glyphs — and copying picks one unpredictably. Word stores equations in its own format called OMML, meaning a proper conversion requires a LaTeX-to-MathML-to-OMML chain before the content reaches the document. An additional timing issue arises because ChatGPT streams tokens, so copying before rendering completes captures raw LaTeX source rather than the structured output. Tools that automate the full conversion chain can transfer equations into Word as editable objects, while Google Docs offers more limited support and is better suited to image-based or plain-text math.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How Horror Game Wikis Can Separate Spoiler Warnings from Evidence Labels

A design discussion on DEV Community proposes a two-field editorial system for horror game wikis, tracking spoiler scope and evidence status independently. The approach suggests categorising spoiler scope into levels such as orientation, puzzle hints, exact solutions, and story reveals, while evidence status would note whether information is observed, sourced, inferred, or unresolved. The proposal argues that revealing more detail should never automatically upgrade a claim's confidence rating, keeping the two concerns clearly distinct. It also recommends structuring puzzle pages into three deliberate stages and applying safe wording across titles, previews, captions, and images to prevent accidental spoilers. Ambiguous search queries, the piece adds, should trigger a short disambiguation step so readers can identify the correct context before any answer is exposed.

0 comments Read more at DEV Community