SShortSingh.
TopTechnologyAI & MLStartupsProgrammingScienceBusinessFinanceCrypto & Web3World
Back to feed
Programming

AI PC Label Misleads: NPU Skips LLM Inference, Leaving CPU to Struggle

0
·1 views

A hands-on benchmark of three consumer machines running Gemma 4 26B (18 GB) reveals that LLM inference splits into two phases — prefill and generation — with very different hardware demands. Prefill, which processes the input prompt before any output is generated, is compute-bound and varies nearly 18-fold across tested machines, while generation speed differs by less than twofold. A laptop marketed as an 'AI PC,' powered by AMD's Ryzen 8840U with a dedicated XDNA NPU, failed at large-prompt tasks because current LLM runners do not utilise the NPU, forcing the CPU to handle prefill at just 20 tokens per second and causing stalls exceeding 13 minutes on long prompts. Separately, model-load times were found to be storage-bound, with a machine using a SATA SSD taking over 50 seconds to load the 18 GB model versus around 8 seconds on NVMe, making a long keep-alive setting essential on slower-disk systems. The findings suggest that advertised AI PC specifications, including TOPS ratings, are largely irrelevant for running large language models against long prompts.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

TryHackMe Writeup: Keylogger Disguised as Hotel Update Script Analyzed

A network capture analysis from a TryHackMe challenge revealed that a victim host at 192.168.1.141 downloaded a malicious Python script named updates.py from an attacker-controlled server masquerading as a hotel update service. The script, built using the pynput library, functioned as a keylogger that captured individual keystrokes on the infected machine. Each keystroke was XOR-encrypted using a hardcoded key, base64-encoded, and silently exfiltrated to a command-and-control server via a cookie field in repeated HTTP GET requests. Forensic analysis of the packet capture identified 30 such requests, and decoding the cookie values revealed the hidden challenge flag. The exercise demonstrated how malware can disguise data exfiltration as routine web traffic to evade detection.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Self-Hosting Nous Research Hermes Agent on Railway: Three Key Gotchas Explained

Nous Research's Hermes Agent is a self-improving AI agent that accumulates skills across sessions, but self-hosting it on Railway surfaces several non-obvious configuration challenges. Since a June 2026 security hardening update, the Hermes dashboard will not bind to a non-loopback address without a registered auth provider, meaning leaving the password field empty yields no dashboard at all rather than an open one. Railway also injects its own PORT variable at runtime set to 8080, while Hermes defaults to port 9119, causing silent 502 errors unless the entrypoint explicitly maps the two before startup. Additionally, persistent storage must be configured manually, as Railway tears down containers on redeploy, erasing any skills or session state the agent has accumulated. The author published an open-source MIT-licensed wrapper on GitHub that addresses all three issues and also offers a one-click Railway deployment template.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Developer releases HostJS, a fast SSR Node.js framework with built-in developer panel

A developer has published an open-source server-side rendering web framework called Authtics Host, also known as HostJS, on the NPM registry under the package name @bananacool467/authtics-host. The framework is designed for speed, reportedly starting its server in under one second by sequentially loading core modules, a configuration file, and then launching the server. Pages are delivered to users within one to four seconds, which the developer claims is faster than many existing frameworks that take two to five or more seconds to start. A built-in Developer Panel, secured via a Developer Access Token, offers console and network monitoring tabs alongside server controls such as restart, shutdown, and user pause. The developer highlights mobile-friendliness as a key advantage, noting that most competing frameworks lack accessible console or network tools on mobile devices.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Robotics Faces a 95% Training Data Shortfall as AI Field Turns to Synthetic Generation

As of early 2026, the global stock of high-quality robot interaction data stands at roughly 500,000 hours, far short of the estimated 10 million hours needed to train a general-purpose embodied AI model. To bridge this gap, researchers are using world models as data engines, generating synthetic training trajectories to multiply limited real-world datasets by a factor of 10 to 100 times. However, experts warn this approach carries a familiar risk: when synthetic outputs feed back into future training rounds, rare and edge-case scenarios gradually disappear while average performance metrics appear stable. Unlike text-based AI, where this degradation produces bland outputs, embodied AI systems trained on flawed synthetic physics can fail dangerously on real-world contact tasks such as grasping or handling deformable objects. Practitioners advise that real recorded data must anchor any training pipeline at critical contact points, and that deliberately collected failure data — largely absent from current datasets — is essential for building robots capable of recovery.

0 comments Read more at DEV Community