SShortSingh.
TopTechnologyAI & MLStartupsProgrammingScienceBusinessFinanceCrypto & Web3World
Back to feed
Programming

Developer finds phone LLM runs 80x slower than theoretical limit

0
·1 views

A developer tested Qwen2.5-1.5B-Instruct on a Google Pixel 4 smartphone and measured generation speeds of 0.5 tokens per second. Theoretical limits for the device's Snapdragon 855 processor suggest it should achieve 30-60 tokens per second. The investigation ruled out three common issues: incorrect compiler flags, memory management problems, and API usage errors. The performance gap persists despite optimizations, indicating deeper computational efficiency issues beyond basic configuration.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Tools for monitoring LLM provider uptime and outages detailed for 2026

The article outlines strategies for detecting AI service disruptions, which require multiple monitoring signals. It identifies three primary telemetry methods: third-party status feeds, scheduled synthetic probes, and real-time production traffic analysis. The text highlights that vendor status pages often lag behind actual incidents by 15 to 45 minutes, making proactive monitoring essential. It states that LLM APIs can fail in complex ways not caught by conventional HTTP checks, such as severe latency spikes or silent quota exhaustion. The piece positions tools like Bifrost as leading solutions for real-time monitoring and automated fallback to maintain application reliability.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Vx build tool avoids daemon architecture, citing performance and simplicity

Build tools Nx and Turborepo use a daemon process to speed up tasks, but the newer tool vx deliberately does not. The vx developers argue a daemon creates a secondary source of truth and adds overhead for startup and maintenance. Instead, vx relies directly on Git's index to manage file hashes and uses efficient algorithms for task scheduling. The team claims this design allows a fully cached run of thousands of tasks in under 400 milliseconds without a background process. Vx is an open-source task runner and build cache designed for JavaScript monorepos.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Developer details journey to reliable AI meeting note parsing using GPT for Asana

A developer spent a week refining a custom GPT model to extract structured action items from free-form meeting notes for Asana. Initial attempts using GPT-4 Turbo produced wildly inconsistent JSON output and invented details despite extensive prompt engineering. The solution involved creating an extremely strict system message enforcing a specific JSON schema for task descriptions and responsible parties. The developer now views the AI as a powerful but fallible first pass in the process rather than a complete solution.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Scan finds 4.9 million Redis services exposed on default port despite security warnings

A security scan of the internet on September 27, 2026, identified 4,937,184 services responding on Redis's default port 6379. Redis documentation explicitly warns that the database should not be exposed to the internet and should only be accessed within a trusted network. Only 142 of these services were definitively fingerprinted as Redis products, but the port itself is a strong indicator of exposure. The scan, conducted via ZoomEye, highlights a widespread misconfiguration that could allow unauthorized data access or file writes. Security guidelines advise restricting Redis to local interfaces, using passwords, enabling protected mode, and configuring firewalls.

0 comments Read more at DEV Community
Developer finds phone LLM runs 80x slower than theoretical limit · ShortSingh