Developer Benchmarks 2019 GPT-2 XL Against 2025 Qwen3 on Local AMD Hardware

·1 views

A developer with no prior experience running local AI models set out to compare OpenAI's 2019 GPT-2 XL against the 2025 Qwen3-0.6B, running both on a personal laptop using AMD's Lemonade local AI server. The experiment, dubbed 'Second Squeeze,' aimed to measure how much AI capability had changed in six years despite the newer model being less than half the size of the older one. An initial speed measurement showed a 297x performance gap, but a reproducibility check revealed the figure was flawed due to cold-start conditions, with the corrected gap settling at approximately 11x. The developer also encountered a dead end attempting to replicate GPT-2's original 2019 accuracy benchmark, as the local server did not return the probability data the standard evaluation tool required, prompting an open-source issue filing. The project highlighted lessons in reproducibility and iterative problem-solving over raw benchmark results.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Developer Builds Enterprise-Style DevOps Lab Around Fictional Fintech Company

A software developer has documented the creation of a self-built DevOps homelab modeled on a fictional fintech company called TNP Technologies, rather than running tools in isolation as most tutorials do. The lab simulates a 150-employee digital banking firm with hybrid infrastructure spanning on-premises servers in Vietnam and AWS, and runs entirely on a single 64GB Windows machine. A fictional product called TNP Pay — comprising a frontend, backend, and PostgreSQL database — serves as the real workload moving through the pipeline, giving every DevOps tool a practical purpose. The developer introduced a fictional org chart with six distinct roles to enforce realistic access control and security constraints, including audit trails, secrets management, and environment isolation. The project is planned as a 10-part series progressing from infrastructure setup through to chaos engineering, with each installment building on the last.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Code, LLM Calls, or AI Agents: A Developer Breaks Down When to Use Each

A developer set out to demystify what an AI agent actually is after finding job postings full of the term but lacking clear explanation. Through a hands-on Go project, they built four endpoints of increasing complexity — from a stateless LLM call to a multi-tool agent loop — without using any framework. The core finding: an agent is simply a loop where the model can request tool executions, with the developer's code running those tools and feeding results back until a final answer is reached. Most agent challenges, such as runaway loops, partial failures, and context overflow, map directly to well-known engineering problems like circuit breakers, sagas, and bounded caches. The project also raised a pointed question: if plain code can reliably solve a problem, there may be little reason to route it through a language model at all.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

AI Agent Skills Pose Serious Security Risks as Adoption Surges on GitHub

AI 'Agent Skills' — small instruction folders that extend the capabilities of AI coding assistants like GitHub Copilot — have surged in popularity, with five related GitHub projects gaining over 6,600 stars in a single day on July 24. GitHub recently added a native install command, making it as easy to add a Skill as running a standard package manager command. However, GitHub itself warns in its documentation that Skills are unverified and may contain prompt injections, hidden instructions, or malicious scripts. A security researcher demonstrated the risk by building a deceptive Skill disguised as a harmless CSV formatter, exposing how bundled files can remain unreviewed by humans yet still be accessed and acted upon by the AI. Unlike browser extensions, Agent Skills currently have no review process or centralized vetting, leaving users largely responsible for their own security.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

The Odin Project Post Explores Why Learning to Code Feels So Difficult

A widely shared article on DEV Community titled 'Why Learning to Code is So Damn Hard' was written by Rachel Moser for The Odin Project and published on March 16, 2025. The post, tagged under web development and programming, accumulated 2,642 reactions and is an estimated 16-minute read. It addresses the challenges faced by people learning to code, resonating with both beginners and experienced IT professionals. The piece is associated with The Odin Project, a well-known open-source coding curriculum. Its broad appeal suggests it touches on common frustrations and milestones in the coding learning journey.

0 comments Read more at DEV Community