Developer trains 10M-parameter LLM from scratch on free Google Colab GPU

·18 views

A developer built a 10-million-parameter decoder-only Transformer language model entirely from scratch using PyTorch, without relying on pre-trained models or paid cloud resources. The model was trained on the Tiny Shakespeare dataset, a roughly 1.1 MB collection of Shakespeare's public-domain works, using only Google Colab's free T4 GPU. The architecture mirrors the GPT family, featuring 6 layers, 6 attention heads, a 384-dimensional embedding space, and character-level tokenization across 65 unique characters. The project was designed to demystify how large language models work by requiring the builder to manually implement every component, from self-attention to the causal mask. The entire training process completed within a few hours on the free-tier GPU, producing coherent Shakespeare-like text output.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Developer Tests Jev Probability Tool Against LLMs for Classifying Cinema Listings

A developer behind Clusterflick, a London cinema listings aggregator, has been using a language model for over a year to categorise 450–600 daily listings into one of ten types, such as movie, talk, or event. The task is complex because many listings, like a Buster Keaton society screening at the Cinema Museum in Kennington, do not fit neatly into a single category. The developer recently trialled Jev, a non-generative tool that returns probability distributions over structured options rather than producing text. Unlike the LLM approach, Jev evaluates multiple yes/no questions in parallel within a single request, allowing underlying classification factors to be tested directly. The experiment explored whether Jev could match or improve on a year's worth of prompt refinement, given that the text explanations produced by the LLM were never actually used.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Shares Weekly Learnings Across ML, Backend, and Web Security Basics

A developer recently documented self-directed learning across three technical domains: machine learning, backend development, and web application security. On the ML front, they studied supervised, unsupervised, and reinforcement learning, along with exploratory data analysis and linear regression. Their backend exploration covered foundational concepts such as domains, subdomains, and the HTTP protocol. They also examined Cross-Site Scripting (XSS) attack types — stored, reflected, and DOM-based — with a focus on how stored XSS exploits vulnerabilities and how developers can defend against them.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

ShortGeek Converts Articles into Narrated Vertical Videos Entirely Offline

A developer has released ShortGeek, a locally run tool that transforms written articles or RSS feeds into narrated vertical short-form videos without any cloud uploads or subscriptions. The tool handles the full pipeline: drafting a script, generating narration, timing captions at the word level, and rendering the final video using Python, FastAPI, ffmpeg, and local AI models. Word-level caption timing was the most technically challenging part, ensuring captions sync accurately with the narration rather than displaying full sentences at once. A built-in render queue allows users to batch-process an entire RSS feed overnight, since individual renders take considerable time. The developer notes that ShortGeek acts as a drafting assistant rather than a finished production tool, and the installer is not yet code-signed, which may trigger a SmartScreen warning on first run.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Video Shows AI Agent Running 50 Social Media Accounts Simultaneously From One Workstation

A video posted on X on September 19, 2026, appears to show a single workstation operating fifty social media accounts simultaneously using a Chinese AI agent, with no human involvement. The footage depicts dozens of emulated smartphone screens generating posts, replies, and likes around the clock, illustrating what researchers call 'dead internet theory' in practice. First proposed in a 2021 forum essay, the theory argued that most online activity was quietly becoming non-human, a prediction that has since gained empirical support. Cloudflare reported in June 2026 that automated traffic had surpassed human traffic, accounting for roughly 57.4% of web page requests — more than a year ahead of its CEO's own forecast. HUMAN Security's 2026 benchmark further found that action-taking AI agent traffic grew approximately 7,851% year over year, signaling a fundamental shift in who — or what — now drives activity online.

0 comments Read more at DEV Community