Gemma 4 on Google TPU v6e vs NVIDIA L4: Speed Gains, Higher Costs, Best at 12B

·4 views

A developer tested a Jev-style decision model — which assigns probabilities to answer options using token scores and softmax — on a single Google Cloud TPU v6e chip running Gemma 4 via vLLM, then compared results against an NVIDIA L4 GPU. The TPU's one v6e chip holds 31.24 GiB of memory, allowing Gemma 4 E2B, E4B, and 12B to run at bf16, while a quantized 26B-A4B fp8 build also fits; all 31B checkpoints exceeded usable memory and were skipped. Both the TPU and L4 produced identical answers across tested model sizes, but the 26B model responded in 27–33 ms on the TPU versus 61 ms on the L4, indicating a clear speed advantage. Despite the speed improvement, the TPU costs more per decision on demand than either the L4 or the hosted Jev service. The 12B model was identified as the most practical size for TPU deployment, balancing performance and cost.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Researcher Argues Memory Mechanisms, Not Scaling Laws, Are LLMs' Next Frontier

A researcher and developer has argued that capability gains in large language models stem from paradigm shifts in understanding intelligence — such as the Transformer and chain-of-thought reasoning — rather than scaling laws alone. The author contends that both breakthroughs essentially mimic higher human cognitive functions, with the Transformer simulating semantic understanding and CoT replicating explicit logical reasoning. Looking ahead, the author proposes that giving LLMs a human-like memory mechanism — distinct from the attention mechanism — is the critical next step. The argument draws on how human memory retains fuzzy state information rather than full detail, enabling reasoning and long-range control, a capability current attention windows cannot replicate. The author conducted a 3-million-token recall experiment to support the case that memory, not expanded context windows alone, is what LLMs fundamentally lack.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Your TLS Fingerprint Exposes Your HTTP Client Before Any Header Is Sent

A developer tested HTTPS requests to the 500 most popular websites from eight countries and found that one in five blocked the requests regardless of location. Investigation revealed that servers can identify a client's software long before reading HTTP headers, through a TLS fingerprinting technique called JA3 or JA4. The TLS ClientHello message — sent before any header — encodes details like cipher suite order, extension count, and supported protocols that differ distinctly between browsers and HTTP libraries. For example, Python's urllib advertises 30 cipher suites and only HTTP/1.1 support, while Chrome advertises 15 suites and HTTP/2, producing entirely different fingerprints. This means a script mimicking a browser via a spoofed user agent can still be detected, since the underlying TLS and HTTP/2 framing behavior gives away the true client.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Coding Still Worth Learning in 2026, But the Approach Must Change

A final-year computer engineering student argues that learning to code remains valuable in 2026, despite widespread claims that AI has made programming obsolete. While entry-level developer job postings have fallen sharply since 2022, the U.S. Bureau of Labor Statistics still projects growth in software developer roles over the next decade. The nature of junior roles is shifting rather than disappearing, with developers increasingly expected to direct AI tools, verify their output, and understand why code is correct or flawed. Research from Anthropic found that novice-led AI coding sessions succeeded only around 15% of the time, compared to 28–33% for intermediate or expert users, suggesting AI amplifies existing skills rather than replacing foundational knowledge. The student concludes that learning programming basics is more important than ever, precisely because AI-generated code requires human understanding to debug and validate effectively.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

OpenAI Tests Sponsored Agents Letting Users Chat With Brands Directly From ChatGPT Ads

OpenAI is piloting a new advertising format called Sponsored Agents, which allows users to open a dedicated, brand-controlled conversation within ChatGPT after clicking an ad. The feature is currently restricted to a limited group of selected advertisers in an alpha phase, with no open route for early-access requests. Sponsored Agents are clearly labeled and kept separate from ChatGPT's own responses, ensuring users can distinguish between the assistant's answers and advertiser-led interactions. OpenAI has confirmed that advertisers will not have access to users' chat histories or memory data. The broader ChatGPT advertising program has already expanded to several countries including the UK, Japan, Brazil, Mexico, and South Korea, though the Sponsored Agents alpha remains far more restricted than general ad availability.

0 comments Read more at DEV Community