Token limit cut a third of AI feedback replies; re-running old pairs exposed the gap

·1 views

An audit of whether telling a small language model that an essay's writer is an English language learner affects how faithfully its feedback quotes that student initially found every result underpowered. For Qwen2.5-1.5B, the main problem was a 450-token output limit that truncated 145 of its 438 replies before the JSON closed, leaving them unreadable. Re-running the first 20 pairs with a 1,024-token limit showed the model is deterministic, and 8 of 13 previously cut-off replies finished, with a median of 268 tokens needed. Five calls never finished even at 1,024 tokens, and all of them fell on the same side of the comparison, so the lost data is not evenly distributed. The author has published testable predictions for a larger follow-up run using an 800-token limit, along with an updated offline quote-checking tool.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Beginner Team Documents First Hackathon and Open-Source Cybersecurity Project

Gurjot, a first-year student at Alta School of Technology, shares his team's first hackathon experience with the team KOS. The team is building the Reachable Vulnerability Analyzer, a cybersecurity tool that checks whether vulnerable functions in third-party dependencies can actually be reached by an application's code. The article notes that current security scanners often flag vulnerabilities that are never used, creating unnecessary alerts for developers. The team described learning new concepts such as static analysis, call graphs, and dependency analysis while setting up the Python environment and exploring the existing GitHub repository. The piece is written as a reflection on moving beyond tutorials into real-world development.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer fixes game UI that spoiled a tribulation climax before the outcome was shown

A developer working on the game Myriad Immortal Sect found that its interface revealed the result of a heavenly tribulation, the game's climax, before the storm animation played out. Toasts, achievements, the realm name, new tabs and other readouts all updated the instant the button was pressed. The fix holds all displays at their prior state until the reveal finishes, while deferring rather than discarding events such as random destiny rolls. The developer also found that many quest directions named places missing from the in-game map, and changed them to reference only labelled locations or a Wayfinding Lamp.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Refresh Token Rotation Without Disrupting Legitimate User Sessions

A refresh token's long-lived nature makes it a target, since an attacker holding a copy can keep minting access tokens until the grant expires or is revoked. OAuth 2.0 Security Best Current Practice (RFC 9700) recommends sender-constrained refresh tokens or rotation for public clients. Implementing rotation is difficult in production because retries, multiple tabs, and concurrent background refreshes can look like replay attacks and revoke real users. The guidance calls for durable server-side state tied to each token family, atomic consumption of the active token during refresh, and family-wide revocation when a consumed token reappears. A narrowly scoped idempotency allowance can handle lost responses without weakening replay detection.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Defense Contractors Keep Earning CMMC Certifications Despite Phase 2 Pause

The Cyber AB held an October town hall where it reported that Cybersecurity Maturity Model Certification (CMMC) certifications continue to grow. The news comes even though the rollout of CMMC Phase 2 may be paused. Defense contractors are still pursuing and obtaining certification under the program. The source is a brief DEV Community post that links to a cybersecurity consultancy's contact page and tags the program's NIST 800-171 framework. The item offers little detail beyond these points, so specifics such as certification counts or timelines are not provided.

0 comments Read more at DEV Community