AI essay feedback changes strong students' quotes more, due to two causes

·2 views

A follow-up audit of GLM-4.7's feedback on 8,946 IELTS essays examined why the model alters words inside student quotes more often on strong essays than on weak ones. The analysis found two contributing causes: weak essays have more quotes excluded as suggested rewrites, which lowers their changed-quote rate, and strong essays still show a genuine higher rate of changed words among close matches. Changed quotes were about equally likely to be loose near-misses in both bands, so looseness does not explain the gap. The author notes the study cannot determine from counts alone whether close matches are true misquotes or suggested phrasing, which requires human review. The practical takeaway is that proficiency band does not reliably indicate whether a short flagged quote is a misquote, though shorter quotes still warrant checking first. The plan, code and counts are publicly available, and the work covers only one model on one exam.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

A 28-day Search Console review: 1,510 impressions but only 6 clicks for a new site

A small English-language site of about 70 posts, aimed at foreigners handling everyday matters in China, examined its 28-day Search Console data after impressions rose sharply over roughly two weeks. Most impressions came from pages already ranking in the top 10, yet those pages drew only 5 clicks from 1,131 impressions, a click-through rate under half a percent. The author attributes the gap largely to weak titles and descriptions that read as topic labels rather than answers to the questions users actually search. Query-level data was incomplete because Search Console applies privacy thresholds at low volume, so the page-level view was used instead. The author's planned response is to rework titles and descriptions rather than publish more posts.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

pgbrake: an undo layer for Postgres when AI agents run SQL

The developer describes pgbrake, an open-source MCP server that lets AI coding agents such as Claude Code and Codex work against Postgres databases without direct access. Each statement is parsed with pglast, and before UPDATE or DELETE operations the affected rows are saved locally so they can be restored with an undo command, which refuses if those rows changed afterward. DDL, statements without WHERE clauses, and changes touching more than 100 rows require human approval through the client or a terminal command that cannot run without a TTY. All statements and decisions are recorded in a local audit log, and nothing leaves the user's machine. The tool is MIT-licensed, written in Python, and has been tested only on Postgres 17, with its author noting that undo does not cover TRUNCATE or DDL and is no substitute for backups.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Checklist for safely switching PHP versions on WordPress: audit, staging clone, and rollback

A developer guide explains that changing a WordPress site's PHP version from a host dropdown is quick, but confirming the site still works can take much longer. The recommended process starts with inventorying installed plugins, themes, must-use plugins, and drop-in files, since drop-ins like object-cache.php can cause white screens on incompatible PHP versions. It also advises verifying that the WP-CLI runtime matches the web-facing PHP-FPM version, because command-line tests otherwise may not reflect what visitors experience. Compatibility scanning with PHPCompatibility via PHP_CodeSniffer is suggested to flag removed functions and deprecated constructs, though it cannot detect behavioural changes or missing extensions. Finally, the guide recommends comparing loaded PHP extensions between the old and new versions, since missing modules such as gd or intl can break sites in ways code scans miss.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Haunted Shell: A Terminal Ghost That Comments on Your Commands Offline

A developer built Haunted Shell, a small tool that adds a ghostly voice to the terminal and occasionally comments on commands just run. It uses a zsh hook to capture each command and its exit code, then passes them to Gemma 3 (1B), an open-weight Google model served locally through Ollama. The ghost speaks after roughly one in three commands, running in the background so it never blocks the prompt. The project runs entirely offline with no cloud calls, which the author argues keeps shell history private and costs nothing per use. It was submitted to the Hacktoberfest Open-Source AI Challenge, entering the Best Use of Gemma category.

0 comments Read more at DEV Community