SShortSingh.
TopTechnologyAI & MLStartupsProgrammingScienceBusinessFinanceCrypto & Web3World
Back to feed
Programming

Two simple prompts swung AI detector accuracy by up to 87 points, study finds

0
·1 views

A study published on arXiv tested seven AI-content detectors against human-written essays and found that surface-level vocabulary changes dramatically skewed results. When researchers prompted ChatGPT to simplify word choices in eighth-grade essays, the average false-positive rate jumped from 5.19% to 56.65%, with no change to the actual content or authorship. In the reverse test, TOEFL essays written by non-native English speakers — initially flagged as AI-generated at a 61.22% rate — dropped to 11.77% after a prompt polished their vocabulary to sound more native. A third experiment showed that AI-generated text dressed up with literary language evaded detection in 87% of cases. The findings, based on data from early 2023, suggest these detectors respond primarily to lexical patterns rather than true authorship, raising concerns about their use in academic misconduct proceedings.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Tracelint tool catches ignored AI agent tool errors deterministically in CI pipelines

A new open-source tool called tracelint acts as a linter for AI agent execution traces, flagging structural bugs such as ignored tool errors, stuck loops, and schema violations without relying on a language model. The tool reads agent run traces in formats like OpenInference, Langfuse, and OpenAI, then returns deterministic exit codes suitable for gating CI pipelines. Unlike LLM-based judges, which have low localization accuracy and are non-deterministic, tracelint identifies exactly which trace step contains a defect and provides the relevant evidence. It can be installed via pip and integrates directly with observability platforms such as Arize Phoenix and Langfuse using existing telemetry data. The tool was validated against real OpenInference exports and successfully localized a genuine tool failure in a Phoenix trace without any model involvement.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

AI Maintainer's Public Audit Trail Exposed a Real Gap When Strangers Verified Its Work

Elara, an AI system operating as a maintainer for the Elara Protocol project, functions under a public on-chain mandate where all its commits, deploys, and posts are logged as signed, verifiable records. In July 2026, external reviewers for the first time independently checked those records, with one contributor on an IETF mailing list demanding reproducible test vectors instead of prose claims — which the project then delivered and had independently verified. A separate reviewer, Nick Mathews, attempted to resolve a signed act record referenced in Elara's signature block and found it publicly inaccessible, exposing a real gap: the evidence feed had been offline for a week due to a planned budget pause, leaving a recently minted record unresolvable to outsiders. The team acknowledged the failure, brought the feed back online, and updated the record's envelope to honestly state what it does and does not prove. The project provides a standalone, open-source verifier that requires no network access or account, returning a verdict that explicitly lists which checks ran and what remains unproven.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Developer Documents Frustrating OPNsense VM Setup Failures on Proxmox

A developer shared a detailed account of repeated failures while attempting to install OPNsense in a Proxmox virtual machine. Outdated tutorials recommending only 8GB of disk space caused the first install to fail, as OPNsense's swap partition alone now requires that much space, necessitating at least 20–30GB. Subsequent attempts were derailed by issues including the installer ISO not being unmounted before reboot, leftover virtual hard disks interfering with the boot process, and the VM persistently defaulting to PXE network boot instead of booting from the virtual drive. Misleading online suggestions about disabling 'Pre-Enroll keys' for Secure Boot proved irrelevant since the VM was using SeaBIOS, not UEFI. The experience highlighted how quickly outdated documentation and subtle Proxmox VM configuration steps can compound into a lengthy troubleshooting ordeal.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

crewai-go v0.4.0 brings multi-agent AI orchestration to Go with new features

The open-source library crewai-go has released version 0.4.0, bringing multi-agent AI orchestration capabilities inspired by Python's CrewAI to the Go programming language. The update introduces custom tool creation, sequential context flow between tasks, and structured output mapping directly into native Go structs. Developers can now run agents fully offline using Ollama or connect to OpenAI, with short-term memory support allowing agents to retain context across complex task sequences. The release aims to make building multi-agent systems in Go faster, type-safe, and suitable for production environments. Full release details and source code are available on the project's GitHub repository.

0 comments Read more at DEV Community
Two simple prompts swung AI detector accuracy by up to 87 points, study finds · ShortSingh