AI in Peer Review Raises Confidentiality Concerns Before Quality Debates

·1 views

Using AI tools during peer review may breach confidentiality obligations, as manuscripts are unpublished works whose authors never consented to third-party disclosure. Transmitting a manuscript to an external AI service constitutes disclosure regardless of whether the provider stores or trains on the data. Major funders, including the US National Institutes of Health, have banned reviewers from using generative AI to analyse or critique grant applications on these grounds. Journal and conference policies vary widely, with some prohibiting manuscript uploads to external tools while others require disclosure or remain silent. Experts also warn that AI-generated reviews tend to be fluent but lack genuine expert judgement, making it difficult for editors to distinguish them from authentic assessments.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Jim Wilson Joins DEV Community to Share Business and Tech Insights

Jim Wilson has introduced himself as a new member of the DEV Community, a platform for software developers and tech enthusiasts. Though not a traditional developer, Wilson works at the intersection of business, technology, finance, marketing, and education. He plans to publish content drawn from conversations with founders and operators, focusing on practical takeaways rather than surface-level commentary. His stated goal is to produce material useful to founders, developers, and anyone making technology-driven decisions. Wilson has invited fellow members to connect and share what they are currently building or learning.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How to Build a Permission-Aware Internal AI Assistant Over Confluence and Drive

Developers building internal knowledge assistants must treat user authorization as the core product, not an afterthought, since serving restricted content to unauthorized users can silently expose sensitive data like salary reviews. Every indexed content chunk should carry an access control list of permitted user and group IDs, evaluated at query time rather than after results are returned. The system must flatten nested group hierarchies and apply a deny-by-default rule, so any chunk missing permissions is visible to no one, ensuring ingestion bugs fail safely. Connectors for Confluence and Google Drive must retrieve effective permissions after inheritance, not just direct grants, and support change feeds to catch permission updates without full re-crawls. Stale permissions represent a security breach rather than a mere inconvenience, requiring separate refresh pipelines distinct from standard content update workflows.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How to Build a Useful AI Eval Set: Size, Statistics, and Clear Decisions

Building an effective evaluation set for AI systems requires anchoring it to a specific decision, such as whether a prompt change causes regressions or whether a cheaper model is sufficient for a task. Developers often make the mistake of building eval sets to broadly 'measure quality' rather than to answer a concrete question, which renders results unactionable. Statistical realities limit what small eval sets can reliably detect: a 50-example set scoring 90% carries a 95% confidence interval of roughly 77–94%, meaning it cannot distinguish a move from 88% to 92% as meaningful. The 'rule of three' further cautions that zero failures across 50 examples still allows for a real failure rate as high as 6%. Small sets remain valuable for catching category-wide breakdowns, and paired comparisons using McNemar's test can make them more informative when evaluating two systems on the same examples.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How One SaaS Team Achieved a Perfect 100/100 Google Lighthouse Score

A software engineering team documented their process of reaching a perfect Google Lighthouse performance score on a complex SaaS application. The team identified three common pitfalls that hinder performance: premature over-engineering, ignoring telemetry data, and misconfiguring standard tools. A key optimization involved replacing sequential API calls with a concurrent batched pipeline, which reduced response latency by up to 75% and lowered CPU spikes during heavy network activity. The findings were shared as part of a broader 2026 High-Performance Engineering Series on DEV Community.

0 comments Read more at DEV Community