RAGBench Launches on Hugging Face as Open Evaluation Framework for RAG Pipelines

·1 views

A developer has released RAGBench, an open evaluation framework for Retrieval-Augmented Generation (RAG) systems, now available on Hugging Face. The tool is designed to help developers test and compare different RAG approaches using consistent documents, queries, and evaluation metrics. RAGBench currently supports benchmarking of semantic, document-level, and parent-child chunking strategies, measuring their impact on retrieval and answer quality. The framework aims to replace guesswork in RAG development with reproducible, measurable results. An interactive benchmark is publicly accessible on Hugging Face Spaces, and the project is open to community contributions and feedback.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Baseline: A Production-Ready FastAPI Starter Kit with JWT Auth and Async SQLAlchemy

A developer has released Baseline, an open-source FastAPI starter kit designed to eliminate repetitive project setup for production applications. The kit follows a strict four-layer architecture — Router, Service, Repository, and Schema — keeping business logic, persistence, and HTTP handling cleanly separated. It includes JWT authentication with short-lived access tokens, a consistent error response shape, and uses Argon2 password hashing instead of the commonly recommended bcrypt wrapper. Tests are run against real PostgreSQL rather than SQLite to better reflect production conditions. The project is intentionally minimal, excluding features like Stripe integration, multi-tenancy, or an admin panel, focusing solely on a reliable, scalable foundation.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Fails to Automate Medium Publishing via Chrome DevTools Protocol

A developer spent two hours attempting to programmatically publish an article to Medium using Chrome DevTools Protocol (CDP) and various browser automation techniques. Medium's story editor relies on a ProseMirror-like architecture whose internal state model cannot be updated through standard DOM manipulation methods such as innerHTML assignment or the deprecated execCommand API. Lower-level CDP methods like Input.insertText and character-by-character keyboard simulation also failed to trigger Medium's internal transaction system, causing the editor to treat the document as empty despite visible text. Clipboard-based workarounds were blocked because navigator.clipboard.writeText requires both explicit user permissions and a genuine user gesture, neither of which CDP can fully replicate. The experience highlights how modern rich-text editors actively resist automation by decoupling their internal state from raw DOM and input events.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer details browser automation workaround for Gumroad's login and product creation

A developer has documented a step-by-step browser automation workflow to create and publish products on Gumroad, which offers no public API for the task. The process required overcoming several technical hurdles, including bypassing React's input tracking using native DOM setters to register form field changes correctly. Gumroad's email-based two-factor authentication added further complexity, requiring a timed polling delay of at least 10 seconds to avoid retrieving stale verification tokens. A persistent Chrome profile with a dedicated user data directory was used to preserve session cookies across script runs, avoiding repeated login cycles. The guide also covers handling a contenteditable description editor and a numeric price field, both of which behave unexpectedly with standard automation approaches.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Harness Engineering Applies to Human Operators, Not Just AI Agents

A concept gaining traction in agentic software development argues that the standard equation 'Agent = Model + Harness' omits a critical counterpart: the human operator who directs what agents build across sessions. Just as an AI agent relies on a harness — loops, tools, context, and constraints — to turn reasoning into useful output, the human directing those agents needs equivalent supporting machinery. This parallel framework, termed the 'human harness,' is defined as a system enabling safe and effective execution by both transmitting a worker's capability into productive work and securing that work against failure states. The concept draws on analogies from physical harnesses, such as those used in horse-drawn carts and rock climbing, to distinguish between transmission and securing functions. Proponents argue that before automating execution through agents, human operators must first orient their work within a structured harness of their own.

0 comments Read more at DEV Community