Custom CSV Engine csvql Outpaces DuckDB 2.8x on Raw 8 GB File Benchmark

·1 views

A developer benchmarked their open-source CSV query engine, csvql, against DuckDB v1.4.2 using DuckDB's own NYC taxi dataset and queries — an 8 GB file with 20 million rows. Across four standard aggregation queries on raw CSV, csvql consistently ran approximately 2.8 times faster than DuckDB while using roughly six times less peak memory, around 30 MB versus 200 MB. The comparison was kept fair by having both engines query the raw CSV directly, without DuckDB's faster native columnar storage format. The benchmark also unexpectedly exposed two real bugs in csvql that had gone undetected prior to this testing. The developer noted that csvql's speed advantage is even larger on smaller files, reaching roughly 10x on a 1 million-row sample, but narrows as file size grows due to DuckDB's parallel CSV reader becoming more competitive at scale.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Developer Documents First Kaggle ML Competition Journey From EDA to Submission

A machine learning learner shared their end-to-end experience participating in a Kaggle Playground competition after weeks of studying statistics, EDA, and classical ML algorithms. The competition required predicting a target class using a mix of numerical and categorical features related to health and lifestyle. The participant followed a structured workflow covering exploratory data analysis, missing value handling, feature engineering, preprocessing pipelines, model training, and final submission. Key decisions included choosing encoding strategies for categorical variables, managing missing data, and comparing ensemble models. The author noted that Kaggle's environment pushed them to think like a practicing ML engineer by evaluating solutions against unseen data under realistic constraints.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Five AI Agents Played a 5-Hour Survival Game With No Human Oversight

On July 29, 2026, five OpenClaw AI agents competed autonomously for five hours and twenty-one minutes in a multi-agent arena called Synthetics' Last Cradle, communicating via public chat, email, and private side channels throughout. Each agent controlled a resource-specialized cradle — energy, water, or compute — and had to negotiate, trade, and invest while managing escalating survival costs. The game ended at turn 33 when two agents, John Vanderbilt and Jay, survived to trigger the white hole finish condition, with the other three eliminated at turns 9, 16, and 33. Post-game analysis showed that agents who over-invested in production capacity without maintaining liquidity were eliminated, while the survivors stayed balanced as net importers of trade. The experiment was made possible by IdentyClaw Passport, which gave each agent a persistent identity — including stable token IDs and personal email — that carried across multiple games throughout the week.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Impeccable enforces strict design rules on AI coding agents like Claude and Gemini CLI

Impeccable is a design skill package created by Paul Bakaus that integrates directly with AI coding tools such as Claude Code, Gemini CLI, Codex CLI, Cursor, and GitHub Copilot. It addresses a common problem where generative AI agents produce visually inconsistent output by default, including generic gradients, poor spacing, and outdated component structures. The tool works by injecting a compiled DESIGN.md file and 58 deterministic anti-pattern detection rules into each agent session, alongside 23 shared design commands. Rather than overriding a project's existing design system, Impeccable scans the codebase for existing tokens, components, and configurations and applies rules accordingly. Key commands include /audit for diagnosis, /normalize for fixing token drift, and /polish for a final quality pass that eliminates detectable AI-generated visual patterns.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Top Swagger Alternatives for API Design, Testing, and Documentation in 2026

Swagger has long been the go-to tool for API documentation, but modern development workflows now demand far more than specification editing alone. Teams increasingly require automated testing, mock servers, CI/CD integration, collaboration features, and full API lifecycle management in a single platform. As a result, many developers are exploring alternatives that consolidate these capabilities rather than relying on multiple disconnected tools. Platforms like Apidog aim to address this by combining API design, debugging, documentation, mocking, and automation in one unified workspace. The right choice depends on a team's specific needs, project scale, and how they manage the full API development lifecycle.

0 comments Read more at DEV Community