Trelix v3.2.1: How a Mocked Test Suite Hid a Critical Embedding Bug for 28 Releases

·1 views

Open-source project Trelix shipped a silent but severe bug across 28 tagged releases, from v2.0.0 through v3.1.6, in which its BGE code embedder used the wrong pooling method due to a mismatch between FlagEmbedding's default CLS pooling and the causal decoder architecture of BAAI/bge-code-v1. Because the model is a Qwen2 decoder rather than an encoder, CLS pooling at position 0 depended on no subsequent tokens, causing every query embedding to be bitwise identical to every other. The defect went undetected because the unit test suite replaced FlagModel with a MagicMock, which never invoked real pooling logic and therefore never exposed the failure. Version v3.1.6 inadvertently made the broken path reachable by fixing an unrelated AttributeError, and v3.1.7 publicly retracted the earlier claim that bge-code was working. The current release, v3.2.1, tagged on 2026-08-26 and available on PyPI, spans 190 commits and 324 changed files since v3.1.1, with the project now running a 4,376-test suite as part of an ongoing self-audit effort.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

New Benchmark PeakBench Reveals AI Agents Can Plan Correctly Yet Crash Systems

A preprint published on August 25 introduces PeakBench, a benchmark designed to expose a production failure mode where AI agents correctly identify parallel tasks but overload the machines executing them. The benchmark tested eight leading AI models — including GPT-5, o3, Claude Sonnet 4.6, and DeepSeek variants — on around 300 executable workflows built from approximately 1,200 MCP-compatible tools. Researchers found that planning accuracy had almost zero correlation with capacity violations, meaning a model could map task dependencies perfectly and still schedule jobs that exceed memory or concurrency limits. PeakBench scores dependency planning and resource scheduling separately, so failures can be traced to reasoning, scheduling, or infrastructure rather than lumped into a single end-to-end result. The authors recommend letting AI models build the logical task graph while deterministic infrastructure enforces resource constraints to prevent crashes.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

BulkImgConvert lets you compress, resize, and convert images entirely in-browser

A developer has launched BulkImgConvert, a free browser-based tool for batch image processing including compression, resizing, cropping, and format conversion. The tool supports formats such as JPG, PNG, WEBP, AVIF, GIF, SVG, PDF, and HEIC as input. All processing runs client-side using the Canvas API, meaning no files are ever uploaded to an external server. The tool works offline once the page is loaded and requires no account, imposes no watermarks, and has no usage limits. It is available at bulkimgconvert.com and was built by the developer to solve a personal frustration with server-dependent online image tools.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Launches Nitya, an AI Assistant for Coding and General Queries

A developer has introduced Nitya, a personal AI assistant built to support everyday development and problem-solving tasks. The tool is capable of handling general conversations, programming questions, debugging, and context-aware responses. Nitya is publicly accessible via a web interface hosted on Vercel. The project is described as ongoing, with the creator actively working to improve its capabilities over time. The developer is seeking user feedback to help guide future enhancements.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

SIMURG Tool Claims to Reduce LLM Hallucinations by Intercepting Bad Outputs

A developer has shared early results using a tool called SIMURG to reduce hallucinations in locally run large language models. SIMURG sits in front of an OpenAI-compatible endpoint, monitors text generation in real time, and automatically aborts and retries requests when it detects the model going off track. The developer tested it with a heavily compressed 2-bit quantized version of Qwen3.8 27B running on a consumer-grade RTX 3060 GPU, a setup prone to generating hallucinations and garbled outputs. According to the developer, responses became noticeably more consistent after deploying SIMURG, with end users never seeing the problematic outputs. The tool is described as a 15-parameter machine learning model and is available via pip install, with early testing still ongoing across varied prompts.

0 comments Read more at DEV Community