AI Driven Testing
Article URL: https://app.deltix.ai Comments URL: https://news.ycombinator.com/item?id=49307099 Points: 5 # Comments: 3
Article URL: https://app.deltix.ai Comments URL: https://news.ycombinator.com/item?id=49307099 Points: 5 # Comments: 3
Thai engineer Chanwit Kaewkasi from Korat, working under Cogentica AI, has built Goish — a project that ports Go 1.25's standard library and runtime into Rust without relying on std, glibc, Tokio, or a garbage collector. The project brings Go's concurrency primitives — goroutines, channels, and select — into Rust while preserving memory safety, producing fully static binaries. A key motivation behind Goish is software supply-chain compliance: each ported function carries a comment tracing it back to the exact file and line in the Go SDK, with CI checks verifying those references remain accurate. This function-level provenance addresses a gap that tools like SLSA and SBOM cannot fill — proving that ported crypto or runtime code faithfully reflects its upstream source rather than being reimplemented loosely. The project is timed against approaching regulatory deadlines, including the EU Cyber Resilience Act's SBOM mandates and the US CNSA 2.0 transition, which together demand traceable, verifiable code lineage.
A technical guide argues that EU startups should evaluate speech-to-text APIs based on cost per accepted invoice rather than advertised per-minute pricing. The approach involves routing providers like OpenAI, Deepgram, AssemblyAI, and Google Cloud through a common interface and testing them against real supplier audio recordings. A small corpus of around 20 carefully chosen clips — covering invoice numbers, VAT identifiers, dates, and currency codes — is recommended to expose meaningful accuracy failures. EU data-handling compliance is treated as a hard disqualifier rather than a soft scoring factor. The framework emphasizes that a cheap transcript which corrupts a VAT number or total has no practical value, making accuracy-adjusted cost the only meaningful metric.
A technical guide outlines how to integrate a feature flags API as a polled configuration source in a React-based customer support console. The approach recommends loading default flag values synchronously before any network request, so the UI renders immediately and remains stable even if the configuration fetch fails or times out. Flags should only control presentation elements like labels, layouts, and diagnostic panels — never server-side decisions such as notification retries, billing, or user authorization. The React state machine should use a single shared context to avoid multiple competing pollers, and retain the last valid configuration during refresh cycles to prevent visual disruption. The guide also cautions that polling alone does not provide audit history, evaluation statistics, or real-time monitoring, and that separate tooling is needed for those requirements.
Choosing a speech-to-text API for an EU startup in 2026 requires evaluating more than the advertised per-minute rate, as billing minimums, language accuracy, latency, and data compliance all affect real costs. Vendors may round short audio clips to larger billing units, meaning actual invoices can far exceed estimates based on average recording length. Quality testing should use real support audio containing product names, error codes, and varied accents rather than generic benchmarks, since transcription errors can corrupt downstream workflows. Asynchronous processing and webhook-based completion are recommended for uploaded call queues to avoid holding open connections. Candidates such as OpenAI, Deepgram, AssemblyAI, and Google Cloud should each be verified for EU data residency, retention policies, and subcontractor disclosures before price comparisons are made.
A software engineer was hired by a Dubai logistics firm to fix a customer-support AI agent that forgot every conversation after each session ended, forcing customers to repeatedly re-explain their issues. The core problem was that the agent relied solely on a context window for memory, which cleared completely between chats. After a month of rebuilding the agent's memory layer, the engineer found that integrating a vector database — not a larger model or longer prompt — was the decisive fix. The vector database enabled retrieval-backed long-term memory, allowing the agent to recall a customer's order history, preferred contact method, and past support tickets in under 60 milliseconds. The engineer distinguishes three types of agent memory — working memory (current context), long-term memory (external stored knowledge), and episodic memory (past actions and outcomes) — arguing each must be designed separately for reliable, cost-effective AI systems.
A software developer discovered that a Docker build silently served a day-old compiled artifact despite new source code being present, with no errors raised during the process. The issue stemmed from Docker's layer cache, where a COPY --from=builder instruction pulled a previous build's output even though the source had changed. To counter this, the developer built a post-build verification step that compared compiled artifacts inside the image against expected source hashes. However, the check only covered three hand-picked modules out of 65, and when a release modified a different module, the check passed with an 'OK' that was technically accurate but practically misleading. The incident highlights a deeper flaw: the verification system never validated its own core assumption that cache staleness always affects an entire layer rather than individual modules.
Urban Lab has announced it is going open source, releasing hardware designs and firmware for a smart electric scooter platform built around microcontrollers like the ESP32. The project aims to give communities the ability to inspect, repair, modify, and contribute to mobility infrastructure that is typically locked behind proprietary systems. A public hardware repository has been published on GitHub, containing CAD models, 3D-printable parts, electronics documentation, and firmware configuration files. The project is also experimenting with privacy-focused payments using Monero 2-of-3 multisig escrow workflows, currently tested on Monero Stagenet with simulated data. Additional components include AI-powered service agents and community development bounties, though the entire platform remains experimental and is not yet ready for real-world deployment.
A technical guide outlines a three-stage architecture for B2B support triage that splits PDF retrieval, reranking, and answer generation into distinct steps. The approach recommends embedding search to find semantically related manual pages, followed by a reranking pass to improve evidence quality before a summary is generated. The final output must cite specific page identifiers and assign a support queue only when the retrieved pages justify that decision, returning a needs-review flag otherwise. Each stage should carry independent timeouts so that failures can be isolated rather than absorbed by a single end-to-end limit. The guide advises shipping the rerank-then-summarize path as the practical default, reserving pure embedding search as a low-latency fallback and deterministic rules only for stable, high-consequence queues.
A developer building Retrieval-Augmented Generation (RAG) pipelines found that most errors blamed on the language model were actually caused by flawed retrieval, a realization that only became clear after adding detailed logging. Four distinct failure types emerged: missing answers in the knowledge base, semantically similar but contextually wrong chunks, model hallucination due to weak system prompts, and malformed chunks produced by token-count-based splitting. Hybrid search combining vector and BM25 retrieval proved more effective than switching embedding models for resolving keyword-sensitive mismatches, while cross-encoders offered a complementary reranking approach. Strict system prompts requiring the model to cite source passages and admit ignorance addressed cases where retrieval was correct but the model still extrapolated. The author concludes that logging chunk scores and sizes from the start is more valuable than any model swap, as most RAG problems originate in the retrieval and chunking stages rather than the LLM itself.
Independent developer Daniel Ioni has launched Urban Lab, an experimental project aimed at combining smart electric scooters, AI services, open-source software, and privacy-focused payments. A key component under development is a 2-of-3 multisig escrow system built on Monero, designed to reduce the trust required between customers and service providers. The backend, called I-ECO-01, is a Node.js and Express REST API that manages escrow states, participant authorization, and transaction tracking. The system currently runs on Monero Stagenet using simulated data and is not presented as a production-ready financial service. Additional components in progress include automated GitHub bounties, AI and robot integrations, real-time monitoring, and experimental MYZ-to-XMR conversion logic.
A developer building fully automated explainer videos discovered that their local image-generation AI depicted the same historical subject as a completely different person in every scene. The root cause was that each scene's image prompt was written independently, causing the AI to reimagine the character's appearance and historical setting from scratch each time. To fix consistency, the developer created a reusable 'character card' defining the subject's appearance and historical constraints, which was automatically injected into every scene's prompt before generation. However, negative instructions like 'don't draw a dome' proved largely ineffective, so a second AI was added as a visual checkpoint to inspect the actual output pixels for anachronisms, fake text, and character inconsistencies. Scenes that failed the automated visual check were regenerated with a new random seed until they passed, successfully eliminating the remaining errors.
VIDRAFT and FINAL-Bench have launched the Open Discovery Challenge, a public leaderboard evaluating AI-generated drug candidates targeting PfDHODH, a key enzyme in the malaria parasite. The initiative addresses a growing gap in AI drug discovery: while generative models can propose thousands of molecules daily, reliably verifying their potency, selectivity, safety, and synthesizability remains unsolved. Submissions are scored across six axes — whole-cell activity, target binding, selectivity, ADMET profile, novelty, and synthesis feasibility — with detailed methodology published on Hugging Face. During scorer validation, the team identified 14 defects, including toxicity thresholds that incorrectly rejected all three approved antimalarials and a binding-efficiency metric that over-rewarded small, weak molecules like caffeine. The challenge highlights that building a fair, scientifically rigorous automated judge for AI-designed molecules is as hard as the molecule generation itself.
AI coding agents like Claude Code access source files, config files, and environment variables, then relay summaries of that content to third-party APIs, creating serious secret-exposure risks. Three main leak vectors exist: context exfiltration, where the agent reads .env files and includes values in prompts; tool output echo, where secrets appear in captured stdout; and prompt injection, where malicious instructions trick the agent into sending credentials externally. Common mitigations such as secret managers and .env hiding tools still leave credentials vulnerable once a command runs. A developer has released an open-source CLI tool called 'trustless' that addresses this by injecting credentials at the process and transport layer rather than exposing them to the agent's context window. The core principle is that agents should receive capabilities, not credentials, and all outbound requests should be scanned to confirm no secrets have leaked.
The CAP theorem states that a distributed system can guarantee only two of three properties — Consistency, Availability, and Partition Tolerance — simultaneously. Since network partitions are unavoidable in real-world distributed systems, engineers must choose between CP (consistency over availability) or AP (availability over consistency). CP systems, such as HBase, Zookeeper, and MongoDB, refuse to return potentially stale data during a partition, making them suitable for banking, inventory, and leader election use cases. AP systems, such as Cassandra, DynamoDB, and CouchDB, continue serving requests even with stale data, which is acceptable for social media feeds, DNS, and product catalogs. The PACELC model extends CAP by also accounting for the latency-versus-consistency trade-off that exists even when no partition is occurring.
SIM swap fraud allows attackers to seize a victim's phone number by impersonating them to their mobile carrier using personal data sourced from breaches, social media, or social engineering. Once the carrier transfers the number to an attacker-controlled SIM, all calls and texts — including bank verification codes — are rerouted to the fraudster. The FBI's Internet Crime Complaint Center has monitored and formally warned about this scheme since 2018, noting that reported losses likely undercount the true damage, as downstream financial theft is often logged under separate fraud categories. Victims typically receive no advance warning, with the first sign usually being an unexplained loss of cell service. The FBI recommends setting a carrier account PIN, avoiding public disclosure of financial details, and replacing SMS-based authentication with an authenticator app or physical security key.

A developer rebuilding a name tattoo tool initially considered using an AI language model to classify user text inputs such as initials, name pairs, and dates. After mapping out the actual classification cases needed, they found the problem was too narrow and well-defined to justify an AI model call. Instead, they wrote a small deterministic function using straightforward conditional rules to identify input structure and reorder lettering style recommendations accordingly. The rule-based approach eliminated inference costs, network latency, and unpredictable outputs, while making edge cases easy to reproduce and debug. The product still uses AI for generating custom lettering compositions, but structured input classification proved simple enough to handle with plain code logic.
Engineering teams running AI experiments on B2B SaaS platforms can manage cost and risk by routing experiment traffic through locally cached feature flags evaluated by each Node.js worker, avoiding live API calls on every request. A small control plane owns flag state with a monotonically increasing revision number, allowing workers to reject stale updates and operators to confirm when changes have propagated. Cost and log events must carry the flag revision and tenant cohort context so that rolling back a flag stops new experiment work without erasing the data needed to evaluate outcomes. Rollback should be treated as a formal state transition with an audit record, including the actor, reason, and a conflict check to prevent operators from accidentally overwriting each other's changes. Keeping the kill switch narrowly scoped and separate from tenant segmentation ensures it remains operable under incident pressure rather than becoming a second, complex routing system.
A beginner developer documented their experience creating a GitHub profile README for the first time, using an Alura article as a reference. The process involved repeated trial and error, including mistakes such as entering their username in incorrect places within the code. They also learned to integrate GitHub Readme Stats via GHStats to display activity metrics and make the profile more complete. Despite the project's apparent simplicity, it marked their first real hands-on exploration of GitHub. The author described the experience as a foundational step in their technology learning journey.
Agency Agents is an open-source project that provides over 230 pre-configured AI expert role profiles — such as Frontend Developer, UI Designer, and DevOps Automator — compatible with tools like Claude Code, Codex, Cursor, and GitHub Copilot. Instead of manually writing lengthy role-setting prompts each time, developers can install specific agent profiles and invoke them directly during their workflow. A tutorial published on DEV Community walks through installing Agency Agents on Windows and Mac, and guides users in setting up a custom Router Skill that automatically selects the most appropriate agent based on the complexity of the task described. The Router Skill is designed to distinguish between simple edits that need no specialist and complex tasks that warrant delegating to a domain expert. Configuration files for both Claude Code and Codex are available on GitHub, along with optional global rules to make the routing behavior more proactive.
