Article URL: https://danluu.com/benchpocalypse/ Comments URL: https://news.ycombinator.com/item?id=49340299 Points: 5 # Comments: 0
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
A developer has created Cosmos, a custom new-tab page extension designed to replace the default blank browser tab. The tool targets developers and designers by offering a workspace with app shortcuts, bookmarks, and useful information in one place. Cosmos features purpose-built widgets tailored to developer and designer workflows, a combination the creator says has not been well executed before. The project is hosted and accessible via Vercel, suggesting it is currently in a public or demo stage. The announcement was shared on DEV Community, positioning Cosmos as a productivity tool for technical and creative users.
A multi-armed bandit is an algorithm that dynamically reallocates website traffic toward better-performing variations in real time, rather than maintaining a fixed split until a test concludes. Unlike traditional A/B testing, which prioritizes statistical precision before declaring a winner, bandit algorithms balance exploration of untested options with exploitation of the current best performer. Three core algorithms drive most bandit implementations: epsilon-greedy, Thompson sampling, and Upper Confidence Bound, each offering a different strategy for managing the explore-exploit tradeoff. The key metric guiding these algorithms is 'regret' — the gap between conversions actually earned and the maximum possible had the best variation been served throughout. Bandit testing is best suited for scenarios where maximizing conversions during the experiment matters more than obtaining a clean, unbiased statistical result.
A developer working on a Unity tactical game discovered that JetBrains Rider's Alt+Enter shortcut for adding assembly definition references appeared to work but left .asmdef files unchanged. The issue surfaced after integrating UnityAtoms via the Package Manager, where types like AtomEvent remained unresolved despite seemingly successful imports. Standard troubleshooting steps — including regenerating .csproj/.sln files and clearing the Library folder — failed to resolve the problem. Manually editing the .asmdef file directly in Rider offered only an inconsistent workaround, with references sometimes reverting without a clear trigger. The bug has since been linked to a known JetBrains issue tracker report, RIDER-94834, confirming the problem affects other users as well.
When setting up an A/B test, the Minimum Detectable Effect (MDE) is the smallest true improvement a test is designed to reliably identify, and it directly determines how much traffic and time an experiment requires. Unlike other inputs in a sample-size calculator, MDE has no default correct value, making it a business judgement as much as a statistical one. A common source of confusion is that most calculators express MDE as a relative lift rather than an absolute percentage-point change, meaning a 10% MDE on a 20% baseline conversion rate targets a 2 percentage-point shift, not 10 points. Setting the MDE too small can leave a test running indefinitely, while setting it too large risks missing meaningful improvements the business actually cares about. Understanding the relationship between MDE, baseline conversion rate, and sample size is essential to designing experiments that are both feasible and informative.
Discussion (0)
Log in to join the discussion and vote.
Log in