Article URL: https://blog.arxiv.org/2026/09/23/arxiv-receives-multiyear-investment/ Comments URL: https://news.ycombinator.com/item?id=49823664 Points: 9 # Comments: 0
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Internet-wide scanning tools like ZoomEye, commonly used by attackers to locate vulnerable systems, can also be used by organizations to audit their own externally visible attack surface. A four-step self-assessment workflow involves building technology fingerprints, running scoped queries, comparing results against internal asset inventories, and remediating unexpected exposures. The approach gained urgency after an August 2026 joint advisory highlighted that attackers used ZoomEye and Censys to identify exposed Siemens S7 PLCs before launching exploits. ZoomEye queries run on September 19, 2026, revealed hundreds of millions of publicly reachable instances of common platforms including Apache, nginx, WordPress, and Fortinet. Security experts stress that the gap between what an organization believes is internet-facing and what is actually reachable is a primary entry point for breaches.
A software developer running a frozen 159-task benchmark suite to compare two AI models discovered that a conservative measurement rule increasingly discarded tasks with each additional test repetition. The stricter rule, designed to filter unstable results, threw away 13 tasks across three repetitions and 17 across six, shrinking the informative dataset and eventually masking a statistically significant difference between the models. A second, less restrictive rule was developed and pre-registered before new test runs began, allowing an independent replication on unseen data. The looser rule reached statistical significance on the fresh data, while pooling all six repetitions under the strict rule returned a non-significant result. The developer concludes that conservative stability filters are not cost-free choices, as they systematically discard genuinely stochastic tasks the more data is collected.
A developer building an LLM drift-detection suite discovered that his exact-match grader was penalizing correct answers due to formatting differences, not factual errors. Tasks asking for the chemical symbol for gold and the smallest planet recorded near-100% failure rates across GPT-4o mini, Claude Sonnet 5, and Llama 3.1 8B, even though all models knew the correct answers. The models were responding with full sentences or markdown styling such as '**Au**' rather than the bare two-letter string the grader expected. A near-identical task asking for a country capital passed cleanly because its instructions specified an exact output shape, exposing the inconsistency in how constraints were worded across the suite. The developer notes the measurement is reliable but the precise explanation is still uncertain, and warns others maintaining exact-match eval suites to audit which tasks have failed consistently without anyone checking the raw model output.
A development team initially estimated they could save roughly $86,000 by training a smaller Qwen2.5-7B model instead of using Claude Sonnet to process 6.5 million financial text units. Early benchmarking showed DeepSeek V3.2 underperforming Sonnet on hard test cases, which initially supported the case for building a distilled student model. However, after refining their extraction rubric to allow 'Unknown' as a valid output, a newer DeepSeek model passed all 46 cases across three held-out test sets. Optimized API costs then converged with the estimated $1,000 GPU cost of training and running their own model. With correction cycles far faster via API than through retraining and redeployment, the team decided renting a model was the more practical choice for this workload.
Discussion (0)
Log in to join the discussion and vote.
Log in