AI Models Map San Francisco From Memory, With Mixed Accuracy

·2 views

Researchers tested six AI chat models from Anthropic, OpenAI and Google by asking what lies at 41,526 coordinates across San Francisco. The models answered without maps or internet access, and each answer was checked against OpenStreetMap. The best models, run at high reasoning settings, labeled the city's water and park areas correctly about 86% of the time. Their results were close to one another and beat a baseline of 61% from simply guessing neither water nor park. Finer-grained tests in Crissy Field, the Marina and the Palace of Fine Arts showed the models could partly distinguish dense street grids from parkland, though small parks were often misidentified.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Calibration-free top-k selector speeds MoE decoding without threshold tuning

A new top-k channel selector for Mixture-of-Experts models removes the need for offline score-threshold calibration while keeping accuracy and latency gains. Previously, thresholds fitted on held-out data were used to limit routed-expert reads, but this varied across inputs and could trigger repetition loops in models such as Gemma-4. The approach instead keeps each token's k highest-scoring channels, requiring no calibration data. In reported comparisons it matched or beat the threshold version in 40 of 43 cases at 13% and 18% budgets. The authors report retaining 98.1% of full-model accuracy at 17.8% of routed-expert reads on three post-trained MoE models, with Triton kernels running MoE blocks 1.60 to 1.90 times faster than optimized vLLM at batch size one. The evaluation is limited to those three checkpoints and Triton kernels, so generalization to other model families and hardware remains unconfirmed.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Two Models for AI-Powered BI: Dashboards with Agents vs. Agent-First Design

The author argues that the right AI-driven business intelligence design depends on data scope, user count, and how predictable questions are. For a narrow procurement project called SpendCube, built on Azure Databricks, Azure SQL Database, and Power BI, a dashboard paired with an AI agent is presented as practical. Power BI Copilot is cited as an example, with the key requirement being that the agent use the existing semantic model so its answers match dashboard KPIs. In contrast, BMW's Mobile Data Recorder collects over 10,000 signals from roughly 3,500 vehicles, making it impossible to anticipate every engineering question in advance. For such large, unpredictable datasets, the author favors an agent-first approach in which engineers describe what they want to investigate and the agent generates queries and visualizations.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Support FAQ bot invented 22 answers until grounding cut that to zero

A tester evaluated a small n8n support-FAQ workflow using a 1.8B-parameter model running locally, asking 20 questions across two prompt versions over 120 runs. The model answered all 30 known questions correctly but gave answers to 29 of 30 unanswerable questions, and 22 of those asserted facts absent from the knowledge base, such as denying a phone number or Slack integration. Prompt wording alone did not stop the fabrication, so the author required the model to cite a knowledge-base entry and a supporting quote. A Code node then verifies that evidence and falls back to a handover message when it is missing. The approach aims to reduce fabricated answers that could mislead customers into not buying.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Confluence Cloud Adds Native Page Approvals, With Limits on Multi-Approver Rules

Atlassian opened native page approvals in Confluence Cloud to open beta for Premium and Enterprise plans on 10 July 2026, ending years of users being told no approval workflow existed. In the author's test on a Cloud site, a page stayed In review until every named approver signed off, showing the all-or-nothing rule. The native feature cannot use groups as approvers or offer any-one or minimum-N thresholds, and Standard plans only have editable content statuses plus automation. The article offers a Standard-plan workaround using content status and automation, and a third-party app, Sentinel Vault, for more flexible multi-approver rules.

0 comments Read more at DEV Community