VIDRAFT's NL2SQL Engine Scores 83.80 on BIRD Mini-Dev, Surpassing Current Top Entry

VIDRAFT's text-to-SQL system, Darwin-180B-NL2SQL, reached 83.80% execution accuracy on BIRD Mini-Dev, a compact official leaderboard for the widely cited BIRD benchmark, edging past the listed top score of 81.00. The company says it used no gold SQL and no fine-tuning, and it has published its predictions and scorer while submitting the result to the BIRD team for listing. A prediction counts as correct only when its query returns exactly the same rows as the reference query, and human data engineers score roughly 93 on the same benchmark. VIDRAFT also released a public demo in Korean and English that converts plain-language questions into verified SQL, tables, charts, and reports, with safeguards such as read-only access and query limits. The company notes that the engine itself is not being released and that it is preparing a submission to the main BIRD test set, which is evaluated on a hidden dataset.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.


Discussion (0)
Log in to join the discussion and vote.
Log in