New AI benchmark tests models on Pakistan-specific knowledge like lakh/crore and Urdu

·1 views

A developer created a benchmark called the Pakistan Local Context LLM Suite to test AI models on everyday Pakistani concepts. It includes questions about the lakh/crore number system, Pakistan Standard Time, local geography, and Urdu script. The benchmark uses fixed-answer questions scored automatically to ensure reproducible results. Several models, including ones from Google and OpenAI, were tested using this suite. The results showed that while model accuracy was high, the cost of using them varied significantly.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

MIT-licensed architecture guide published for building a Polymarket Perps trading bot

A former trader from MIT and Jump Trading has published a technical guide for building a trading bot for Polymarket's perpetual contracts. The guide, written as an architecture reference, outlines a four-layer system comprising data ingestion, a strategy engine, execution, and a primary risk management layer. It emphasizes critical practices like idempotent order handling, inventory management, and automated risk limits with a kill switch. The author states the complete code for this architecture is available under an MIT license on GitHub.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Kagaz tool creates offline nature scavenger hunts for kids, using local AI

Kagaz is an offline-first software tool that generates printable bilingual nature scavenger hunts for children. It uses a local open-weight language model to create customized hunts based on a city, age range, and month, with seasonal filters ensuring realistic items. The tool produces a PDF checklist that children complete outdoors, aiming to replace screen time with one to two hours of nature interaction. It operates completely offline after initial setup, keeping all data on the user's device with no cloud involvement or costs.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

BreakTrail App Uses AI to Help Users Plan Short Outdoor Breaks

A developer has built BreakTrail, an application designed for developers, students, and remote workers. The app helps users plan short outdoor activities based on their available time, location, mood, and budget. It functions as a Progressive Web App, allowing saved plans and a timer to work offline. The prototype is currently focused on Jaipur and does not offer live navigation or weather updates. Its open-source code is available on GitHub.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

DarkWave extension solves dark mode for Google editors without altering documents

A developer created an extension called DarkWave to enable dark mode in Google's web-based editors. The challenge was that tools like Google Sheets render editing surfaces in canvas elements, which standard CSS filters affect indiscriminately. DarkWave applies a targeted solution, ensuring visual changes do not alter the document's actual saved data or distort embedded images. The stable version is free, works offline, and supports Google Docs, Sheets, Slides, and other websites.

0 comments Read more at DEV Community