SShortSingh.
TopTechnologyAI & MLStartupsProgrammingScienceBusinessFinanceCrypto & Web3World
Back to feed
Programming

Study Tests 8 LLMs on Informal Indonesian Customer Chat for Small Businesses

0
·1 views

A developer who builds WhatsApp bots for Indonesian small businesses (UMKM) created a benchmark to test how well large language models handle the way customers actually type online. The benchmark, called UMKM-Bench, used three fictional shops set in Yogyakarta, Bandung, and Semarang, each with its own product catalog, shipping table, and policies. A total of 84 customer messages were written across three versions — formal, slang, and slang with typos — drawing on Javanese and Sundanese informal language mixed with abbreviations and English loanwords. Eight LLMs were evaluated on two key abilities: understanding garbled real-world queries and avoiding hallucinated answers when the shop's data could not support a response. Scoring was handled with plain Python, checking whether models correctly identified intent, product, quantity, and city, while penalizing any fabricated figures not found in the shop data or the customer's message.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Splitting vision AI into two calls improves screen assistant accuracy and reliability

Developer of Handrail, an open-source screen assistant, found that asking a vision model to simultaneously answer a question and identify UI coordinates in a single API call degraded performance on both tasks. The app now makes two separate calls against the same screenshot: one for reasoning and answering, and another focused solely on locating the relevant on-screen control. Coordinates are returned on a normalised 0–1000 grid rather than raw pixels, eliminating resolution and multi-monitor arithmetic across different hardware setups. The two passes also use different image formats — a compressed JPEG for reading and a full-resolution PNG for precise localisation of small UI elements. Handrail runs locally with no account or server, stores data as plain JSON, and is available as an Apache-2.0 open-source project for Windows and macOS.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

How to Monitor HPE Morpheus HVM and HKS Clusters in Existing Grafana Dashboards

Teams running HPE Morpheus Enterprise can now integrate HVM cluster, HKS Kubernetes cluster, and appliance health metrics directly into their existing Grafana dashboards without installing agents or writing custom exporters. The setup relies solely on the Morpheus REST API, a read-only service account, the Infinity data source plugin for Grafana, and the Prometheus instance already running inside HKS clusters. A dedicated read-only role called 'Grafana Reader' is created in Morpheus with minimal permissions covering operations, infrastructure, health, and monitoring sections. To avoid monthly token expiry causing dashboard outages, a custom OAuth client is configured with a one-year token validity period tied exclusively to the service account. The entire setup, covering Morpheus Enterprise 9.0.2 and Grafana 13.2 with the Infinity plugin, can be completed in approximately one hour by anyone with admin access to both platforms.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Nearly Half of 1,064 Freelance Listings Open With the Wrong First Sentence

An analysis of 1,064 independent sellers' cards across nine categories found that 497 opened by describing the seller's method rather than addressing the buyer's specific situation. The review, covering 863 distinct people with prices ranging from free to $200, also found that 262 listings included no public examples of past work. Cards that led with the word 'free' were among the least clear, despite the assumption that low-cost entry attracts buyers. The core finding is that strangers browsing such listings are typically mid-task and searching for a solution to an immediate problem, not shopping by category. Listings that open with a concrete example or outcome were found to be more effective than those leading with tools, techniques, or pricing.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

What Is CORS and Why Does Your Browser Block Cross-Origin API Requests?

CORS, or Cross-Origin Resource Sharing, is a browser security mechanism that restricts web pages from making requests to a different origin — defined by protocol, domain, and port. Browsers enforce this by default to prevent malicious sites from silently accessing sensitive APIs using a user's credentials. Servers must explicitly permit cross-origin requests by returning headers such as Access-Control-Allow-Origin in their responses. In frameworks like ASP.NET Core, developers can configure named CORS policies to whitelist specific frontend origins rather than allowing all origins. A common debugging tip is to check whether a preflight OPTIONS request is failing before the actual request, and to note that tools like Postman bypass browser CORS enforcement entirely.

0 comments Read more at DEV Community
Study Tests 8 LLMs on Informal Indonesian Customer Chat for Small Businesses · ShortSingh