Claude 3.5 Sonnet, GPT-4o-mini, or Gemini 1.5 Pro: Testing Chatbots for E-Commerce Support

·4 views

A developer guide argues that the best chatbot API for e-commerce support should be chosen by testing with real support tickets rather than relying on generic leaderboards. It recommends Claude 3.5 Sonnet for the most natural conversational quality, GPT-4o-mini for faster, budget-friendly deployments, and suggests testing Gemini 1.5 Pro before deciding. The author proposes scoring label accuracy, escalation decisions, required output format, and tone separately, and running each test more than once to avoid relying on single responses. The article also advises designing the API boundary so the chosen model can be swapped without rewriting the triage workflow.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Developer Builds Backwards Word Game Noillirk, Where Answers Must Be Recalled in Reverse

A developer describes Noillirk, a word game whose name is 'Krillion' spelled backwards and which is the third title in a family that includes Dive Krillion and Krillion Infinite. In the game, players answer prompts like 'name a fruit' by producing words reversed, which disrupts the brain's usual fast, forward-only retrieval and makes the task feel noticeably harder. Building it required curating a separate prompt bank, keeping rarity-based scoring intact, and designing the interface to signal the mirrored mode without a tutorial. The developer found that playtesting the backwards version revealed where the difficulty of the original games comes from, namely the race to retrieve less obvious answers. The backwards game ended up serving as a diagnostic of players' own word retrieval as well as entertainment.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Architecting for AI: What Changes in Software Design and What Still Holds

A developer-authored book argues that AI introduces a probabilistic, costly, and unaccountable component into systems designed before it existed. The author contends that the industry's business model, development methods, and architecture must all adapt, while foundational practices like capability maps, domain modeling, and documented decisions remain essential. The book offers practical guidance for extending enterprise architecture frameworks to cover AI, including evaluation-based testing, guardrails for agents, and risk-tiered trust design. Each chapter addresses a common belief about AI in software, examines what is genuinely valuable about it, and closes with a checklist of recommended practices. The author aims the book at solution and enterprise architects, tech leads, and engineering managers, as well as developers concerned about their roles.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

O'BLOCK dev borrows Balatro's UI feel, not its visuals, after one player's bug list

The developer of O'BLOCK, a free Steam sneaker-reseller game, rebuilt the interface overnight after a tester listed usability problems. The list covered a scroll wheel that moved the selection, a card drawn beneath text, a hard-to-read price bar, and a custom sneaker price stuck at 1.1k. The team's guiding rule was to copy interaction behavior from Balatro, such as hover and press responses, while keeping every on-screen object drawn from the game's own world. Across five builds shipped in one night, the street screen grew from 8 to 27 visible shoeboxes, and the update touched 18 source files. The developer published a before-and-after dataset alongside the Build 18 patch notes.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Why local LLMs give different answers at temperature 0

Setting temperature to 0 and fixing the seed does not guarantee identical outputs from a local language model. A test with llama.cpp on the gsm8k benchmark changed the generated text on 41 of 100 problems when requests were run one at a time versus eight at once. Other studies cite similar results, including 80 different completions from 1,000 identical temperature-0 requests and accuracy swings of up to 9% when only hardware or batch size changed. The root cause is that floating-point addition is not associative, so differences in GPU kernel reduction order, batching, and prompt caching alter the computed probabilities slightly. Issue trackers for several popular inference engines report the same behavior, making it a broad rather than engine-specific problem.

0 comments Read more at DEV Community