How one dev team built a single AI tutor routing 60+ features across five subjects

·4 views

A development team built a multi-subject AI tutoring platform covering general academics, Indian competitive exams like JEE and NEET, language learning up to CEFR C1, career coaching, and an arena mode. Rather than letting each feature manage its own model calls, the team centralized all AI-backed features into a single registry with defined pricing, model chains, and fallback logic. A cost audit revealed that 'thinking tokens' were the biggest expense, prompting a full architectural redesign of the routing layer. The team also discovered a critical flaw where mastery scores never updated because quiz triggers relied on undocumented user-typed keywords, which was fixed by adding explicit UI controls and a shared progress engine. Key engineering lessons included treating response caching as an allowlist, issuing automatic refunds when cheaper fallback models respond, and ensuring errors surface visibly to users rather than silently rendering blank screens.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Engineering Team Cuts RAG Pipeline Latency 40% Using Bayesian Search and Hybrid Retrieval

A development team overhauled their retrieval-augmented generation (RAG) pipeline after standard production deployments proved inadequate for legal contracts, API documentation, and customer support tickets. The team replaced fixed 512-token chunking with document-specific strategies — including recursive, semantic, and agentic chunking — achieving recall@10 scores between 91% and 97% across content types. They combined vector search with BM25 keyword search using Reciprocal Rank Fusion, then applied a cross-encoder reranker to narrow 50 candidates down to 5, adding only 50ms of latency while improving recall by 15%. A query transformation layer was also introduced to handle poorly phrased user inputs before retrieval. The combined changes reduced overall query latency by 40%, bringing the pipeline from over one second per query to a significantly faster, more reliable system.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Engineering team cuts RAG pipeline latency 40% using Bayesian search and hybrid retrieval

A development team overhauled their retrieval-augmented generation (RAG) pipeline after standard production deployments revealed significant performance gaps across legal, API, and support document types. The team replaced fixed 512-token chunking with document-specific strategies — including recursive, semantic, and agentic chunking — achieving recall@10 scores between 91% and 97% depending on content type. They combined vector search with BM25 and cross-encoder reranking using Reciprocal Rank Fusion, which improved relevance correlation from roughly 0.75 to 0.92. A query transformation layer was also introduced to handle poorly formed user queries before retrieval. Together, these changes reduced end-to-end query latency by 40% while pushing overall recall@10 to 95%.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How One Team Built Automated LLM Evaluation That Catches 92% of Hallucinations

A software team discovered the cost of informal AI testing after deploying a RAG-based customer support assistant that served hallucinated responses to over 500 users before any issues were caught. The assistant fabricated a billing policy and pulled API rate-limit figures from a competitor's documentation, exposing a complete absence of automated evaluation. In the post-mortem, the team identified that academic benchmarks like MMLU are insufficient for production use cases and that domain-specific, automated evaluation pipelines are essential. They subsequently built a CI/CD-integrated evaluation system using a judge ensemble that scores responses on faithfulness, instruction-following, schema validity, and safety. The resulting pipeline now catches 92% of hallucinations before deployment by running versioned test cases against multiple automated judges with defined pass/fail thresholds.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Class vs Object in OOP: Key Differences Every Developer Should Know

In object-oriented programming (OOP), a Class acts as a blueprint that defines the rules and structure for creating objects, without occupying any memory itself. An Object is a real instance created from that class, holding actual values and consuming memory at runtime. Every class is built from two core components: attributes, which store data, and methods, which define behavior. A single class can be written once and used to create multiple objects, each with its own set of values. Notably, an object is not required to assign values to every attribute or use every method defined in its class.

0 comments Read more at DEV Community