How to design a web search engine in a system design interview

·11 views

A web search engine prompt is deliberately oversized, hiding three distinct systems: a crawler, an indexer, and a serving layer. Strong candidates scope the problem early and treat it as top-K retrieval rather than crawling. The core structure is an inverted index that maps each word to the documents containing it, turning queries into lookups. Because the index is too large for one machine, it is sharded by document and each query fans out to all shards, with results merged afterward. The rest of the interview covers how the index is built, how fan-out stays fast, and how matching pages are ranked.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Designing AI Study Notes That Learners Can Actually Use

The article argues that an AI note tool producing a summary, flashcards, and a quiz from one document should give each output a distinct purpose and let users verify its claims. It uses an invented offline recording specification to illustrate how summaries must keep critical conditions, such as the difference between recording, uploading, and processing. Flashcards work best as separate questions that each cite a specific source detail, rather than one card that restates the whole passage. The author also cautions that fixed card counts can encourage repetitive filler, so the number should depend on the material. Suggested controls include editing questions, correcting answers, and removing duplicates, along with scenario questions that test understanding.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Committed Android Baseline Profiles Can Silently Do Nothing

A baseline profile can be checked into a project and bundled into the APK yet still have no effect on startup speed. A common cause is generating it against a debug build rather than a release-shaped build, since R8 renames classes and the profile then references symbols absent from the shipped app. The profile also goes stale when new screens or navigation destinations are added without regenerating it, leaving those code paths unoptimized. Because none of this fails the build, developers are advised to verify installation and compilation state directly with adb commands like dumpsys package compiler-stats. Regeneration should be treated as a release-checklist item, with before-and-after startup measurements used to confirm the profile is working.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Automating .NET Assembly Obfuscation via Visual Studio CLI Post-Build Steps

A developer at Rustemsoft describes how manual GUI-based obfuscation of .NET assemblies can be skipped by mistake, leaving code exposed in CI/CD pipelines. The guide explains wiring Opaquer's command-line interface into Visual Studio post-build events so Release builds are protected automatically. It covers limiting parallel project builds to avoid file-lock errors, and running obfuscation with options for private member renaming, string encryption, and control-flow obfuscation. A batch script is provided for CI/CD use, along with a note that setup projects need obfuscation before packaging. The Command Line Interface requires the Pro or Enterprise edition, as the free edition includes only the GUI.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

SQL Interview Question: Finding Departments Paying Above the Company Average Salary

A practical SQL interview problem asks for departments whose average salary exceeds the company-wide average. The article supplies two sample tables, one of employees with department IDs and salaries, and one of departments with names. The company-wide average is given as ₹67,500, and the IT department's average of ₹90,000 is the only one above it. The suggested solution uses a common table expression to compute each department's average and a subquery for the overall average, then filters the results. The author also poses a follow-up question about whether HR would qualify if a new ₹120,000 employee joined it.

0 comments Read more at DEV Community