Tutorial: Build a URL Metadata Cache Using Python and SQLite

·1 views

A developer tutorial on DEV Community explains how to build a URL metadata caching system using Python and SQLite to eliminate slow, repeated network requests during link previews. The approach shifts metadata fetching to ingestion time by using Apify's Link Preview & Metadata Extractor actor, which parses Open Graph tags, Twitter Cards, favicons, and HTTP details. Python code calls the actor via a synchronous API endpoint, normalizes the returned fields, and stores results in a local SQLite database for fast, predictable reads. Because metadata formats vary across sites, the tutorial recommends storing the full actor response as JSON while also copying frequently queried fields into dedicated columns. Helper functions are provided to flatten nested metadata objects and extract the best available value across multiple possible field names.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Why Next.js Sites Serve Empty HTML to Users but Full Content to Search Bots

Some Next.js websites display near-empty HTML in 'View Page Source' because they intentionally serve a minimal JavaScript shell to regular users, loading content on the client side to reduce server load. Meanwhile, search engine crawlers like Googlebot receive fully server-side rendered pages so they can properly index all content. This hybrid approach, known as Dynamic Rendering, uses bot-detection tools such as Botd by FingerprintJS alongside Next.js middleware and edge functions to distinguish real users from bots. Unlike simple User-Agent checks, Botd runs at the network edge and offers more accurate detection. The strategy reflects a modern development philosophy where SSR and CSR are not competing choices but complementary tools applied based on who is accessing the page.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How Node.js Handles File Reads Without Freezing: Sync vs Async Explained

A technical explainer breaks down the difference between synchronous and asynchronous file operations in Node.js, focusing on what actually occurs when fs.readFile() is called. In synchronous execution, the JavaScript call stack is occupied by the operation until it completes, preventing any other code from running in the meantime. The article uses the call stack — a data structure tracking active function execution — to illustrate why JavaScript is considered single-threaded. Asynchronous methods like fs.readFile() allow Node.js to offload the waiting to lower-level layers, including libuv and the operating system, without blocking the main thread. This architectural design is central to Node.js's ability to handle multiple operations efficiently despite running on a single JavaScript thread.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Audit of 20 AI-Built Apps Reveals SEO, Security, and Tracking Failures

A developer conducted live deployment audits on 20 apps built with AI tools such as Lovable, Bolt, v0, and Cursor, checking each against its live URL rather than its source code. The most widespread issue was incorrect canonical tags, found in 14 of 20 apps, which directed search engines to builder preview URLs instead of the actual custom domain. Other common problems included broken SEO metadata, non-firing analytics snippets, and missing contact options, with five apps exposing open databases due to disabled row-level security. Two apps were found to have secret API keys embedded in the client-side bundle, making them visible to any visitor. The author noted that all 20 apps lacked any ongoing monitoring, meaning issues introduced by routine redeployments could go undetected indefinitely.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Top 10 AI Papers on Hugging Face Highlight Agents, World Models, and Interpretability

Hugging Face's most upvoted AI papers on July 24, 2026 reflect key trends across agent research, generative modeling, and AI benchmarking. Among the highlights is AREX, a recursive self-improving research agent that evaluates and restructures its own investigation process rather than following a fixed pipeline. Another notable paper examines how Diffusion Transformers internally encode semantic information through text template tokens acting as hidden registers. A third work introduces AlayaRenderer-Flash, a generative world renderer designed to achieve real-time interactive speeds for applications in gaming, robotics, and simulation. The collection also includes papers on large-scale LLM post-training, embodied AI benchmarks, spatial reasoning, and video generation evaluation.

0 comments Read more at DEV Community