Article URL: https://drewdevault.com/blog/AI-in-Linux/ Comments URL: https://news.ycombinator.com/item?id=49097649 Points: 6 # Comments: 0
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
A verified registry of 15 AI bots has been published, categorizing them into training crawlers, assistant crawlers, search crawlers, and scrapers — with each category carrying different consequences for blocking. A key correction highlighted is that Google-Extended and Applebot-Extended are not crawlers but robots.txt tokens that control AI training data use, not search indexing or AI Overviews. Blocking assistant crawlers, which fetch pages to answer live user queries, removes a site from AI-generated answers, while blocking training crawlers only affects model training with no traffic impact. Cloudflare introduced a dashboard toggle in mid-2025 to manage AI crawler rules without manual robots.txt editing, though the underlying token logic remains unchanged. The guide urges website owners to make bot-by-bot blocking decisions rather than blanket blocks, which sacrifice AI-answer visibility without necessarily achieving the intended opt-outs.
A foundational guide for beginner DevOps engineers explains how key internet and networking technologies work together behind everyday online activities. The article covers essential concepts including protocols, packet switching, IP addresses, and the TCP/IP model that enables reliable device communication. It also explains the roles of HTTP and HTTPS in web data transfer, as well as DNS in translating domain names to IP addresses. These topics are presented as critical building blocks for anyone starting a career in DevOps, cloud computing, or software engineering.
A cache-coherence bug in Next.js can cause sitemap.xml to display fewer entries than actually exist on a site, even after new content is published. The issue stems from two separate cache layers: tag-based revalidation clears the inner data cache but leaves the outer route handler cache serving outdated XML. Because Next.js treats sitemap.ts as a Route Handler and caches its rendered output by default, the sitemap never re-executes after a content update. Adding a single export — 'export const revalidate = 3600' — enables Incremental Static Regeneration at the route level, forcing the sitemap to re-render on a set interval. Once route-level revalidation is in place, the inner unstable_cache wrapper becomes redundant and can be removed to simplify the data-fetching logic.
A software developer has documented an architectural discipline called 'Lateral Isolation,' which requires all communication between peer services to pass through a controlled boundary rather than occurring directly. The approach applies established principles — Information Hiding and the Acyclic Dependencies Principle — strictly at the service level. To move beyond theory, the developer published a GitHub repository containing runnable code and an ArchUnit test that verifies the isolation holds in practice. The pattern comes with acknowledged trade-offs, including a 5–10 ms latency cost per hop and centralization risks introduced by the shared boundary. A decision framework is also included to help engineers determine when enforcing the pattern is appropriate and when it can be skipped.
Discussion (0)
Log in to join the discussion and vote.
Log in