Free tool OGCheck now detects broken sitemaps with a single API call

·1 views

A developer has added a sitemap validation feature to OGCheck, a free open-source tool built with Python's standard library. The tool checks whether a site's sitemap.xml is reachable, contains valid XML, and lists actual URLs rather than being empty. Common sitemap issues it flags include invalid XML caused by unescaped characters, empty files left unchanged after CMS updates, unreachable URLs following site migrations, and sitemaps not referenced in robots.txt. The same API also audits a site's robots.txt and Open Graph image tags via separate endpoints. The project is publicly available on GitHub and the developer is seeking community feedback on other silent site visibility issues.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Refactor or Rewrite? A Practical Framework for Engineering Teams

Engineering teams often debate whether to refactor existing code or rewrite it entirely, a choice driven by risk, economics, and time rather than emotion. Refactoring is generally recommended when a system is stable and functional but locally messy, while a rewrite is justified only when the core architecture fundamentally blocks progress. Before committing to a rewrite, teams must deeply understand the existing system, since legacy code often contains undocumented business logic that a rushed rewrite will simply lose. Rewrites also demand significant time and resources, requiring a deliberate business decision rather than a developer's frustration in the moment. In both cases, investing in tests first is essential, as neither refactoring nor rewriting is safe without a reliable safety net.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Lossless Compression Techniques Can Boost RAG Agent Performance and Cut LLM Costs

Large Language Models operate within a fixed token limit called a context window, which poses a challenge for Retrieval Augmented Generation (RAG) agents that must inject retrieved documents into that space. When retrieved content exceeds the context limit, agents typically resort to truncation or summarization, both of which risk losing critical information or introducing inaccuracies. Lossless compression offers an alternative by reducing the token footprint of retrieved text without discarding any factual content, preserving full fidelity for accuracy-sensitive domains like law and medicine. Techniques such as keyword extraction, deduplication, semantic chunking, and syntactic simplification can be applied before or after retrieval to maximize information density. Beyond improving response quality and reducing hallucinations, these methods can also lower operational costs since most LLM APIs charge based on token usage.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Monorepo vs Polyrepo: Lessons from Meta, AWS, and a 15-Repo Migration

A software engineer who worked at both Meta and AWS shares firsthand observations on how each company manages code organization at scale. Meta's single Mercurial monorepo enabled cross-cutting changes in one commit and made code discoverable, but demanded distributed build tooling and careful dependency management. AWS's polyrepo model offered clear team ownership and independent deployments, yet led to duplicated infrastructure patterns and poor discoverability across thousands of repositories. Drawing on both experiences, the engineer migrated SID Technologies from 15 separate repositories into a single monorepo over a weekend. The key takeaway is that the monorepo versus polyrepo choice is fundamentally an organizational decision rather than a purely technical one.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Same Arithmetic Flaw Behind Three Sui DeFi Exploits Costing Over $224M

Three Sui-based DeFi protocols — Cetus, Aftermath, and Bucket Protocol — suffered or remain vulnerable to critical bugs rooted in the same class of arithmetic error: integer overflow or underflow in math functions. Cetus lost $223 million to a shift overflow, while Aftermath lost $1.14 million to a negative fee bug, and Bucket Protocol carries an undetonated decimal scaling vulnerability identified by an independent researcher. All three protocols had been audited by reputable firms, yet the flaws went undetected because the math appeared correct for expected inputs and failed only at unconsidered edge cases. The researcher argues the recurring issue is structural — auditors review logic flow but rarely stress-test arithmetic boundaries systematically. A pull request with a u128 intermediate cast fix has been submitted for the Bucket Protocol vulnerability before it can be exploited.

0 comments Read more at DEV Community