Replacing Screaming Frog with an API: A 10,000-URL Agency Workflow

·4 views

Screaming Frog SEO Spider is the most-installed SEO tool in agency workflows. Most audits we've seen referenced over the past decade started with a Screaming Frog crawl. It's a good piece of software. It is also a desktop application that has to be running on someone's laptop, that has to be configured per-project, that has to be re-run when anything changes, and that produces a CSV which then has to be opened in Excel and triaged by hand. This post is for agencies and in-house teams asking the obvious question: what does the API equivalent look like, and is it actually cheaper?

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Container Registries and Orchestration: 57,017 Harbor and 78,059 Consul Fingerprints

Container Registries and Orchestration: 57,017 Harbor and 78,059 Consul Fingerprints Two sides of the same deployment Container registries distribute images to the hosts that run them. Service discovery and configuration systems tell those hosts where everything else is. Together they describe an application platform's internal map, and their internet exposure is worth measuring separately because each exposes a different capability. Queried on 25 September 2026 through the ZoomEye SDK with application fingerprints: app="Harbor" returned 57,017 matching assets, app="Consul" returned 78,059, an

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How would you robustly assert the presence and content of a transient UI element that appears and disappears rapidly in

👻 Playwright Challenge: Testing Transient UI Elements? Real-time web applications (dashboards, chat, stock tickers) feature UI elements that frequently appear, move, or vanish within milliseconds. Traditional Playwright assertions like toBeVisible() or toHaveText() often fail here due to race conditions. The most robust approach uses Playwright's expect.toPass() to retry an assertion block until success, combined with locator.all() for dynamic DOM re-evaluation. This "catches" the transient element during its brief appearance.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Zmanin-WP: Common Codes, part 1

If you’ve been following this blog, you’ll know that I’ve been working on my Zmanim-WP plugin for WordPress for a while now. This blog series is designed to help you dig deeper into the plugin and understand what it does, how it works, and whether it’s right for your needs. There are a four shortcodes that help calculate times related to the start of the day. Sunrise Alot hashachar Sof Zman Kria Shema Sof Zman Tefilah I’ve discussed some of these times in other parts of this series and I want to keep the focus of these blogs on HOW to use each shortcode. Feel free to let me know in the comment

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Combining Images into a PDF Does Not Make Them Searchable: A Practical Browser Export Workflow

You photograph ten receipts, combine them into a PDF, and send the result to a colleague. The file opens. Every receipt is visible. Then your colleague searches for a supplier name and finds nothing. Nothing necessarily went wrong with the conversion.

0 comments Read more at DEV Community