Gaffa's capture_snapshot lets developers save webpages for later extraction

·1 views

Web pages often change, disappear, or move behind logins before developers know which data they need. Gaffa's capture_snapshot action saves a static, self-contained HTML copy of a page at the moment of the request, with images and CSS embedded and JavaScript disabled. The saved file can be opened offline and returns its own URL, which can be used as the target of a new request to run extraction actions later. The article illustrates this with a demo page that generates new content on each visit, showing how snapshots preserve a fixed version. It also notes that snapshot availability depends on the user's plan data retention period.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Building a Local SQLite Archive of DuckDB Tweets and Their API Costs

A developer built a small standard-library Python script that pages through a Twitter search for DuckDB-related posts and stores them in SQLite, keyed by tweet ID so re-runs avoid duplicates. The query is filtered to English, non-reply posts with at least 20 likes to keep the archive focused on well-engaged content. The author notes that the API bills per tweet returned at 15 credits each, with a 15-credit minimum per request, and that 100,000 credits equal one dollar. A full page of 20 tweets costs about $0.003, and a first run of 15 pages came to roughly $0.045. The writeup also highlights a mistake that made a second run cost as much as the first, which the author explores in the article. The author works on the TwitterAPI.io service being used and says the code and figures come from their own runs.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Independent study re-tests Direct Nash Optimization for aligning language models

Researchers independently re-implemented DNO-Prct, a method that iterates on-policy sampling, LLM-based rating against a teacher response, preference-pair selection, and a DPO-style update. The team evaluated the approach on Llama 3.1 8B Instruct and Mistral 7B Instruct v0.1. Independent replication is significant for alignment work because it checks whether Nash-equilibrium preference optimization holds up beyond the original implementation and annotators. The findings point toward a possible shift from fixed reward models to game-theoretic preference equilibria and iterative on-policy methods. The work was reported via arXiv.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Vercel Introduces vercel.ts as Typed Replacement for vercel.json

Vercel now recommends vercel.ts, a TypeScript file that replaces the static vercel.json for configuring projects. The file exports a typed config object covering build settings, routing, headers, and cron jobs, so mistakes like misspelled keys or invalid cron strings fail the build instead of being silently ignored. Because it is executable code evaluated at build or deploy time, it can read environment variables and vary settings per environment, which a pure JSON file cannot do. It does not run per request, so request-time logic still belongs in Next.js Middleware or Routing Middleware. The format is built on the @vercel/config package and is intended to replace the schema-less JSON configuration used since the platform's early days.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

iPhone users can use Screen Time or minulo to curb nighttime Reels and Shorts scrolling

A developer's guide explains how iPhone owners can stop late-night scrolling through Instagram Reels and YouTube Shorts. Apple's built-in Downtime feature can block entire apps on a schedule, but it also cuts off messages and regular videos. In-app limits such as YouTube's Shorts feed setting and Instagram's Sleep mode are free but easy to dismiss or do not hide content. The author's app, minulo, hides Reels and Shorts while keeping messages and standard videos available, and it can lock the official apps at bedtime. The article notes that most of the app's usage data, drawn mainly from Japan, shows social media sessions clustering between 8 pm and 1 am.

0 comments Read more at DEV Community