Study finds raw HTML wastes most tokens in web content processing for AI

·1 views

A study analyzing 20 web pages found that raw HTML contains a median of 128,000 tokens, mostly from scripts and markup. Converting HTML to text retains unnecessary elements like menus and footers, tripling token counts versus actual content. Extracting main content reduces tokens by a median of 51% compared to full-page text, with additional cleanup saving about 9%. The savings vary by page type, with portal pages benefiting most and long articles like Wikipedia gaining less.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

vm2 3.11.7 patches six sandbox escape vulnerabilities affecting Node.js

The vm2 library, version 3.11.7, released on October 1, 2026, patches six critical sandbox escape vulnerabilities. These flaws allowed untrusted JavaScript code running within a Node.js sandbox to bypass security boundaries and access host system modules. The vulnerabilities stemmed from mismatches between policy enforcement rules and the Node.js runtime's actual module resolution behavior. Attackers could exploit these issues to execute arbitrary code, modify files, or intercept secure communications on the host machine. The fixes address problems in built-in module filtering, promise prototype handling, and package name allowlist matching.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Cryptography challenge cracked by factoring weak RSA modulus

A cryptography challenge on the picoGym platform involved cracking an RSA encryption system. The challenge provided a small, deliberately weak RSA modulus along with public exponent and ciphertext. By factoring the modulus using Python's sympy library, participants could derive the two prime factors. This allowed reconstruction of the private key needed to decrypt the ciphertext and retrieve the hidden flag. The exercise demonstrated how improperly sized cryptographic parameters compromise security.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Jira's 'Time in Status' metric helps teams identify workflow bottlenecks

Jira's 'Time in Status' metric tracks how long tasks remain in each workflow stage, revealing where delays occur. Native Jira tools like Days in Column and Control Chart provide some visibility, but calculating precise status durations typically requires API calls to analyze issue changelogs. Third-party apps like Time in Status by SaaSJet enhance this by accounting for business hours and holidays. Analyzing these metrics allows teams to calculate flow efficiency and identify recurring bottlenecks in their processes.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer creates browser-based tool to strip photo metadata locally for privacy

A developer named Singh has built a free, browser-based tool called ToolVero to remove metadata from photos. This tool runs entirely on a user's device, ensuring photos never leave the local computer or phone. It works by converting uploaded JPEG files into PNGs, a format that inherently lacks the segments used to store metadata like location and timestamps. This local processing method enhances privacy by eliminating the need to upload sensitive images to external servers. The tool is part of a larger collection of over 70 free utilities offered by the developer.

0 comments Read more at DEV Community