SShortSingh.
Back to feed

Only 7 of 42 major employers have scrapeable job listings, developer finds

0
·1 views

A developer tested the careers pages of 42 large employers in one region and found that only seven actually exposed job data that could be retrieved programmatically. The accessible companies were newer, faster-moving firms that adopted modern applicant tracking systems like Greenhouse, Lever, Ashby, or SmartRecruiters, which publish open JSON APIs by design. The remaining 35 — including major airlines, banks, oil companies, and retail groups — ran enterprise ATS platforms such as Taleo, Oracle, Phenom, and iCIMS, which serve jobs through JavaScript-heavy, session-bound portals with no public feed. The developer noted that no amount of parsing skill can overcome the absence of a public data endpoint, making browser automation the only alternative for enterprise portals — an approach that is fragile and time-consuming. This structural divide also explains why job aggregators often show incomplete or outdated listings for the most sought-after employers.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

LibreFang 2026.7.21 Patches Multi-User Data Leaks and Adds MCP Resources

LibreFang has released version 2026.7.21, incorporating 61 pull requests from four contributors since the previous release ten days ago. The update addresses critical security issues, including a knowledge graph data leak that could expose one user's memory to another on shared agents, and adds cross-account channel guards to block unauthorized agent-to-channel messages. A four-pass repository-wide security audit identified and resolved more than 15 bugs spanning authorization gaps and token-quota race conditions. New features include per-user LLM provider credentials with spend tracking, operator-controlled provider allowlists, real-time Slack progress updates for long-running tasks, and online editing of HAND.toml manifests from the dashboard. The release also completes MCP integration by implementing the resources primitive, extending agent access beyond tools alone.

0
ProgrammingDEV Community ·

Developer Uses AI Agents to Build 50-Game Educational Platform GamesMom

A software developer built GamesMom, a browser-based educational gaming platform featuring over 50 games, with significant assistance from AI agents throughout the development process. The developer found that AI was most valuable in reducing repetitive tasks such as coding boilerplate, debugging, documentation, SEO workflows, and testing, rather than replacing core engineering judgment. Key decisions around architecture, user experience, accessibility, and product direction still required human expertise and were not delegated to AI tools. The experience led the developer to conclude that AI functions best as a development partner integrated across the full lifecycle, not merely as a code generator.

0
ProgrammingDEV Community ·

How bilingual Arabic-English web scrapers silently return empty data fields

Developers building scrapers for bilingual Arabic-English platforms can encounter a subtle bug where data fields return null without any errors or failed requests. This happens because regional subdomains often serve localized Arabic pages, causing English-language CSS selectors to match nothing while still returning a valid HTTP 200 response. Arabic pages also use Eastern Arabic numerals, which standard ASCII regex patterns like [0-9]+ silently miss, causing numeric fields to appear absent rather than unreadable. Defensive checks that raise loud errors when critical fields are entirely empty across all records help catch locale mismatches early. Additional pitfalls include substring keyword matching across languages, where terms like 'coo' or 'partner' incorrectly match unrelated job titles, making word-boundary enforcement essential.

0
ProgrammingDEV Community ·

How to build a reliable hiring intent score using volume, seniority, and momentum

A DEV Community article outlines a method for scoring how actively a company is hiring by combining three inputs: open role volume, seniority mix, and momentum based on role count changes over time. The author warns that raw job counts are misleading without normalisation for company size, recommending a capped logarithmic volume calculation so large firms do not dominate the score. Seniority weighting is highlighted as particularly error-prone, since job title classifiers can misread words like 'coo' in 'Coordinator' or treat 'Assistant Vice President' as an executive role. The article provides specific regex fixes using word boundaries to prevent such misclassifications, which in the author's own data reduced the apparent share of executive-level roles from 18% to 16% and shifted company rankings. Momentum on a first run is also addressed, with the author recommending a neutral placeholder value rather than zero to avoid making all companies appear stagnant on day one.