Developer Builds Universal Property Scraper Using Python and JSON-LD

·1 views

A developer has released a universal property listing scraper that extracts structured data from real estate websites such as Zillow, Rightmove, and Imobiliare without requiring custom parsers for each platform. The tool uses three layered extraction methods — JSON-LD structured data, OpenGraph metadata, and regex-based HTML pattern matching — falling back through each when the previous is unavailable. It captures details including price, property type, bedroom and bathroom counts, area, address components, coordinates, and up to 20 images per listing, with multi-language support. The scraper can handle both individual property pages and search results pages, processing up to five listings concurrently. It is available on the Apify Store at $0.01 per extracted property, via RapidAPI, and as open-source code on GitHub, built using Python, BeautifulSoup4, and httpx.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Why London Shapes Developers Differently Than Other Tech Markets

Developers who have worked in London consistently report gaining professional skills that are difficult to acquire in other markets. The city's dense concentration of experienced engineers creates a strong talent network that compounds in value over time. Working in London exposes developers to informal knowledge-sharing about real-world production systems and complex technical challenges. Engineers also benefit from architectural thinking shaped by colleagues who have tackled diverse problem domains across industries. These compounding advantages are seen as a distinctive feature of London's developer ecosystem compared to other global tech hubs.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How London Quietly Built One of the World's Top JavaScript Engineering Hubs

London has emerged as a leading global centre for JavaScript engineering, driven largely by its dominant European fintech sector. Banks, trading platforms, and payment processors demanded high-performance, enterprise-grade JS solutions, pushing local developers to adopt rigorous engineering standards around real-time data, compliance, and sub-second response times. By 2026, the city's professional frontend community has coalesced around a production stack centred on Next.js, TypeScript, and modern tooling, with TypeScript adoption rates among the highest worldwide. Senior London frontend teams are distinguished by their architectural approach to rendering strategies, state management, and accessibility as a quality metric rather than a compliance formality. Hiring experts suggest that probing candidates on performance budgeting, real-time state design, and bundle-splitting strategies is the clearest way to separate experienced London engineers from those with only surface-level expertise.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Orphaned Staging RDS Instances Are Quietly Draining AWS Budgets

AWS cloud costs in engineering teams are often inflated by forgotten staging resources, particularly RDS database instances that remain active long after the feature branches they supported have been merged and shipped to production. Unlike EC2 instances, which have clear individual owners, staging databases are typically assigned to a team, meaning no single person feels responsible for decommissioning them. A typical pattern sees one engineer provision a database for a sprint, another later run tests against it, and the instance then linger for months with no one willing to delete something a colleague might still need. The problem is compounded by the fact that ECS services and Auto Scaling Groups tied to the same workflow also keep running, since no default AWS lifecycle policy stops them. The root cause is less a tooling gap and more a visibility and incentives failure, where cleanup costs are diffuse, ownership is unclear, and ignoring idle resources requires less effort than removing them.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How Title-Safe and Action-Safe Margins Prevent Video Text From Getting Cropped

Video editors often lose text and key visual elements at frame edges because they overlook safe zone margins, a standard inherited from broadcast television. Action-safe margins sit at 90% of the frame, keeping a 5% buffer on each side, while title-safe margins occupy 80% of the frame with a 10% buffer for text and logos. These standards originate from SMPTE and ITU-R broadcast guidelines designed to account for CRT overscan, but remain relevant today as each platform — from YouTube to TikTok — crops video differently. Calculating safe zones requires subtracting twice the margin percentage multiplied by the frame dimension; for a 1920×1080 frame, the title-safe area works out to 1536×864 pixels. Designers working across multiple formats are advised to build layouts around the tightest safe zone first, ensuring critical content survives cropping on every target platform.

0 comments Read more at DEV Community