Guide Shows How to Scrape X Tweets Without Empty or Fake Results

·2 views

A guide explains how to collect tweets from X (formerly Twitter) by keyword, profile, link, or replies without paying for blank or fabricated rows. It identifies common problems such as placeholder data, loose row limits, date filters that aren't strictly enforced, duplicate entries, and runs that stop silently. The author, who discloses building the tool and earning from its use on Apify, presents the X Twitter Scraper Actor as a solution. The tool drops and counts placeholder rows, enforces exact row limits, checks dates and filters on each tweet, saves duplicates once, and provides a SUMMARY record explaining each input's outcome. Setup requires only a free Apify account, with no X account or developer keys needed.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Tracking Google Flights prices with Python via a scraper, no API key needed

Google Flights offers no public API, so the article proposes a scraper-based approach to monitor fares across many routes and dates. The author's Apify Actor, Google Flights Scraper, collects total price, airlines, times, duration, stops, layovers, leg details, CO2 emissions, and a direct booking link for each flight. Users specify origin and destination airports and date ranges, and can filter by cabin class, stops, currency, and round-trip length. A short Python script using the Apify client exports the cheapest fare per departure date to CSV. The article also explains how to schedule daily runs to track price changes, noting the disclosed cost of roughly $0.20 per 1,000 flights.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Paused CDC Connector Left Replication Slot Holding WAL, Filling Postgres Disk

A Postgres primary database shut down after its disk filled, because a paused CDC connector's replication slot had kept all write-ahead log since it was stopped nine days earlier. Replication slots record the oldest WAL position a consumer needs, and by default Postgres retains that WAL without limit, so it keeps growing while the connector is idle. A quieter variant can occur even when the connector runs, if captured tables rarely change and the confirmed position lags behind. The article recommends heartbeat settings, including a dedicated heartbeat table in the captured database, plus Postgres safeguards such as max_slot_wal_keep_size and idle slot timeouts.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

CrewAI and AutoGen Compared on Coordination, Token Use, and Stability for Multi-Agent Systems

A DEV Community analysis compares CrewAI and AutoGen, the two leading multi-agent orchestration frameworks used in production AI engineering. CrewAI uses role-based hierarchies and deterministic Flows, while AutoGen relies on conversational, message-passing agent groups. In benchmarks of 5,000 multi-step research and code triage tasks on Python 3.12 with GPT-4o and Claude 3.5 Sonnet, CrewAI's structured Flows reportedly cut token waste by 46%. The article says CrewAI also reduced runaway chat loops by 91.4% in business pipelines, whereas AutoGen offers greater flexibility for open-ended, exploratory debugging.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Freelance web designers can protect profits by setting clear client revision limits

A DEV Community guide explains how freelance web designers can manage client revisions without losing money. It recommends stating a fixed number of included revision rounds in the contract instead of offering unlimited changes. The guide advises separating routine revisions from scope changes, such as adding new pages or features, which should be quoted separately. It also suggests tracking feedback, reminding clients of remaining rounds, and securing approvals at staged milestones before development begins.

0 comments Read more at DEV Community