aiagent.club
中文
GitHub repo

D4Vinci/Scrapling

Python Tracked since 2024-10-13 Updated 2026-09-12 View source ↗

#19 of 803 by Stars among GitHub repos

80.3k Stars

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!

About

Selection methods · Fetchers · Spiders · Proxy Rotation · CLI · MCP

Scrapling is an adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl.

Its parser learns from website changes and automatically relocates your elements when pages update. Its fetchers bypass anti-bot systems like Cloudflare Turnstile out of the box. And its spider framework lets you scale up to concurrent, multi-session crawls with pause/resume and automatic proxy rotation - all in a few lines of Python. One library, zero compromises.

Blazing fast crawls with real-time stats and streaming. Built by Web Scrapers for Web Scrapers and regular…

Excerpted from github.com/D4Vinci/Scrapling

Latest metrics

Stars 80.3k 2026-09-12
Forks 8.1k 2026-09-12
Commits 1.6k 2026-09-12
Releases 53 2026-09-12
Watchers 294 2026-09-12
Open issues 8 2026-09-12
Open PRs 12 2026-09-12