# Scrapling

> An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!

## Facts
- **Category:** [Research & Data Agents](https://agentsearchengine.app/category/research-agents)
- **Type:** Infrastructure
- **Pricing:** Open source
- **GitHub stars:** 72,248
- **Language:** Python
- **License:** BSD-3-Clause
- **Last repo activity:** 4 days ago (Jul 30, 2026)
- **Official site:** https://scrapling.readthedocs.io/en/latest/
- **Source code:** https://github.com/D4Vinci/Scrapling
- **Listing on Agent Search Engine:** https://agentsearchengine.app/agents/scrapling
- **Data verified:** July 2026

## Best for
Scraping sites that change layout or block bots, with selectors that self-repair.

## Avoid if
You need a non-Python stack — Scrapling requires Python 3.10 or higher.

## About Scrapling
Selection methods · Fetchers · Spiders · Proxy Rotation · CLI · MCP Scrapling is an adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl. Its parser learns from website changes and automatically relocates your elements when pages update. Its fetchers bypass anti-bot systems like Cloudflare Turnstile out of the box. And its spider framework lets you scale up to concurrent, multi-session crawls with pause/resume and automatic proxy rotation - all in a few lines of Python. One library, zero compromises.

_From the project's README._

## Install

```
pip install scrapling
```

## Alternatives
Scrapling sits in research & data agents. Related tools indexed here:
- [firecrawl](https://agentsearchengine.app/agents/firecrawl) — The API to search, scrape, and interact with the web at scale. · Open source · Infrastructure · 160k★ · TypeScript · AGPL-3.0 · active 1 day ago
- [TrendRadar](https://agentsearchengine.app/agents/trendradar) — AI-driven public opinion & trend monitor with multi-platform aggregation, RSS, and smart alerts. · Open source · Agent · 61k★ · Python · GPL-3.0 · active 17 days ago
- [BettaFish](https://agentsearchengine.app/agents/bettafish) — 微舆：人人可用的多Agent舆情分析助手，打破信息茧房，还原舆情原貌，预测未来走向，辅助决策！从0实现，不依赖任何框架。 · Open source · Agent · 42k★ · Python · GPL-2.0 · active 13 days ago
- [khoj](https://agentsearchengine.app/agents/khoj) — Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. · Open source · Agent · 36k★ · Python · AGPL-3.0 · active 1 day ago
- [storm](https://agentsearchengine.app/agents/storm) — An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations. · Open source · Agent · 31k★ · Python · MIT · active 10 months ago
- [FastGPT](https://agentsearchengine.app/agents/fastgpt) — FastGPT is a knowledge-based platform built on the LLMs, offers a comprehensive suite of out-of-the-box capabilities such as data processing, RAG retrieval, and visual AI workflow orchestration, · Open source · Platform · 29k★ · TypeScript · active today
- [gpt-researcher](https://agentsearchengine.app/agents/gpt-researcher) — An autonomous agent that conducts deep research on any data using any LLM providers · Open source · Agent · 29k★ · Python · Apache-2.0 · active 16 days ago
- [haystack](https://agentsearchengine.app/agents/haystack) — Open-source AI orchestration framework for building context-engineered, production-ready LLM applications. · Open source · Framework · 26k★ · Python · Apache-2.0 · active 2 days ago

---
Source: [Agent Search Engine](https://agentsearchengine.app) — an independent, hand-curated index of AI agents, MCP servers, and frameworks. Rankings are never sold; sponsored placements are labeled and never numbered. Full directory: https://agentsearchengine.app/llms-full.txt