# Docs Scraper

> Use this tool when you need to extract structured content from web-based documentation for AI systems, and want to respect site access policies while maintaining clean markdown output. The Docs Scraper tool solves the problem of ingesting documentation at scale, handling single or multiple URLs, sitemaps, and navigation menus, with features like rate limiting and concurrent request handling. It provides a flexible interface for crawling and extracting content, making it ideal for use cases where accurate and compliant documentation scraping is required.

Canonical page: https://skillsregistry.net/skills/felores-docs-scraper  
JSON: https://api.skillsregistry.net/v1/skills/felores-docs-scraper

## Description

Documentation scraping server that enables AI assistants to extract structured content from web-based documentation through multiple crawling strategies. Built with Python and the crawl4ai library, it provides tools for single URL crawling, multi-URL batch processing, sitemap-based crawling, and menu-driven navigation extraction, with features like rate limiting, concurrent request handling, and robots.txt compliance. The implementation is particularly valuable for users who need to ingest documentation into AI systems while respecting site access policies and maintaining clean markdown output.

## Trust

- **Trust score (0–1):** 0.74
- **Verification tier:** verified
- **Last scanned:** 2026-09-19

## Facts

- **Version:** 1.0.0
- **Skill type:** atomic
- **Execution layer:** mcp-remote
- **Runtime environment:** api
- **Category:** browser-automation
- **Updated:** 2026-09-19

## Source

- **Source listing:** [PulseMCP](https://www.pulsemcp.com/servers/felores-docs-scraper)
- **Repository:** <https://github.com/felores/docs_scraper_mcp>

## Use it

Resolve this record through the SkillsRegistry MCP server (no auth, read-only):

```
claude mcp add --transport http --scope user skillsregistry https://api.skillsregistry.net/mcp
```

```json
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "get_skill",
    "arguments": {
      "slug": "felores-docs-scraper"
    }
  }
}
```

REST: `GET https://api.skillsregistry.net/v1/skills/felores-docs-scraper` · pull for local use: `GET https://api.skillsregistry.net/v1/skills/felores-docs-scraper/pull`

---
SkillsRegistry indexes agent skills from public registries and GitHub. Skills we have analysed are scanned with Circle-IR and scored on six dimensions; each listing states its scan coverage. More: https://skillsregistry.net/llms.txt