# Scraper

> Use this tool when you need to extract content from websites efficiently, as it solves problems in research automation, content analysis, and web scraping pipelines by providing raw HTML scraping, markdown conversion, plain text extraction, and link discovery with configurable concurrency and caching. It accepts URLs or batches as input and outputs extracted content in various formats, allowing for targeted extraction via CSS selectors and integration with ScrapeOps proxies. Ideal for use cases requiring resilient and cached web scraping operations, such as data mining and content monitoring workflows.

Canonical page: https://skillsregistry.net/skills/cotdp-scraper  
JSON: https://api.skillsregistry.net/v1/skills/cotdp-scraper

## Description

A web scraping MCP server built by Carrotly AI that provides four core tools for extracting content from websites: raw HTML scraping, markdown conversion, plain text extraction, and link discovery. Built with FastMCP and featuring an extensible provider architecture (currently using requests with exponential backoff retry logic), it supports both single URL and batch operations with configurable concurrency, intelligent disk-based caching with TTL management, CSS selector filtering for targeted content extraction, and optional ScrapeOps proxy integration for JavaScript rendering and anti-bot bypass. The implementation includes a web dashboard for monitoring and testing, comprehensive error handling with graceful degradation, and Docker deployment support, making it useful for research automation, content analysis workflows, and building web scraping pipelines with built-in resilience and caching.

## Trust

- **Trust score (0–1):** 0.65
- **Verification tier:** scanned
- **Last scanned:** 2026-09-19

## Facts

- **Version:** 1.0.0
- **Skill type:** atomic
- **Execution layer:** mcp-remote
- **Runtime environment:** api
- **Category:** cloud-infra
- **Updated:** 2026-09-19

## Source

- **Source listing:** [PulseMCP](https://www.pulsemcp.com/servers/cotdp-scraper)
- **Repository:** <https://github.com/cotdp/scraper-mcp>

## Use it

Resolve this record through the SkillsRegistry MCP server (no auth, read-only):

```
claude mcp add --transport http --scope user skillsregistry https://api.skillsregistry.net/mcp
```

```json
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "get_skill",
    "arguments": {
      "slug": "cotdp-scraper"
    }
  }
}
```

REST: `GET https://api.skillsregistry.net/v1/skills/cotdp-scraper` · pull for local use: `GET https://api.skillsregistry.net/v1/skills/cotdp-scraper/pull`

---
SkillsRegistry indexes agent skills from public registries and GitHub. Skills we have analysed are scanned with Circle-IR and scored on six dimensions; each listing states its scan coverage. More: https://skillsregistry.net/llms.txt