# crawl-mcp

> Use this tool when you need to extract content from websites, search for specific data, or crawl entire sites for text, images, or videos. It solves problems related to web data extraction, search, and content analysis, supporting various crawling modes and search types. The crawl-mcp tool takes in URLs, search queries, or site specifications as input and outputs extracted content, search results, or crawled data.

Canonical page: https://skillsregistry.net/skills/gqy20-crawl-mcp  
JSON: https://api.skillsregistry.net/v1/skills/gqy20-crawl-mcp

## Description

MCP server for web crawling, searching, and AI-powered content extraction, supporting single-page, batch, and full-site crawling along with text, news, image, book, and video search.

## Trust

- **Trust score (0–1):** 0.68
- **Verification tier:** scanned
- **Last scanned:** 2026-09-01

## Facts

- **Version:** 1.0.0
- **Skill type:** atomic
- **Execution layer:** mcp-remote
- **Runtime environment:** api
- **Category:** browser-automation
- **Updated:** 2026-09-01

## Source

- **Source listing:** [Glama](https://glama.ai/mcp/servers/my95umioqs)
- **Repository:** <https://github.com/gqy20/crawl-mcp>

## Use it

Resolve this record through the SkillsRegistry MCP server (no auth, read-only):

```
claude mcp add --transport http --scope user skillsregistry https://api.skillsregistry.net/mcp
```

```json
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "get_skill",
    "arguments": {
      "slug": "gqy20-crawl-mcp"
    }
  }
}
```

REST: `GET https://api.skillsregistry.net/v1/skills/gqy20-crawl-mcp` · pull for local use: `GET https://api.skillsregistry.net/v1/skills/gqy20-crawl-mcp/pull`

---
SkillsRegistry indexes agent skills from public registries and GitHub. Skills we have analysed are scanned with Circle-IR and scored on six dimensions; each listing states its scan coverage. More: https://skillsregistry.net/llms.txt