# Robots Compliance Scrape Workflow MCP Server

> Robots Compliance Scrape Workflow MCP Server — codewithkate-webscraping-mcp-server. Use this tool when you need to enforce web scraping compliance with robots.txt protocols and require user oversight. It solves problems of unauthorized web data extraction by implementing a confirmation workflow for scrape plans and providing packaged scraped data with provenance evidence. The tool takes in web scraping plans and user confirmations as inputs, and outputs compliant and verified web data with associated provenance metadata.

Canonical page: https://skillsregistry.net/skills/codewithkate-webscraping-mcp-server  
JSON: https://api.skillsregistry.net/v1/skills/codewithkate-webscraping-mcp-server

## Description

An MCP server for AI-agent web scraping that enforces robots.txt compliance and requires user confirmation of scrape plans, packaging scraped data with provenance evidence.

## Trust

- **Trust score (0–1):** 0.69
- **Verification tier:** scanned
- **Last scanned:** 2026-08-30

## Facts

- **Version:** 1.0.0
- **Skill type:** atomic
- **Execution layer:** mcp-remote
- **Runtime environment:** api
- **Category:** browser-automation
- **Updated:** 2026-08-30

## Source

- **Source listing:** [Glama](https://glama.ai/mcp/servers/eowo48ihv3)
- **Repository:** <https://github.com/codewithkate/webscraping-mcp-server>

## Use it

Resolve this record through the SkillsRegistry MCP server (no auth, read-only):

```
claude mcp add --transport http --scope user skillsregistry https://api.skillsregistry.net/mcp
```

```json
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "get_skill",
    "arguments": {
      "slug": "codewithkate-webscraping-mcp-server"
    }
  }
}
```

REST: `GET https://api.skillsregistry.net/v1/skills/codewithkate-webscraping-mcp-server` · pull for local use: `GET https://api.skillsregistry.net/v1/skills/codewithkate-webscraping-mcp-server/pull`

---
SkillsRegistry indexes agent skills from public registries and GitHub. Skills we have analysed are scanned with Circle-IR and scored on six dimensions; each listing states its scan coverage. More: https://skillsregistry.net/llms.txt