# smesh-crawler

> smesh-crawler — copyleftdev-smesh-crawler. Use this tool when you need to track changes and gather evidence from websites with precision, as it provides a deterministic web crawling policy and compliance with robots rules. The smesh-crawler solves problems of monitoring online content and identifying historical changepoints, accepting inputs like URLs and policies, and outputting exact evidence and change records. It is particularly useful in contexts requiring version control, such as git-based projects, where accurate tracking of online data is crucial.

Canonical page: https://skillsregistry.net/skills/copyleftdev-smesh-crawler  
JSON: https://api.skillsregistry.net/v1/skills/copyleftdev-smesh-crawler

## Description

Agent-orchestrated web crawler for exact evidence and historical changepoints—with deterministic policy, robots compliance, and local MCP support.

## Trust

- **Trust score (0–1):** 0.22
- **Verification tier:** scanned
- **Last scanned:** 2026-09-28

## Facts

- **Version:** 1.0.0
- **Skill type:** atomic
- **Execution layer:** container
- **Runtime environment:** vm
- **License:** MPL-2.0
- **Updated:** 2026-09-28

## Source

- **Source listing:** [GitHub](https://github.com/copyleftdev/smesh-crawler)

## Use it

Resolve this record through the SkillsRegistry MCP server (no auth, read-only):

```
claude mcp add --transport http --scope user skillsregistry https://api.skillsregistry.net/mcp
```

```json
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "get_skill",
    "arguments": {
      "slug": "copyleftdev-smesh-crawler"
    }
  }
}
```

REST: `GET https://api.skillsregistry.net/v1/skills/copyleftdev-smesh-crawler` · pull for local use: `GET https://api.skillsregistry.net/v1/skills/copyleftdev-smesh-crawler/pull`

---
SkillsRegistry indexes agent skills from public registries and GitHub. Skills we have analysed are scanned with Circle-IR and scored on six dimensions; each listing states its scan coverage. More: https://skillsregistry.net/llms.txt