# website-content-crawler

> website-content-crawler — dev55acc-ai-website-content-crawler. Use this tool when you need to extract and structure website content from URLs, solving problems like data mining, content analysis, and web scraping. It takes URLs as input and outputs structured JSON data, including title, markdown, links, word count, and error status. Ideal for use cases requiring automated content retrieval and analysis, such as monitoring website updates or gathering data for machine learning models.

Canonical page: https://skillsregistry.net/skills/dev55acc-ai-website-content-crawler  
JSON: https://api.skillsregistry.net/v1/skills/dev55acc-ai-website-content-crawler

## Description

Enables agents to crawl URLs and retrieve page content as fixed-shape structured JSON, including title, markdown, links, word count, and error status.

## Trust

- **Trust score (0–1):** 0.68
- **Verification tier:** scanned
- **Last scanned:** 2026-09-04

## Facts

- **Version:** 1.0.0
- **Skill type:** atomic
- **Execution layer:** mcp-remote
- **Runtime environment:** api
- **Category:** browser-automation
- **Updated:** 2026-09-04

## Source

- **Source listing:** [Glama](https://glama.ai/mcp/servers/giiplfnhsn)
- **Repository:** <https://github.com/dev55acc-ai/website-content-crawler>

## Use it

Resolve this record through the SkillsRegistry MCP server (no auth, read-only):

```
claude mcp add --transport http --scope user skillsregistry https://api.skillsregistry.net/mcp
```

```json
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "get_skill",
    "arguments": {
      "slug": "dev55acc-ai-website-content-crawler"
    }
  }
}
```

REST: `GET https://api.skillsregistry.net/v1/skills/dev55acc-ai-website-content-crawler` · pull for local use: `GET https://api.skillsregistry.net/v1/skills/dev55acc-ai-website-content-crawler/pull`

---
SkillsRegistry indexes agent skills from public registries and GitHub. Skills we have analysed are scanned with Circle-IR and scored on six dimensions; each listing states its scan coverage. More: https://skillsregistry.net/llms.txt