# Crawly

> Use this tool when you need to integrate web search and content fetching capabilities into local large language models (LLMs) without relying on remote models. Crawly solves problems related to accessing live web page content and searching the web, providing tools for search, fetch, and page search with evasion of bot detection. It accepts queries and URLs as inputs and returns organic search results, HTML content, or extracted text as outputs, making it suitable for applications requiring up-to-date web data.

Canonical page: https://skillsregistry.net/skills/dshein-alt-crawly  
JSON: https://api.skillsregistry.net/v1/skills/dshein-alt-crawly

## Description

This MCP server enables local LLMs to search the web and fetch live page content without relying on remote models. It provides three tools: search (browser-backed queries on DuckDuckGo, Google, or Yandex returning up to 5 organic results), fetch (browser-rendered content from 1–5 URLs in HTML or extracted text), and page_search (content search on a single page). Uses Playwright/Patchright with persistent profiles and fingerprint patches for bot-detection evasion. Available as a CLI tool, stdio MCP server, or HTTP endpoint.

## Trust

- **Trust score (0–1):** 0.49
- **Verification tier:** scanned
- **Last scanned:** 2026-09-01

## Facts

- **Version:** 1.0.0
- **Skill type:** atomic
- **Execution layer:** mcp-remote
- **Runtime environment:** api
- **Category:** browser-automation
- **Updated:** 2026-09-01

## Source

- **Source listing:** [PulseMCP](https://www.pulsemcp.com/servers/dshein-alt-crawly)
- **Repository:** <https://github.com/dshein-alt/crawly-mcp>

## Use it

Resolve this record through the SkillsRegistry MCP server (no auth, read-only):

```
claude mcp add --transport http --scope user skillsregistry https://api.skillsregistry.net/mcp
```

```json
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "get_skill",
    "arguments": {
      "slug": "dshein-alt-crawly"
    }
  }
}
```

REST: `GET https://api.skillsregistry.net/v1/skills/dshein-alt-crawly` · pull for local use: `GET https://api.skillsregistry.net/v1/skills/dshein-alt-crawly/pull`

---
SkillsRegistry indexes agent skills from public registries and GitHub. Skills we have analysed are scanned with Circle-IR and scored on six dimensions; each listing states its scan coverage. More: https://skillsregistry.net/llms.txt