# Crawl4AI RAG MCP Server

> Use this tool when you need to integrate web crawling, data storage, and query capabilities for AI agents. It solves problems related to data collection, storage, and retrieval by enabling AI agents to crawl websites, store content in a vector database, and perform Retrieval-Augmented Generation (RAG) queries. The tool takes website URLs as input and outputs stored content and query results, making it ideal for use cases that require efficient data collection and retrieval for AI model training and inference.

Canonical page: https://skillsregistry.net/skills/anshumaan031-web-knowledge-crawler  
JSON: https://api.skillsregistry.net/v1/skills/anshumaan031-web-knowledge-crawler

## Description

An MCP server that integrates Crawl4AI with Supabase to enable AI agents to crawl websites, store content in a vector database, and perform RAG queries.

## Trust

- **Trust score (0–1):** 0.57
- **Verification tier:** scanned
- **Last scanned:** 2026-09-01

## Facts

- **Version:** 1.0.0
- **Skill type:** atomic
- **Execution layer:** mcp-remote
- **Runtime environment:** api
- **Category:** database
- **Updated:** 2026-09-01

## Source

- **Source listing:** [Glama](https://glama.ai/mcp/servers/blvashp17q)
- **Repository:** <https://github.com/Anshumaan031/Web-knowledge-Crawler>

## Use it

Resolve this record through the SkillsRegistry MCP server (no auth, read-only):

```
claude mcp add --transport http --scope user skillsregistry https://api.skillsregistry.net/mcp
```

```json
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "get_skill",
    "arguments": {
      "slug": "anshumaan031-web-knowledge-crawler"
    }
  }
}
```

REST: `GET https://api.skillsregistry.net/v1/skills/anshumaan031-web-knowledge-crawler` · pull for local use: `GET https://api.skillsregistry.net/v1/skills/anshumaan031-web-knowledge-crawler/pull`

---
SkillsRegistry indexes agent skills from public registries and GitHub. Skills we have analysed are scanned with Circle-IR and scored on six dimensions; each listing states its scan coverage. More: https://skillsregistry.net/llms.txt