# Crawl4AI RAG

> Use this tool when you need to integrate web crawling and semantic search capabilities into AI workflows, enabling assistants to extract and query web-based content, such as documentation or code repositories. It solves problems related to information gathering, indexing, and retrieval, providing inputs like website URLs and outputs like extracted content and contextual embeddings. Ideal for AI coding workflows, it streamlines the process of gathering and querying web-based data through advanced features like hybrid search and knowledge graph-based hallucination detection.

Canonical page: https://skillsregistry.net/skills/crawl4ai-rag  
JSON: https://api.skillsregistry.net/v1/skills/crawl4ai-rag

## Description

MCP server that integrates web crawling capabilities with RAG (Retrieval-Augmented Generation) functionality, built by Jason Guo to enable AI agents and coding assistants to crawl websites, extract content, and perform semantic search over the collected data. The implementation combines Crawl4AI for web scraping with Supabase for vector storage, supporting advanced features like contextual embeddings, hybrid search, code example extraction, and AI hallucination detection through Neo4j knowledge graphs. Designed for AI coding workflows where assistants need to gather, index, and query web-based documentation or code repositories, with Docker deployment and SearXNG integration for enhanced search capabilities.

## Trust

- **Trust score (0–1):** 0.53
- **Verification tier:** scanned
- **Last scanned:** 2026-09-19

## Facts

- **Version:** 1.0.0
- **Skill type:** atomic
- **Execution layer:** mcp-remote
- **Runtime environment:** api
- **Category:** database
- **Updated:** 2026-09-19

## Source

- **Source listing:** [PulseMCP](https://www.pulsemcp.com/servers/crawl4ai-rag)
- **Repository:** <https://github.com/tokidoo/crawl4ai-rag-mcp>

## Use it

Resolve this record through the SkillsRegistry MCP server (no auth, read-only):

```
claude mcp add --transport http --scope user skillsregistry https://api.skillsregistry.net/mcp
```

```json
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "get_skill",
    "arguments": {
      "slug": "crawl4ai-rag"
    }
  }
}
```

REST: `GET https://api.skillsregistry.net/v1/skills/crawl4ai-rag` · pull for local use: `GET https://api.skillsregistry.net/v1/skills/crawl4ai-rag/pull`

---
SkillsRegistry indexes agent skills from public registries and GitHub. Skills we have analysed are scanned with Circle-IR and scored on six dimensions; each listing states its scan coverage. More: https://skillsregistry.net/llms.txt