# LLM Evaluation Harness MCP Server

> LLM Evaluation Harness MCP Server — saiarja-llm-eval-mcp. Use this tool when you need to evaluate the performance of large language models, specifically assessing faithfulness, answer relevancy, and context precision. It solves problems related to model validation and quality control, providing outputs such as evaluation scores and threshold checks. The tool accepts RAG outputs as input and is ideal for use cases involving continuous integration gating and on-demand model assessment.

Canonical page: https://skillsregistry.net/skills/saiarja-llm-eval-mcp  
JSON: https://api.skillsregistry.net/v1/skills/saiarja-llm-eval-mcp

## Description

Evaluates RAG outputs on faithfulness, answer relevancy, and context precision using an LLM-as-a-Judge backend. Exposes tools for running evaluations, scoring individual samples, and checking thresholds, enabling CI gating and on-demand assessment via MCP.

## Trust

- **Trust score (0–1):** 0.69
- **Verification tier:** scanned
- **Last scanned:** 2026-08-30

## Facts

- **Version:** 1.0.0
- **Skill type:** atomic
- **Execution layer:** mcp-remote
- **Runtime environment:** api
- **Category:** ai-ml
- **Updated:** 2026-08-30

## Source

- **Source listing:** [Glama](https://glama.ai/mcp/servers/qwq41fm6kn)
- **Repository:** <https://github.com/SaiArja/llm-eval-mcp>

## Use it

Resolve this record through the SkillsRegistry MCP server (no auth, read-only):

```
claude mcp add --transport http --scope user skillsregistry https://api.skillsregistry.net/mcp
```

```json
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "get_skill",
    "arguments": {
      "slug": "saiarja-llm-eval-mcp"
    }
  }
}
```

REST: `GET https://api.skillsregistry.net/v1/skills/saiarja-llm-eval-mcp` · pull for local use: `GET https://api.skillsregistry.net/v1/skills/saiarja-llm-eval-mcp/pull`

---
SkillsRegistry indexes agent skills from public registries and GitHub. Skills we have analysed are scanned with Circle-IR and scored on six dimensions; each listing states its scan coverage. More: https://skillsregistry.net/llms.txt