# screen-use

> screen-use — tongriyaotxt-screen-use. Use this tool when you need to interact with Windows desktop applications using natural language, enabling AI agents to see and operate UI elements with precision. It solves problems of automating tasks, testing, and accessing desktop apps through accessibility-tree matching and vision-model fallback. Ideal for use cases requiring autonomous visual interaction, such as automated testing, data entry, or virtual assistance.

Canonical page: https://skillsregistry.net/skills/tongriyaotxt-screen-use  
JSON: https://api.skillsregistry.net/v1/skills/tongriyaotxt-screen-use

## Description

Enables AI agents to see, locate UI elements, and operate any Windows desktop app through natural language, using accessibility-tree matching with optional vision-model fallback, plus an autonomous visual loop with introspection and meta-learning.

## Trust

- **Trust score (0–1):** 0.69
- **Verification tier:** scanned
- **Last scanned:** 2026-09-04

## Facts

- **Version:** 1.0.0
- **Skill type:** atomic
- **Execution layer:** mcp-remote
- **Runtime environment:** api
- **Category:** ai-ml
- **Updated:** 2026-09-04

## Source

- **Source listing:** [Glama](https://glama.ai/mcp/servers/ih22qp6t0s)
- **Repository:** <https://github.com/tongriyaotxt/screen-use>

## Use it

Resolve this record through the SkillsRegistry MCP server (no auth, read-only):

```
claude mcp add --transport http --scope user skillsregistry https://api.skillsregistry.net/mcp
```

```json
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "get_skill",
    "arguments": {
      "slug": "tongriyaotxt-screen-use"
    }
  }
}
```

REST: `GET https://api.skillsregistry.net/v1/skills/tongriyaotxt-screen-use` · pull for local use: `GET https://api.skillsregistry.net/v1/skills/tongriyaotxt-screen-use/pull`

---
SkillsRegistry indexes agent skills from public registries and GitHub. Skills we have analysed are scanned with Circle-IR and scored on six dimensions; each listing states its scan coverage. More: https://skillsregistry.net/llms.txt