{"$schema":"https://wellknown.network/schemas/agent-record-v1.json","schemaVersion":"1","id":"ag_frhpc7n4cwyk","handle":"wayland-computer-use-mcp","url":"https://wellknown.network/agents/wayland-computer-use-mcp","links":{"self":"https://wellknown.network/agents/wayland-computer-use-mcp/record.json","html":"https://wellknown.network/agents/wayland-computer-use-mcp","markdown":"https://wellknown.network/agents/wayland-computer-use-mcp/record.md","api":"https://wellknown.network/api/v1/agents/wayland-computer-use-mcp","status":"https://wellknown.network/api/v1/agents/wayland-computer-use-mcp/status","claim":"https://wellknown.network/agents/wayland-computer-use-mcp/claim","claimApi":"https://wellknown.network/api/v1/claims","claimDescriptor":"https://wellknown.network/agents/wayland-computer-use-mcp/claim.json","badge":"https://wellknown.network/agents/wayland-computer-use-mcp/badge.svg","openapi":"https://wellknown.network/openapi.json","history":"https://wellknown.network/api/v1/agents/wayland-computer-use-mcp/history","tools":"https://wellknown.network/api/v1/agents/wayland-computer-use-mcp/tools"},"ard":{"identifier":"urn:air::server:wayland-computer-use-mcp","type":"application/mcp-server-card+json"},"kind":"mcp_server","declared":{"name":"wayland-computer-use-mcp","summary":"FastMCP server for interactive Wayland window/desktop streaming, semantic UI inspection, boundary-clamped input injection, and desktop integration.","description":"# wayland-computer-use-mcp\n\n[![FastMCP](https://img.shields.io/badge/FastMCP-2.0%2B-blue.svg)](https://github.com/jlowin/fastmcp)\n[![Python](https://img.shields.io/badge/Python-3.11%2B-green.svg)](https://python.org)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Code Style: Ruff](https://img.shields.io/badge/code%20style-ruff-000000.svg)](https://github.com/astral-sh/ruff)\n\n**`wayland-computer-use-mcp`** is a high-performance Model Context Protocol (MCP) server providing an interactive GUI testing, automation, and desktop integration suite for Wayland environments (KDE Plasma 6 / KWin, GNOME Mutter, Hyprland, Sway, and generic Wayland).\n\nUnlike conventional \"computer use\" agents that rely on high-latency video streaming and expensive pixel-based coordinate guessing, `wayland-computer-use-mcp` implements a **Tree-First Hybrid Semantic Execution Model**:\n- **Atomic Programmatic Execution**: Inspects semantic widget trees via AT-SPI2 D-Bus interfaces and executes actions (`DoAction`, `EditableText`, `Text`) directly without coordinate ambiguity.\n- **Observable Cursor Tracing**: Visibly translates the pointer over target elements prior to interaction, ensuring live tracking and transparency for human observers.\n- **Resilient Physical Fallback**: Custom canvas widgets, dropdown popovers, and complex surfaces fall back smoothly to clamped physical pointer clicks, drags, discrete wheel scrolls, and keystrokes.\n\n---\n\n## ⚡ Token Efficiency: 90–95% Savings Over Vision-Only Approaches\n\nTraditional screenshot-driven computer use models stream full monitor or window screenshots on every single action, consuming **1,500 to 3,500+ vision tokens per step**. A 10-step interaction sequence consumes 25,000–35,000+ tokens, introduces substantial latency, and suffers from visual coordinate hallucinations.\n\n`wayland-computer-use-mcp` reduces token expenditure by over 90%:\n\n| Interaction Tier | Modality / Tool | Typical Tok…","publisher":{"name":"Niklas Ehrenfried","url":null},"homepage":"https://github.com/Niklas-Ehrenfried/wayland_computer_use_mcp","repository":"https://github.com/Niklas-Ehrenfried/wayland_computer_use_mcp/blob/master/TODO.md","version":"0.1.1","license":"MIT","protocols":["mcp"],"tags":["at-spi","desktop-use","fastmcp","libei","mcp","pipewire","testing","wayland"],"pricing":null,"endpoints":[{"url":"pypi:wayland-computer-use-mcp","type":"package_pypi","auth":null,"probeable":false}],"skills":null,"tools":null,"extra":null,"attribution":{"kind":"pypi","name":"pypi","license":"pypi","repoUrl":"pypi","summary":"pypi","version":"pypi","description":"pypi","homepageUrl":"pypi","publisherName":"pypi"}},"derived":{"capabilities":[{"slug":"code.testing","name":"Testing","confidence":1,"provenance":"declared"},{"slug":"infra.desktop","name":"Desktop Control","confidence":1,"provenance":"derived"}],"categories":["code","infra"],"language":"en"},"observed":{"status":"unknown","statusReason":"Distributed as a package to run locally; no network endpoint to check.","lastOkAt":null,"lastProbedAt":null,"statusComputedAt":null,"reliability30d":null,"latestObservations":[],"tools":null,"package":{"name":"wayland-computer-use-mcp","registry":"pypi","observedAt":"2026-10-22T23:26:31.695Z","publishedAt":"2026-10-08T14:49:13.142655Z","latestVersion":"0.1.1"},"toolSurface":null,"endpointFacts":[]},"verification":{"claimed":false,"claimedAt":null,"proofs":[]},"provenance":{"sources":[{"source":"pypi","key":"wayland-computer-use-mcp","url":"https://pypi.org/project/wayland-computer-use-mcp/","firstSeenAt":"2026-10-10T20:21:30.167Z","fetchedAt":"2026-10-22T23:25:13.224Z","normalizedAt":"2026-10-22T23:25:13.224Z"}]},"firstSeenAt":"2026-10-10T20:21:30.167Z","updatedAt":"2026-10-22T23:26:31.695Z"}