OmniMCP - OmniParser with Model Control Protocol for UI Automation
Wellknown found it in public sources; nobody has proven control of it yet. Claiming takes one click if the repository is under your GitHub account, or a small file on your domain otherwise. Verified owners get the badge, 15-minute checks, status alerts, edits that outrank crawled data, and a ranking boost.
Agents can do it too: POST https://wellknown.network/api/v1/claims with {"agent":"iflow-mcp-openadaptai-omnimcp","method":"well_known_file"} — machine-readable steps at claim.json, guide at /docs/claim.
Everything here was measured by our prober or read from a registry. Nothing is self-reported.
Attributed to the source that supplied each field. Treated as claims, not facts.
# OmniMCP [](https://github.com/OpenAdaptAI/OmniMCP/actions/workflows/ci.yml) [](https://opensource.org/licenses/MIT) [](https://www.python.org/) [](https://github.com/astral-sh/ruff) OmniMCP provides rich UI context and interaction capabilities to AI models through [Model Context Protocol (MCP)](https://github.com/modelcontextprotocol) and [microsoft/OmniParser](https://github.com/microsoft/OmniParser). It focuses on enabling deep understanding of user interfaces through visual analysis, structured planning, and precise interaction execution. ## Core Features - **Visual Perception:** Understands UI elements using OmniParser. - **LLM Planning:** Plans next actions based on goal, history, and visual state. - **Agent Executor:** Orchestrates the perceive-plan-act loop (`omnimcp/agent_executor.py`). - **Action Execution:** Controls mouse/keyboard via `pynput` (`omnimcp/input.py`). - **CLI Interface:** Simple entry point (`cli.py`) for running tasks. - **Auto-Deployment:** Optional OmniParser server deployment to AWS EC2 with auto-shutdown. - **Debugging:** Generates timestamped visual logs per step. ## Overview `cli.py` uses `AgentExecutor` to run a perceive-plan-act loop. It captures the screen (`VisualState`), plans using an LLM (`core.plan_action_for_ui`), and executes actions (`InputController`). ### Demos - **Real Action (Calculator):** `python cli.py` opens Calculator and computes 5*9.  - **Synthetic UI (Login):** `python demo_synthetic.py` uses generated images (no real I/O). *(Note: Pending refactor to use AgentExecutor).* ![OmniMCP Sy…
Mapped onto the structured taxonomy from declared text and observed tool names. Confidence shown for derived entries.
Every source is kept verbatim. Field changes are logged as events.