{"$schema":"https://wellknown.network/schemas/agent-record-v1.json","schemaVersion":"1","id":"ag_jytghzwn4e7c","handle":"iflow-mcp-openadaptai-omnimcp","url":"https://wellknown.network/agents/iflow-mcp-openadaptai-omnimcp","links":{"self":"https://wellknown.network/agents/iflow-mcp-openadaptai-omnimcp/record.json","html":"https://wellknown.network/agents/iflow-mcp-openadaptai-omnimcp","markdown":"https://wellknown.network/agents/iflow-mcp-openadaptai-omnimcp/record.md","api":"https://wellknown.network/api/v1/agents/iflow-mcp-openadaptai-omnimcp","status":"https://wellknown.network/api/v1/agents/iflow-mcp-openadaptai-omnimcp/status","claim":"https://wellknown.network/agents/iflow-mcp-openadaptai-omnimcp/claim","claimApi":"https://wellknown.network/api/v1/claims","claimDescriptor":"https://wellknown.network/agents/iflow-mcp-openadaptai-omnimcp/claim.json","badge":"https://wellknown.network/agents/iflow-mcp-openadaptai-omnimcp/badge.svg","openapi":"https://wellknown.network/openapi.json","history":"https://wellknown.network/api/v1/agents/iflow-mcp-openadaptai-omnimcp/history","tools":"https://wellknown.network/api/v1/agents/iflow-mcp-openadaptai-omnimcp/tools"},"ard":{"identifier":"urn:air::server:iflow-mcp-openadaptai-omnimcp","type":"application/mcp-server-card+json"},"kind":"mcp_server","declared":{"name":"iflow-mcp_openadaptai-omnimcp","summary":"OmniMCP - OmniParser with Model Control Protocol for UI Automation","description":"# OmniMCP\n\n[![CI](https://github.com/OpenAdaptAI/OmniMCP/actions/workflows/ci.yml/badge.svg)](https://github.com/OpenAdaptAI/OmniMCP/actions/workflows/ci.yml)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Python Version](https://img.shields.io/badge/python-3.10%20|%203.11%20|%203.12-blue)](https://www.python.org/)\n[![Code style: ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)\n\nOmniMCP provides rich UI context and interaction capabilities to AI models through [Model Context Protocol (MCP)](https://github.com/modelcontextprotocol) and [microsoft/OmniParser](https://github.com/microsoft/OmniParser). It focuses on enabling deep understanding of user interfaces through visual analysis, structured planning, and precise interaction execution.\n\n## Core Features\n\n- **Visual Perception:** Understands UI elements using OmniParser.\n- **LLM Planning:** Plans next actions based on goal, history, and visual state.\n- **Agent Executor:** Orchestrates the perceive-plan-act loop (`omnimcp/agent_executor.py`).\n- **Action Execution:** Controls mouse/keyboard via `pynput` (`omnimcp/input.py`).\n- **CLI Interface:** Simple entry point (`cli.py`) for running tasks.\n- **Auto-Deployment:** Optional OmniParser server deployment to AWS EC2 with auto-shutdown.\n- **Debugging:** Generates timestamped visual logs per step.\n\n## Overview\n\n`cli.py` uses `AgentExecutor` to run a perceive-plan-act loop. It captures the screen (`VisualState`), plans using an LLM (`core.plan_action_for_ui`), and executes actions (`InputController`).\n\n### Demos\n\n- **Real Action (Calculator):** `python cli.py` opens Calculator and computes 5*9.\n  ![OmniMCP Real Action Demo GIF](images/omnimcp_demo.gif)\n- **Synthetic UI (Login):** `python demo_synthetic.py` uses generated images (no real I/O). *(Note: Pending refactor to use AgentExecutor).*\n  ![OmniMCP Sy…","publisher":null,"homepage":null,"repository":null,"version":"0.1.0","license":null,"protocols":["mcp"],"tags":["mcp"],"pricing":null,"endpoints":[{"url":"pypi:iflow-mcp_openadaptai-omnimcp","type":"package_pypi","auth":null,"probeable":false}],"skills":null,"tools":null,"extra":null,"attribution":{"kind":"pypi","name":"pypi","summary":"pypi","version":"pypi","description":"pypi"}},"derived":{"capabilities":[{"slug":"infra.desktop","name":"Desktop Control","confidence":0.791,"provenance":"derived"},{"slug":"dev.ci-cd","name":"CI/CD & Deploy","confidence":0.779,"provenance":"derived"}],"categories":["dev","infra"],"language":"en"},"observed":{"status":"unknown","statusReason":"Distributed as a package to run locally; no network endpoint to check.","lastOkAt":null,"lastProbedAt":null,"statusComputedAt":null,"reliability30d":null,"latestObservations":[],"tools":null,"package":{"name":"iflow-mcp_openadaptai-omnimcp","registry":"pypi","observedAt":"2026-09-15T19:23:25.875Z","publishedAt":"2026-02-07T11:14:16.660859Z","latestVersion":"0.1.0"},"toolSurface":null,"endpointFacts":[]},"verification":{"claimed":false,"claimedAt":null,"proofs":[]},"provenance":{"sources":[{"source":"pypi","key":"iflow-mcp_openadaptai-omnimcp","url":"https://pypi.org/project/iflow-mcp_openadaptai-omnimcp/","firstSeenAt":"2026-09-09T20:24:43.892Z","fetchedAt":"2026-09-15T19:22:19.210Z","normalizedAt":"2026-09-15T19:22:19.210Z"}]},"firstSeenAt":"2026-09-09T20:24:43.892Z","updatedAt":"2026-09-15T19:23:25.875Z"}