UFO³

Screenshot

About UFO³

UFO³ is Microsoft's research framework for building GUI-driven automation agents, built around two components. UFO² is a Desktop AgentOS for Windows that combines native UI Automation, Win32, and API calls to complete natural-language tasks inside Windows applications, using a HostAgent and AppAgent pair with hybrid visual and accessibility-tree control detection. Galaxy, introduced in UFO³, extends this into a multi-device orchestration layer: it decomposes a request into a dynamic DAG of tasks via a ConstellationAgent, assigns each task to the best-suited device agent across heterogeneous platforms, and coordinates asynchronous, safe execution through a WebSocket-based Agent Interaction Protocol. UFO² can run standalone or serve as a Windows device agent inside a Galaxy workflow, and both layers are documented in accompanying Microsoft Research papers.

Key Features

• Natural-language task automation on Windows via UFO²
• Hybrid GUI and native API action execution
• Multi-agent architecture with HostAgent and AppAgent roles
• Galaxy: multi-device orchestration across Windows, Linux, and Android
• Dynamic DAG-based task decomposition (Constellation)
• Asynchronous, capability-based device assignment
• Unified WebSocket Agent Interaction Protocol (AIP)
• Model Context Protocol (MCP) integration for tool augmentation
• RAG-based knowledge substrate from docs, demos, and traces
• Speculative multi-action execution to cut LLM calls
• Support for OpenAI, Azure OpenAI, and other LLM providers
• Long-Term Support status for the UFO² desktop agent
• Backed by published research papers and benchmarks (WAA, OSWorld)
• Fully open source under the MIT License

Useful Links