SoftTech AI Registry
Microsoft · Microsoft Copilot Studio

Computer use

A Copilot Studio agent tool that uses a vision-and-reasoning model to operate websites and Windows desktop applications through a virtual mouse and keyboard on a configured Windows machine, enabling multi-step GUI tasks where no API integration exists.

Recorded characteristics

Function
Computer use is a tool added to a Copilot Studio agent that "lets your agent interact with and automate tasks on a Windows computer. It works with websites and desktop apps by selecting buttons, choosing menus, and entering text into fields on the screen." The maker describes the task in natural language and the tool performs it on a configured Windows machine using a virtual mouse and keyboard. Microsoft documents an iterative loop of three steps: Perception (it captures screenshots to visually understand the current state of the screen), Reasoning (it evaluates the current state, previous actions and screenshots through a chain-of-thought approach), and Action (clicking, typing or scrolling) repeated "until the task is complete or requires further user input". It processes screen pixel data rather than DOM or API access, can manage multistep workflows and adapts to screen changes. Documented task classes: automated data entry, invoice processing and data extraction, including submitting web forms and returning extracted values as text or JSON. Optional inputs supply dynamic values combined with the instructions at execution time. Available only for agents with generative orchestration turned on.
Data access
Screen pixel data of the machine on which it runs, plus whatever content is visible in the websites and desktop applications it operates (for example PDF invoice contents and web form data in Microsoft's own samples). Screenshots of each action are captured and, where advanced logging is on, stored in Dataverse. Extracted values can be returned to the agent as text or JSON and passed to other agent tools. Field-level detail of application data beyond what is on screen is not established.
Actions
Can take actions
External actions
Yes
Human confirmation
Conditional
Permission basis
Mixed
Administrative control
Environment-level Computer Use feature in the Power Platform admin center (with Store logs in Dataverse on by default, log verbosity, log retention and optional Purview audit export); access to external models must be allowed by an administrator for Anthropic models; generative orchestration must be turned on for the agent; the tool must be added, named, given a model and instructions and saved; machine selection and connection; Credentials to use setting (maker-provided, the default, or end user credentials); stored credentials in Power Platform internal storage or a customer Azure Key Vault; Access control allow list of websites and desktop applications; Enforce HTTPS; Human supervision reviewer list and response time limit; agent publication; machine-side enablement in Power Automate machine settings.
Default state
Disabled
Availability
Generally available. Microsoft's Copilot Studio What's new lists under May 2026: "(General availability) Computer use is now generally available, letting your agents automate web and desktop apps by controlling browsers and desktop applications on behalf of users." Requires an agent with generative orchestration and a Windows machine. Execution environments: bring-your-own Windows machine registered in Power Automate (requires Power Automate for desktop 2.61.132.25266 or later and the web extension), hosted browser (preview) and Cloud PC pool (preview), both powered by Windows 365 for Agents. Region availability is not documented on the operational pages reviewed.
Licensing
Billed through the Agent action feature. Each step consumes 5 Copilot Credits, or 15 Copilot Credits with a premium model. Microsoft's worked example: a four-step timesheet run consumes 20 credits on a standard model or 60 on a premium model.
External model or provider
Microsoft states computer use is "powered by Computer-Using Agents (CUA), an AI model that combines vision capabilities with advanced reasoning to interact with graphical user interfaces". The tool configuration exposes a Model field listing OpenAI Computer-Using Agent (CUA), Standard, generally available; Anthropic Claude Sonnet 4.5, Standard, generally available; Anthropic Claude Sonnet 4.6, Standard, experimental; Anthropic Claude Opus 4.6, Premium, experimental. The human supervision page lists Azure OpenAI as the provider for CUA and records OpenAI Computer Using Agent V1 as retired.
Limitations and uncertainty
Microsoft documents known limitations: success rate varies by task, about 80 percent on web-based tasks and about 35 percent on desktop apps; inconsistent performance for the same task depending on visual or timing changes; difficulty with non-standard or dynamic controls such as dropdowns, date pickers and custom widgets; loops and stuck states; difficulty with detailed graphical interfaces or intricate text manipulation. Microsoft states the tool is not intended for recommending or scoring in sensitive domains such as hiring, healthcare or finance, for making financial transactions, for harmful actions, or for sharing data outside the organization without authorization. Microsoft warns that "in some cases, the AI might take unintended actions due to ambiguous instructions or unexpected content on screen" and that computer-use agents may encounter prompt injection through hidden instructions in screenshots or web pages. Human supervision is probabilistic: it may fail to trigger when a pause is wanted and may trigger unnecessarily, and Microsoft says it should not be relied on as a safety fail-safe or as enforcement of customer rules. Access control prevents interaction with, but not opening of, non-allow-listed sites. Password entry is not supported in some app types (Electron, Java, Unity, games, command-line, Citrix or other virtualized environments). Not established: transactional atomicity, rollback or compensating actions if a run fails after earlier UI changes; retry semantics; behaviour when a UI element cannot be found or an application changes unexpectedly beyond loops and stuck states; run duration or step limits; screenshot access controls and masking of credentials in screenshots; region availability. Adaptation to interface changes is recorded as documented functional behaviour, not a reliability guarantee.

Evidence