Give the phone a task.
Phone use is the reference agent being developed alongside the phone infrastructure.
The loop
- Read the app’s screen or UI tree.
- Act: tap, type, swipe or open a link.
- Check the result. Continue until the task is complete.
Structured app information can avoid unnecessary screenshots. Screenshots help when the app does not expose readable text.
Two ways to use it
- Your own agent
- Connect through MCP and use the phone tools.
- Reference agent
- The phoneuse harness runs Claude against a task and records a trace. Its setup and examples are still being developed.
Start with a small task
Read a page heading, inspect a setting or navigate an installed app. Verify the result on the phone.
App support depends on the image, installation source and the app itself.