Apple's Machine Learning team conducted a two-phase study to explore user experience design for LLM-based computer use agents. Phase 1 reviewed existing systems and interviewed eight UX/AI practitioners to create a taxonomy covering user prompts, explainability, user control, and more. The work aims to understand optimal user interactions with these UI-interacting agents.
Key Points
- 1.Two-phase UX study for computer agents
- 2.Taxonomy includes prompts, explainability, control
- 3.Interviews with 8 UX/AI experts
Impact Analysis
This research could shape intuitive designs for Apple's future AI agents, enhancing user trust and adoption. It highlights key factors like control and transparency, potentially influencing industry standards for agent UX.
Technical Details
Agents use LLMs to execute commands via UI elements. Taxonomy refined from system reviews and practitioner input. Focuses on interaction paradigms for seamless computer use.