Skip to main content
TapKit integrates with Lux, the computer-use model from OpenAGI Foundation, enabling AI agents to control iPhones through visual understanding and action execution. The integration provides drop-in replacements for the standard OAGI SDK components:
  • TapKitAsyncActionHandler - Replaces AsyncPyautoguiActionHandler to execute actions on the phone
  • TapKitAsyncImageProvider - Replaces AsyncScreenshotMaker to capture phone screenshots

​
Installation

pip install tapkit

​
Quick Start

import asyncio

from oagi import AsyncDefaultAgent
from tapkit import TapKitClient
from tapkit.oagi import TapKitAsyncActionHandler, TapKitAsyncImageProvider

async def main():
    # Connect to your phone
    client = TapKitClient()
    phone = client.phone("my-phones-name")

    # Set up TapKit providers for Lux
    image_provider = TapKitAsyncImageProvider(phone=phone)
    action_handler = TapKitAsyncActionHandler(phone=phone)

    # Create and run the agent
    agent = AsyncDefaultAgent(
        model="lux-actor-1",
        max_steps=20
    )

    await agent.execute(
        instruction="Open up messages and text Edgar that we're going on the boat tomorrow.",
        action_handler=action_handler,
        image_provider=image_provider
    )

asyncio.run(main())

​
Supported Actions

The TapKit action handler supports the following Lux action types:
ActionPhone Behavior
CLICKTap at coordinates
LEFT_DOUBLEDouble tap
RIGHT_SINGLELong press (2 second hold)
DRAGDrag from one point to another
TYPEType text
SCROLLFlick in a direction
WAITPause for 3 seconds
FINISHReset handler state
CALL_USERRequest user intervention

​
Learn More

For more on Lux models, execution modes, and the OAGI SDK, see the OpenAGI Developer Documentation.