Robot Actions
Drive real Android & iOS devices and web browsers from natural language for mobile + web QA.
Community: Submitted by a user or imported; check the owner before granting accessOnlineAPI key requiredGlobalFreeRead-only
What it can do
- Device List: List available devices (Android and iOS). Shows only free devices and devices currently used by you.
- Device Info: Get detailed info about a device including physical screen size. device_tap/device_swipe use PHYSICAL pixel coordinates — same as page_source bounds. No scaling needed.
- Device Screenshot: Take a screenshot of a device. Returns the JPEG image PLUS, by default, a compact list of labeled UI elements with their bounds (in device-tap coord space — use directly with device
What data it sees
Do you need an account
An API key from the service settings is required
Drive real Android & iOS devices and web browsers from natural language for mobile + web QA. 145+ MCP tools across device control, app management, automation sessions, browser automation, and flow recording / replay.
Auth required: get an API token from your Robot Actions account → Profile → API Tokens, then paste it with the Bearer prefix into the apiToken field when installing.
Sign up: (free tier on the shared test.robotactions.com tenant; paid customers get a dedicated subdomain).
Source / docs:
How to connect
How to connect Robot Actions to Claude
The server address to paste into Claude settings:
https://remote-device-server--krishpavulur.run.tools
- Open Claude (claude.ai or the desktop app).
- Go to Settings → Connectors.
- Click Add custom connector.
- Paste the server address copied below into Remote MCP server URL and click Add.
- In a chat, click + → Connectors and switch the new connector on.
Custom connectors are available on Free, Pro, Max, Team and Enterprise plans (Free is limited to one). On Team and Enterprise an organization Owner adds the connector first under Organization settings → Connectors.
Authorization
After you click Add, a sign-in window for the service opens. Sign in with your own account and approve access. Claude never sees your password.
How to connect Robot Actions to ChatGPT
The server address to paste into ChatGPT settings:
https://remote-device-server--krishpavulur.run.tools
- Open ChatGPT in a browser (chatgpt.com). A Plus, Pro, Business, Enterprise or Edu plan is required.
- Turn on developer mode once: Settings → Apps → Advanced settings → Developer mode.
- Open Settings → Connectors and click Create.
- Fill in the form: Name (anything), Description (one line about what the service does), MCP server URL (copy it below).
- Under Authentication choose OAuth if the service requires sign-in, otherwise None. Click Create.
- In a new chat open + → Apps/Connectors and enable the connector.
OpenAI has renamed this section before (Connectors → Apps/Plugins). If the label differs, search settings for "developer mode". On Business/Enterprise workspaces an admin must allow custom connectors first.
Authorization
On first use ChatGPT opens the service's sign-in window. Sign in and approve access.
How to connect Robot Actions to Cursor
The server address to paste into Cursor settings:
https://remote-device-server--krishpavulur.run.tools
Fastest: click Open in Cursor below and confirm the prompt.
Manually:
- In Cursor open Settings → Cursor Settings → MCP and click Add new global MCP server.
- Paste the JSON copied below into
~/.cursor/mcp.json(per project:.cursor/mcp.jsonin the repo root). - Save the file. The server appears in the MCP list; authorize it there if asked.
Authorization
If the service needs sign-in, an authorize button appears next to the server in the MCP list.
Server tool list (60)
Raw names from tools/list. Only developers need these.
| device_list | List available devices (Android and iOS). Shows only free devices and devices currently used by you. |
| device_info | Get detailed info about a device including physical screen size. device_tap/device_swipe use PHYSICAL pixel coordinates — same as page_source bounds. No scaling needed. |
| device_screenshot | Take a screenshot of a device. Returns the JPEG image PLUS, by default, a compact list of labeled UI elements with their bounds (in device-tap coord space — use directly with device_tap, no scaling). The bundled element list eliminates the second roundtrip to device_page_source and removes visual-estimation guesswork for tap targets that appear in the accessibility tree. Set includeElements=false to skip the page-source fetch and return only the image. |
| device_page_source | Get UI hierarchy XML of a device . Bounds are in physical pixel coordinates — use them directly with device_tap/device_swipe (no scaling needed). Supports element filtering by text/class and compact description format. |
| device_shell | Execute a shell command on an Android device and return output |
| device_terminate_app | Force-stop an Android app by package name. Tries the control channel first, falls back to `am force-stop`, verifies with `pidof`, and surfaces diagnostic info if the app persists (e.g. Samsung FGS resurrection). Returns `{ stopped, wasRunning, transport, retryRecommended }`. |
| device_tap | Tap at (x,y) coordinates on a device screen. Coordinates are in DEVICE TAP-COORD SPACE (the "Tap-coord space" dims printed in the device_screenshot footer; same space as device_page_source bounds). First call starts a device control session (~3s). COORDINATE SOURCES — in priority order: 1. PRIMARY: device_page_source bounds [L,T][R,B] (or the "Labeled elements" block bundled with device_screenshot) → tap center = ((L+R)/2, (T+B)/2). NO scaling. Pixel-exact. 2. FALLBACK ONLY (element not in page_source — image-only widget / custom Canvas): visual estimate from the screenshot pixels, scaled with the formula below. VISUAL → TAP COORDINATE FORMULA (Android): scale = device_width / rendered_chat_width tap_x = visual_x × scale tap_y = visual_y × scale where `device_width` is the "Tap-coord space" width from the device_screenshot footer and `rendered_chat_width` is the "Image" width from the same footer. Both axes share one scale (aspect preserved). The footer prints concrete values per device — never assume any constant. Skipping the scale on a visual estimate is the #1 cause of taps landing in the wrong place — the agent sees a downscaled image but device_tap expects full-resolution tap-space coords. |
| device_swipe | Swipe from (x1,y1) to (x2,y2) on a device screen. Coordinates are PHYSICAL pixels — same as the bounds in device_page_source (no scaling needed). |
| device_focus_events | Read recent Android focus-change events from logcat. Useful AFTER a tap on an input field and BEFORE device_type to verify focus actually landed on the expected element — diagnoses the "type went into the wrong field" race that bites parallel tap+type sequences. Filters to ViewRootImpl (window focus), WindowManager (focus transitions), and InputMethodManager (IME show/hide) events. Returns a short summary line followed by the matching logcat lines (newest last). Empty result = no focus events in the recent window, which usually means the tap did NOT change focus (tap missed, or you tapped a non-focusable element). |
| device_key | Send a keycode to a device. Accepts numeric keycode OR named key (HOME, BACK, ENTER, VOLUME_UP, VOLUME_DOWN, POWER, APP_SWITCH, ESCAPE, DELETE, TAB, SEARCH, MENU). System-policy keys (APP_SWITCH/187, MENU/82, POWER/26) are always sent via shell — they require policy-level handling that the control channel cannot reach. Other keys try the control channel first and fall back to shell on error/timeout. |
| device_type | Type text into the focused field on Android. CRITICAL: call this AFTER device_tap / device_tap_by_text completes — do NOT issue the focusing tap and this type in parallel, or the type will race the focus change and land in the previously-focused field (observed on a banking-app login: username+password concatenated into username box). method="keys" (default): decompose ASCII to keycode events (DOWN/UP with shift) — most reliable; non-ASCII chars (CJK/emoji) auto-fall-back to IME injection for that segment. method="ime": IME injection only — full UTF-8 in one shot, but Samsung IMEs intercept `.`/`@`/`_` as autocomplete/action shortcuts and may trigger system gestures. method="shell": shell `input text` — slowest, and Samsung IME drops `@`/`.`/`_`. Optional clearFirst wipes the field first; pressKey sends a keycode after typing. |
| device_find_element | Find a UI element on an Android device by visible text or content description. Returns element center coordinates in PHYSICAL pixels — use directly with device_tap (no scaling needed). |
| device_elements_in_region | List addressable UI elements (resource-id, text, or content-desc present) whose bounding box intersects the given physical-pixel rectangle. Returns each element with its attributes, bounds, center coords, and ranked locator strategies. Sorted smallest-first so the most specific element comes back first. Use to enumerate the contents of a region (bottom nav, dialog, list section) without parsing the full XML. |
| device_locators_for | Generate ranked locator strategies for a UI element on Android. Pass either coordinates (x, y — picks the smallest containing element) OR a text/contentDesc/resourceId to look up. Returns the same priority-ordered list the UI inspector shows: id, text, content-desc, accessibility selector, XPath. Use these to fill page-object selectors when generating test scripts so you do not have to re-derive them. |
| device_tap_by_text | Find a UI element by text, content-description, or resource-id and tap it. NOT always usable — for elements without stable text/contentDesc/resourceId (image-only icons, custom Canvas widgets, dynamic/localized labels), use `device_tap(x, y)` with bounds from `device_page_source` instead. Both tools are first-class. When multiple nodes share the same text, this tool ranks candidates so an interactive widget (EditText, Button) wins over a passive label (TextView) — pass `resourceId` to pin a specific element. Returns after the tap is dispatched; an additional ~150ms focus-settle wait is included when the matched element is an EditText so a following `device_type` lands in the right field. CRITICAL: call this BEFORE `device_type` sequentially — do NOT issue both in parallel, or the type may race the focus change and write into the previously-focused field. |
| device_dismiss_recent_app | Open the Recents screen and dismiss an app card. Optionally tap "Close all". Useful for clearing background apps or verifying an app was killed. Sends keycode 187 (APP_SWITCH) shell (policy-gated), waits for the UI to settle, then locates and swipes away the target card. Returns { success, action, ... }. |
| device_screen | Control device screen power state and rotation. action="on": wake the screen if asleep (idempotent — checks current wakefulness first). action="off": put the screen to sleep (idempotent). action="rotate": set orientation to portrait/landscape/portrait-reverse/landscape-reverse; auto-rotation gets disabled so the new orientation sticks. Returns { ok, action, screenState, orientation? }. |
| device_panel | Pull down the notification panel, the Quick Settings panel, or collapse open panels. action="notifications": pull down the first panel (notifications). action="settings": pull down Quick Settings (some OEMs require notifications first). action="collapse": close any open panel. Tries control channel first; falls back to `cmd statusbar` shell on error. Returns { ok, action, transport }. |
| device_file_list | List files and directories at a path on the device. Uses the device file-sync protocol (no shell roundtrip). Returns { ok, remotePath, count, entries: [{ name, isDir, size, modTime }] }. Common roots: /sdcard/ (user storage), /sdcard/Download, /sdcard/DCIM. Symlinks are resolved one level. |
| device_file_pull | Pull a file from the device. Returns base64-encoded content. Default size cap 5242880 bytes (5MB); pass maxSizeBytes to override up to 20971520 bytes (20MB). Refuses directories and oversized files before transferring. Returns { ok, remotePath, sizeBytes, content (base64) }. |
| device_file_push | Push a base64-encoded file to the device. Allowed destinations: /sdcard/ (user storage) or /data/local/tmp/ (writable scratch dir — frida-server, tcpdump, etc.). Decoded size hard cap 20971520 bytes (20MB). remotePath must contain no shell metacharacters. For APKs, prefer app_install instead. Returns { ok, remotePath, sizeBytes }. |
| device_app_apk_paths | List APK file paths for an installed app via `pm path`. Returns base APK plus any split APKs (config splits, dynamic feature modules). Each entry includes path and sizeBytes — use device_file_pull on individual paths to fetch the bytes (default 5MB cap, raise via maxSizeBytes up to 20MB). Returns { ok, packageName, apks: [{ path, sizeBytes, role: 'base'|'split' }] }. |
| device_navigate_url | Open a URL in Chrome on an Android device. Much simpler than manually tapping the address bar. Launches Chrome with the given URL directly. |
| device_launch_app | Launch an Android app. Accepts a package name (e.g. com.android.chrome) OR a display name (e.g. "Chrome"). Display-name lookup is case-insensitive substring match against installed apps. |
| device_current_app | Get the currently active (foreground) app on an Android device. Returns package name and activity. |
| device_scroll | Scroll the screen in a direction on an Android device. Uses video coordinate space automatically. No coordinate math needed. |
| device_get_browser_url | Get the current URL loaded in Chrome on an Android device. Returns the URL string without needing to parse page source. |
| device_long_press | Long press at (x,y) on an Android device screen. Triggers context menus, drag handles, and other long-press actions. Coordinates are PHYSICAL pixels (from page_source bounds). |
| device_drag_drop | Drag-and-drop on an Android device: hold at (x1,y1) for holdMs to grab, then move to (x2,y2) over durationMs. Distinct from device_swipe (no hold — scroll-style) and device_long_press (no motion — context menus). Use for app-icon reorder, drag-into-folder, slide gestures that need a deliberate grab. Coordinates are PHYSICAL pixels (from page_source bounds). |
| device_wait_for_element | Poll the UI until an element with the given text or resource-id appears, or timeout. Returns element coordinates in PHYSICAL pixels (ready for device_tap). |
| device_scroll_to_element | Scroll the screen until an element matching ANY of the provided locators (text, contentDesc, resourceId) appears, then return its coordinates in PHYSICAL pixels. Scrolls up to maxScrolls times. |
| device_notifications | Read current notifications on an Android device. Returns structured list of active notifications (app, title, text). |
| device_battery | Get battery status of an Android device (level, charging state, temperature). |
| device_clipboard_get | Read the current clipboard content from an Android device. Reliable on Android 12 and earlier. On Android 13+, both this path and the legacy `cmd clipboard get` path are blocked by the OS (the device-side control process runs as shell, which lacks foreground/READ_CLIPBOARD_IN_BACKGROUND), so this returns an empty-clipboard sentinel rather than the cleartext "No shell command implementation." error. To round-trip text on 13+, set the clipboard via device_clipboard_set and paste it directly — do not rely on reading it back. |
| device_clipboard_set | Set the Android device clipboard via the device control channel. Pass paste: true to have the device inject KEYCODE_PASTE immediately after (built into the device control protocol — no separate Ctrl+V needed). Empty string clears the clipboard. |
| agent_memory_append | Append a block to the calling user's persistent agent memory. Use this when the user explicitly says "remember X", "save this", or "note for next time" — DO NOT auto-update on every interaction. Memory is auto-prepended to your system prompt on every chat in this user's account. Cap: 10240 bytes total; over-cap appends return 413. Markdown is encouraged; keep entries terse and factual (preferences, project context, references), not verbose. |
| agent_memory_replace | Replace the entire memory file for the calling user. USE SPARINGLY — append is the safer primary. Only call replace when memory has drifted wrong, accumulated stale entries, or the user asks for a clean rewrite. Cap: 10240 bytes; over-cap calls return 413. |
| device_record_start | Start a screen recording on an Android device. Recording subscribes to the per-device shared device control session (same session that powers live UI streaming and MCP control — never a duplicate device control process). H.264/H.265 video frames are remuxed server-side into an mp4 on the server host. No on-device storage, no FLAG_SECURE limits, no 180 s on-device recorder cap. The recording auto-stops after maxDurationSec (default 300, max 600) so a forgotten stop call cannot fill disk indefinitely. Returns a recordingId to pass to device_record_stop. Only one recording per device at a time. |
| device_record_stop | Stop a screen recording started by device_record_start. Closes the remux pipeline cleanly so the mp4 has a valid moov atom. Returns hostPath (already on the server host), sizeBytes, durationMs, and the negotiated codec/resolution. |
| ios_record_start | Start a screen recording on an iOS device. Captures the per-device MJPEG broadcast and remuxes server-side into an H.264 mp4 written under /recordings/. An iOS session must already be running (call ios_start_session first). The recording auto-stops after maxDurationSec (default 300, max 600) so a forgotten stop call cannot fill the disk. Returns a recordingId to pass to ios_record_stop. Only one recording per device at a time. |
| ios_record_stop | Stop an iOS screen recording started by ios_record_start. Sends SIGTERM to the remux pipeline so the mp4 has a valid moov atom. Returns hostPath, sizeBytes, durationMs, and codec. |
| ios_record_cleanup | Delete a finished iOS recording mp4 from disk. Pass the httpPath returned by ios_record_stop. Idempotent — deleting a non-existent file is not an error. Refuses to delete recordings that are still being written (call ios_record_stop first). |
| device_network_info | Get network info from an Android device: WiFi SSID, IP address, and signal strength. |
| device_list_apps | List installed apps on an Android device. Optionally filter to user-installed apps only. |
| device_uninstall_app | Uninstall an app from an Android device by package name. |
| device_clear_app_data | Clear all data and cache for an app on an Android device. Equivalent to "Clear Storage" in Settings. |
| device_set_location | Mock GPS coordinates on an Android device for testing location-aware apps. Uses a bundled mock-location helper service (Apache-2.0, auto-installed on first call). API 26+. Scope: every app that reads LocationManager / FusedLocationProviderClient sees the mock fix; apps that check Location.isFromMockProvider (banks, ride-share, Pokémon GO) will detect it and refuse — that's an OS-level signal we can't hide. |
| device_clear_location | Stop mock GPS on an Android device. Counterpart to device_set_location — the helper service stops pushing mocked fixes and apps fall back to the real GPS / network provider. |
| device_launch_app_in_language | Launch an Android app forced into a specific language / locale without changing the device's system settings. Requires Android 13+ (API 33). Uses the per-app LocaleManager API via `cmd locale set-app-locales`. The tool force-stops the app first so cold launch picks up the new locale. Override persists until cleared (or until the app is uninstalled). |
| device_set_device_language | Change the Android device's system language and locale (persistent, affects every app, survives reboot). Uses a bundled locale-change helper with reflection into ActivityManagerNative.updateConfiguration. For per-app testing without changing the whole device, prefer device_launch_app_in_language. Caveats: Samsung One UI / MIUI may re-apply their own locale after a few seconds; Android 14+ requires hidden_api_policy=1 (set automatically); MDM-managed devices may refuse the CHANGE_CONFIGURATION grant. |
| device_clear_app_locale | Drop the per-app locale override on an Android app so it falls back to the device system language. Counterpart to device_launch_app_in_language. Requires Android 13+. |
| device_toggle_wifi | Enable or disable WiFi on an Android device. |
| device_toggle_bluetooth | Enable or disable Bluetooth on an Android device. |
| ios_start_session | Start a iOS automation session on an iOS device . Must be called before any iOS control commands. Takes ~10-30s to launch. |
| ios_end_session | End a iOS automation session on an iOS device, releasing resources. |
| ios_screenshot | Take a screenshot of an iOS device. Returns base64 PNG image. Requires an active iOS automation session (auto-starts if needed). |
| ios_mjpeg_screenshot | Take a fast screenshot of an iOS device via MJPEG stream. Returns JPEG image. Much faster than ios_screenshot (slow path). Requires an active iOS automation session. |
| ios_page_source | Get the UI hierarchy (page source) of an iOS device. Default format is "description" — a compact, readable summary of visible named elements (type, label, position). Use "xml" for the full hierarchy. Requires an active iOS automation session (auto-starts if needed). |
| ios_window_size | Get the screen width of an iOS device from the iOS automation session. Use these dimensions for tap/swipe coordinates. Requires an active iOS automation session. |
| ios_tap | Tap at (x,y) on an iOS device screen. Coordinates are in physical screen points. Requires an active iOS automation session. |