Robot Actions
Drive real Android & iOS devices and web browsers from natural language for mobile + web QA.
От сообщества: Добавлен пользователем или импортирован; проверьте владельца перед подключениемРаботаетНужен API-ключГлобальныйБесплатноТолько чтение
Что умеет
- Device List: List available devices (Android and iOS). Shows only free devices and devices currently used by you.
- Device Info: Get detailed info about a device including physical screen size. device_tap/device_swipe use PHYSICAL pixel coordinates — same as page_source bounds. No scaling needed.
- Device Screenshot: Take a screenshot of a device. Returns the JPEG image PLUS, by default, a compact list of labeled UI elements with their bounds (in device-tap coord space — use directly with device
Какие данные видит
Нужен ли аккаунт
Нужен API-ключ из настроек сервиса
Drive real Android & iOS devices and web browsers from natural language for mobile + web QA. 145+ MCP tools across device control, app management, automation sessions, browser automation, and flow recording / replay.
Auth required: get an API token from your Robot Actions account → Profile → API Tokens, then paste it with the Bearer prefix into the apiToken field when installing.
Sign up: (free tier on the shared test.robotactions.com tenant; paid customers get a dedicated subdomain).
Source / docs:
Список инструментов сервера (60)
Технические названия из tools/list. Нужны только разработчикам.
| device_list | List available devices (Android and iOS). Shows only free devices and devices currently used by you. |
| device_info | Get detailed info about a device including physical screen size. device_tap/device_swipe use PHYSICAL pixel coordinates — same as page_source bounds. No scaling needed. |
| device_screenshot | Take a screenshot of a device. Returns the JPEG image PLUS, by default, a compact list of labeled UI elements with their bounds (in device-tap coord space — use directly with device_tap, no scaling). The bundled element list eliminates the second roundtrip to device_page_source and removes visual-estimation guesswork for tap targets that appear in the accessibility tree. Set includeElements=false to skip the page-source fetch and return only the image. |
| device_page_source | Get UI hierarchy XML of a device . Bounds are in physical pixel coordinates — use them directly with device_tap/device_swipe (no scaling needed). Supports element filtering by text/class and compact description format. |
| device_shell | Execute a shell command on an Android device and return output |
| device_terminate_app | Force-stop an Android app by package name. Tries the control channel first, falls back to `am force-stop`, verifies with `pidof`, and surfaces diagnostic info if the app persists (e.g. Samsung FGS resurrection). Returns `{ stopped, wasRunning, transport, retryRecommended }`. |
| device_tap | Tap at (x,y) coordinates on a device screen. Coordinates are in DEVICE TAP-COORD SPACE (the "Tap-coord space" dims printed in the device_screenshot footer; same space as device_page_source bounds). First call starts a device control session (~3s). COORDINATE SOURCES — in priority order: 1. PRIMARY: device_page_source bounds [L,T][R,B] (or the "Labeled elements" block bundled with device_screenshot) → tap center = ((L+R)/2, (T+B)/2). NO scaling. Pixel-exact. 2. FALLBACK ONLY (element not in page_source — image-only widget / custom Canvas): visual estimate from the screenshot pixels, scaled with the formula below. VISUAL → TAP COORDINATE FORMULA (Android): scale = device_width / rendered_chat_width tap_x = visual_x × scale tap_y = visual_y × scale where `device_width` is the "Tap-coord space" width from the device_screenshot footer and `rendered_chat_width` is the "Image" width from the same footer. Both axes share one scale (aspect preserved). The footer prints concrete values per device — never assume any constant. Skipping the scale on a visual estimate is the #1 cause of taps landing in the wrong place — the agent sees a downscaled image but device_tap expects full-resolution tap-space coords. |
| device_swipe | Swipe from (x1,y1) to (x2,y2) on a device screen. Coordinates are PHYSICAL pixels — same as the bounds in device_page_source (no scaling needed). |
| device_focus_events | Read recent Android focus-change events from logcat. Useful AFTER a tap on an input field and BEFORE device_type to verify focus actually landed on the expected element — diagnoses the "type went into the wrong field" race that bites parallel tap+type sequences. Filters to ViewRootImpl (window focus), WindowManager (focus transitions), and InputMethodManager (IME show/hide) events. Returns a short summary line followed by the matching logcat lines (newest last). Empty result = no focus events in the recent window, which usually means the tap did NOT change focus (tap missed, or you tapped a non-focusable element). |
| device_key | Send a keycode to a device. Accepts numeric keycode OR named key (HOME, BACK, ENTER, VOLUME_UP, VOLUME_DOWN, POWER, APP_SWITCH, ESCAPE, DELETE, TAB, SEARCH, MENU). System-policy keys (APP_SWITCH/187, MENU/82, POWER/26) are always sent via shell — they require policy-level handling that the control channel cannot reach. Other keys try the control channel first and fall back to shell on error/timeout. |
| device_type | Type text into the focused field on Android. CRITICAL: call this AFTER device_tap / device_tap_by_text completes — do NOT issue the focusing tap and this type in parallel, or the type will race the focus change and land in the previously-focused field (observed on a banking-app login: username+password concatenated into username box). method="keys" (default): decompose ASCII to keycode events (DOWN/UP with shift) — most reliable; non-ASCII chars (CJK/emoji) auto-fall-back to IME injection for that segment. method="ime": IME injection only — full UTF-8 in one shot, but Samsung IMEs intercept `.`/`@`/`_` as autocomplete/action shortcuts and may trigger system gestures. method="shell": shell `input text` — slowest, and Samsung IME drops `@`/`.`/`_`. Optional clearFirst wipes the field first; pressKey sends a keycode after typing. |
| device_find_element | Find a UI element on an Android device by visible text or content description. Returns element center coordinates in PHYSICAL pixels — use directly with device_tap (no scaling needed). |
| device_elements_in_region | List addressable UI elements (resource-id, text, or content-desc present) whose bounding box intersects the given physical-pixel rectangle. Returns each element with its attributes, bounds, center coords, and ranked locator strategies. Sorted smallest-first so the most specific element comes back first. Use to enumerate the contents of a region (bottom nav, dialog, list section) without parsing the full XML. |
| device_locators_for | Generate ranked locator strategies for a UI element on Android. Pass either coordinates (x, y — picks the smallest containing element) OR a text/contentDesc/resourceId to look up. Returns the same priority-ordered list the UI inspector shows: id, text, content-desc, accessibility selector, XPath. Use these to fill page-object selectors when generating test scripts so you do not have to re-derive them. |
| device_tap_by_text | Find a UI element by text, content-description, or resource-id and tap it. NOT always usable — for elements without stable text/contentDesc/resourceId (image-only icons, custom Canvas widgets, dynamic/localized labels), use `device_tap(x, y)` with bounds from `device_page_source` instead. Both tools are first-class. When multiple nodes share the same text, this tool ranks candidates so an interactive widget (EditText, Button) wins over a passive label (TextView) — pass `resourceId` to pin a specific element. Returns after the tap is dispatched; an additional ~150ms focus-settle wait is included when the matched element is an EditText so a following `device_type` lands in the right field. CRITICAL: call this BEFORE `device_type` sequentially — do NOT issue both in parallel, or the type may race the focus change and write into the previously-focused field. |
| device_dismiss_recent_app | Open the Recents screen and dismiss an app card. Optionally tap "Close all". Useful for clearing background apps or verifying an app was killed. Sends keycode 187 (APP_SWITCH) shell (policy-gated), waits for the UI to settle, then locates and swipes away the target card. Returns { success, action, ... }. |
| device_screen | Control device screen power state and rotation. action="on": wake the screen if asleep (idempotent — checks current wakefulness first). action="off": put the screen to sleep (idempotent). action="rotate": set orientation to portrait/landscape/portrait-reverse/landscape-reverse; auto-rotation gets disabled so the new orientation sticks. Returns { ok, action, screenState, orientation? }. |
| device_panel | Pull down the notification panel, the Quick Settings panel, or collapse open panels. action="notifications": pull down the first panel (notifications). action="settings": pull down Quick Settings (some OEMs require notifications first). action="collapse": close any open panel. Tries control channel first; falls back to `cmd statusbar` shell on error. Returns { ok, action, transport }. |
| device_file_list | List files and directories at a path on the device. Uses the device file-sync protocol (no shell roundtrip). Returns { ok, remotePath, count, entries: [{ name, isDir, size, modTime }] }. Common roots: /sdcard/ (user storage), /sdcard/Download, /sdcard/DCIM. Symlinks are resolved one level. |
| device_file_pull | Pull a file from the device. Returns base64-encoded content. Default size cap 5242880 bytes (5MB); pass maxSizeBytes to override up to 20971520 bytes (20MB). Refuses directories and oversized files before transferring. Returns { ok, remotePath, sizeBytes, content (base64) }. |
| device_file_push | Push a base64-encoded file to the device. Allowed destinations: /sdcard/ (user storage) or /data/local/tmp/ (writable scratch dir — frida-server, tcpdump, etc.). Decoded size hard cap 20971520 bytes (20MB). remotePath must contain no shell metacharacters. For APKs, prefer app_install instead. Returns { ok, remotePath, sizeBytes }. |
| device_app_apk_paths | List APK file paths for an installed app via `pm path`. Returns base APK plus any split APKs (config splits, dynamic feature modules). Each entry includes path and sizeBytes — use device_file_pull on individual paths to fetch the bytes (default 5MB cap, raise via maxSizeBytes up to 20MB). Returns { ok, packageName, apks: [{ path, sizeBytes, role: 'base'|'split' }] }. |
| device_navigate_url | Open a URL in Chrome on an Android device. Much simpler than manually tapping the address bar. Launches Chrome with the given URL directly. |
| device_launch_app | Launch an Android app. Accepts a package name (e.g. com.android.chrome) OR a display name (e.g. "Chrome"). Display-name lookup is case-insensitive substring match against installed apps. |
| device_current_app | Get the currently active (foreground) app on an Android device. Returns package name and activity. |
| device_scroll | Scroll the screen in a direction on an Android device. Uses video coordinate space automatically. No coordinate math needed. |
| device_get_browser_url | Get the current URL loaded in Chrome on an Android device. Returns the URL string without needing to parse page source. |
| device_long_press | Long press at (x,y) on an Android device screen. Triggers context menus, drag handles, and other long-press actions. Coordinates are PHYSICAL pixels (from page_source bounds). |
| device_drag_drop | Drag-and-drop on an Android device: hold at (x1,y1) for holdMs to grab, then move to (x2,y2) over durationMs. Distinct from device_swipe (no hold — scroll-style) and device_long_press (no motion — context menus). Use for app-icon reorder, drag-into-folder, slide gestures that need a deliberate grab. Coordinates are PHYSICAL pixels (from page_source bounds). |
| device_wait_for_element | Poll the UI until an element with the given text or resource-id appears, or timeout. Returns element coordinates in PHYSICAL pixels (ready for device_tap). |
| device_scroll_to_element | Scroll the screen until an element matching ANY of the provided locators (text, contentDesc, resourceId) appears, then return its coordinates in PHYSICAL pixels. Scrolls up to maxScrolls times. |
| device_notifications | Read current notifications on an Android device. Returns structured list of active notifications (app, title, text). |
| device_battery | Get battery status of an Android device (level, charging state, temperature). |
| device_clipboard_get | Read the current clipboard content from an Android device. Reliable on Android 12 and earlier. On Android 13+, both this path and the legacy `cmd clipboard get` path are blocked by the OS (the device-side control process runs as shell, which lacks foreground/READ_CLIPBOARD_IN_BACKGROUND), so this returns an empty-clipboard sentinel rather than the cleartext "No shell command implementation." error. To round-trip text on 13+, set the clipboard via device_clipboard_set and paste it directly — do not rely on reading it back. |
| device_clipboard_set | Set the Android device clipboard via the device control channel. Pass paste: true to have the device inject KEYCODE_PASTE immediately after (built into the device control protocol — no separate Ctrl+V needed). Empty string clears the clipboard. |
| agent_memory_append | Append a block to the calling user's persistent agent memory. Use this when the user explicitly says "remember X", "save this", or "note for next time" — DO NOT auto-update on every interaction. Memory is auto-prepended to your system prompt on every chat in this user's account. Cap: 10240 bytes total; over-cap appends return 413. Markdown is encouraged; keep entries terse and factual (preferences, project context, references), not verbose. |
| agent_memory_replace | Replace the entire memory file for the calling user. USE SPARINGLY — append is the safer primary. Only call replace when memory has drifted wrong, accumulated stale entries, or the user asks for a clean rewrite. Cap: 10240 bytes; over-cap calls return 413. |
| device_record_start | Start a screen recording on an Android device. Recording subscribes to the per-device shared device control session (same session that powers live UI streaming and MCP control — never a duplicate device control process). H.264/H.265 video frames are remuxed server-side into an mp4 on the server host. No on-device storage, no FLAG_SECURE limits, no 180 s on-device recorder cap. The recording auto-stops after maxDurationSec (default 300, max 600) so a forgotten stop call cannot fill disk indefinitely. Returns a recordingId to pass to device_record_stop. Only one recording per device at a time. |
| device_record_stop | Stop a screen recording started by device_record_start. Closes the remux pipeline cleanly so the mp4 has a valid moov atom. Returns hostPath (already on the server host), sizeBytes, durationMs, and the negotiated codec/resolution. |
| ios_record_start | Start a screen recording on an iOS device. Captures the per-device MJPEG broadcast and remuxes server-side into an H.264 mp4 written under /recordings/. An iOS session must already be running (call ios_start_session first). The recording auto-stops after maxDurationSec (default 300, max 600) so a forgotten stop call cannot fill the disk. Returns a recordingId to pass to ios_record_stop. Only one recording per device at a time. |
| ios_record_stop | Stop an iOS screen recording started by ios_record_start. Sends SIGTERM to the remux pipeline so the mp4 has a valid moov atom. Returns hostPath, sizeBytes, durationMs, and codec. |
| ios_record_cleanup | Delete a finished iOS recording mp4 from disk. Pass the httpPath returned by ios_record_stop. Idempotent — deleting a non-existent file is not an error. Refuses to delete recordings that are still being written (call ios_record_stop first). |
| device_network_info | Get network info from an Android device: WiFi SSID, IP address, and signal strength. |
| device_list_apps | List installed apps on an Android device. Optionally filter to user-installed apps only. |
| device_uninstall_app | Uninstall an app from an Android device by package name. |
| device_clear_app_data | Clear all data and cache for an app on an Android device. Equivalent to "Clear Storage" in Settings. |
| device_set_location | Mock GPS coordinates on an Android device for testing location-aware apps. Uses a bundled mock-location helper service (Apache-2.0, auto-installed on first call). API 26+. Scope: every app that reads LocationManager / FusedLocationProviderClient sees the mock fix; apps that check Location.isFromMockProvider (banks, ride-share, Pokémon GO) will detect it and refuse — that's an OS-level signal we can't hide. |
| device_clear_location | Stop mock GPS on an Android device. Counterpart to device_set_location — the helper service stops pushing mocked fixes and apps fall back to the real GPS / network provider. |
| device_launch_app_in_language | Launch an Android app forced into a specific language / locale without changing the device's system settings. Requires Android 13+ (API 33). Uses the per-app LocaleManager API via `cmd locale set-app-locales`. The tool force-stops the app first so cold launch picks up the new locale. Override persists until cleared (or until the app is uninstalled). |
| device_set_device_language | Change the Android device's system language and locale (persistent, affects every app, survives reboot). Uses a bundled locale-change helper with reflection into ActivityManagerNative.updateConfiguration. For per-app testing without changing the whole device, prefer device_launch_app_in_language. Caveats: Samsung One UI / MIUI may re-apply their own locale after a few seconds; Android 14+ requires hidden_api_policy=1 (set automatically); MDM-managed devices may refuse the CHANGE_CONFIGURATION grant. |
| device_clear_app_locale | Drop the per-app locale override on an Android app so it falls back to the device system language. Counterpart to device_launch_app_in_language. Requires Android 13+. |
| device_toggle_wifi | Enable or disable WiFi on an Android device. |
| device_toggle_bluetooth | Enable or disable Bluetooth on an Android device. |
| ios_start_session | Start a iOS automation session on an iOS device . Must be called before any iOS control commands. Takes ~10-30s to launch. |
| ios_end_session | End a iOS automation session on an iOS device, releasing resources. |
| ios_screenshot | Take a screenshot of an iOS device. Returns base64 PNG image. Requires an active iOS automation session (auto-starts if needed). |
| ios_mjpeg_screenshot | Take a fast screenshot of an iOS device via MJPEG stream. Returns JPEG image. Much faster than ios_screenshot (slow path). Requires an active iOS automation session. |
| ios_page_source | Get the UI hierarchy (page source) of an iOS device. Default format is "description" — a compact, readable summary of visible named elements (type, label, position). Use "xml" for the full hierarchy. Requires an active iOS automation session (auto-starts if needed). |
| ios_window_size | Get the screen width of an iOS device from the iOS automation session. Use these dimensions for tap/swipe coordinates. Requires an active iOS automation session. |
| ios_tap | Tap at (x,y) on an iOS device screen. Coordinates are in physical screen points. Requires an active iOS automation session. |