Skip to content

The Display Input API provides low-level control over mouse, keyboard, and window management on a remote display. Use these endpoints to automate UI interactions, capture screenshots, manipulate windows, and build scripted workflows against a virtualized desktop.

Query the current state of the display, cursor, and windows.

GET /api/v1/display/input/display-geometry

Section titled “GET /api/v1/display/input/display-geometry”

Returns the dimensions of the current display.

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
{
"success": true,
"width": 1920,
"height": 1080,
"screen": 0
}
const geometry = await client.display.input.geometry();

Returns the current cursor position, screen, and the window under the cursor.

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
{
"success": true,
"x": 540,
"y": 312,
"screen": 0,
"window": 1234567
}
const location = await client.display.input.mouseLocation();

Returns the currently active (focused) window ID.

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
{
"success": true,
"windowId": 1234567
}
const active = await client.display.input.windowActive();

GET /api/v1/display/window/{windowId}/geometry

Section titled “GET /api/v1/display/window/{windowId}/geometry”

Returns the position and size of a specific window.

NameInTypeRequiredDescription
windowIdpathstringYesWindow ID (decimal or hex)
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
{
"success": true,
"windowId": 1234567,
"x": 100,
"y": 50,
"width": 1280,
"height": 720
}
const geometry = await client.display.input.windowGeometry({ windowId: "0x12d687" });

GET /api/v1/display/window/{windowId}/name

Section titled “GET /api/v1/display/window/{windowId}/name”

Returns the title of a specific window.

NameInTypeRequiredDescription
windowIdpathstringYesWindow ID (decimal or hex)
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
{
"success": true,
"windowId": 1234567,
"name": "Visual Studio Code"
}
const name = await client.display.input.windowName({ windowId: "0x12d687" });

Composite actions that bundle multiple low-level operations into a single request.

Executes a single named action (e.g. mouse/click, keyboard/type) with an optional screenshot.

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
NameTypeRequiredDefaultDescription
actionstringYesAction path (e.g. mouse/click, keyboard/type). Max length: 50
paramsobjectNo{}Action-specific parameters
screenshotbooleanNotrueCapture screenshot after action
screenshotDelayintegerNo100Delay before screenshot in milliseconds. Range: 05000
screenshotRegionstringNoCrop region in format x1,y1,x2,y2
{
"success": true,
"action": {
"success": true,
"action": "mouse/click",
"details": {
"button": 1,
"x": 540,
"y": 312
}
},
"screenshot": {
"timestamp": "2026-01-15T12:34:56.000Z",
"image": {
"data": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAAB...",
"mimeType": "image/png",
"dataUrl": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUg..."
}
}
}
const result = await client.display.input.act({
action: "mouse/click",
params: { x: 540, y: 312, button: 1 },
screenshot: true,
});

Executes a sequence of actions in order. Up to 50 actions per request.

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
NameTypeRequiredDescription
actionsarrayYesList of actions to execute. 150 items

Each action item has the following structure:

NameTypeRequiredDescription
actionstringYesAction path (e.g. mouse/click, keyboard/type)
paramsobjectNoAction-specific parameters
{
"success": true,
"completed": [
{ "index": 0, "action": "mouse/move", "success": true },
{ "index": 1, "action": "mouse/click", "success": true },
{ "index": 2, "action": "keyboard/type", "success": true }
],
"failed": {
"index": 3,
"action": "keyboard/key",
"error": "Unknown keysym: Foo"
},
"skipped": [4, 5]
}
const result = await client.display.input.batch({
actions: [
{ action: "mouse/move", params: { x: 100, y: 100 } },
{ action: "mouse/click", params: { button: 1 } },
{ action: "keyboard/type", params: { text: "hello" } },
],
});

Moves the cursor to a coordinate and clicks.

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
NameTypeRequiredDefaultDescription
xintegerYesTarget X coordinate
yintegerYesTarget Y coordinate
buttonintegerNo1Mouse button (1=left, 2=middle, 3=right, 47=extra). Range: 17
{
"success": true,
"action": "click",
"details": {
"x": 540,
"y": 312,
"button": 1
}
}
await client.display.input.clickAt({ x: 540, y: 312, button: 1 });

Drags the cursor from a start position to an end position.

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
NameTypeRequiredDefaultDescription
startXintegerYesStart X coordinate
startYintegerYesStart Y coordinate
endXintegerYesEnd X coordinate
endYintegerYesEnd Y coordinate
buttonintegerNo1Mouse button (1=left, 2=middle, 3=right, 47=extra). Range: 17
stepsintegerNoNumber of intermediate mouse positions for smooth drag. Range: 11000
{
"success": true,
"action": "drag",
"details": {
"startX": 100,
"startY": 200,
"endX": 800,
"endY": 600,
"button": 1,
"steps": 20
}
}
await client.display.input.drag({
startX: 100,
startY: 200,
endX: 800,
endY: 600,
steps: 20,
});

Selects a range by clicking at a start position and shift-clicking at an end position.

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
NameTypeRequiredDescription
xintegerYesStart X coordinate
yintegerYesStart Y coordinate
endXintegerYesEnd X coordinate
endYintegerYesEnd Y coordinate
{
"success": true,
"action": "select",
"details": {
"x": 120,
"y": 240,
"endX": 480,
"endY": 360
}
}
await client.display.input.select({ x: 120, y: 240, endX: 480, endY: 360 });

Moves the cursor to a coordinate, clicks, and types text in one operation.

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
NameTypeRequiredDescription
xintegerYesTarget X coordinate
yintegerYesTarget Y coordinate
textstringYesText to type. Max length: 10000
delayintegerNoInter-keystroke delay in milliseconds. Range: 01000
{
"success": true,
"action": "type-at",
"details": {
"x": 540,
"y": 312,
"text": "user@example.com",
"delay": 0
}
}
await client.display.input.typeAt({ x: 540, y: 312, text: "user@example.com" });

Waits for a specified duration, with an optional screenshot on completion.

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
NameTypeRequiredDefaultDescription
msintegerYesWait duration in milliseconds. Range: 5030000
screenshotbooleanNofalseCapture screenshot after wait
{
"success": true,
"action": "wait",
"details": {
"ms": 500
},
"screenshot": {
"timestamp": "2026-01-15T12:34:56.000Z",
"image": {
"data": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAAB...",
"mimeType": "image/png",
"dataUrl": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUg..."
}
}
}
await client.display.input.wait({ ms: 500, screenshot: false });

Emergency release of all held inputs (mouse buttons and modifier keys).

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999

This endpoint takes no body.

{
"success": true,
"action": "reset",
"details": {
"released": ["mouse:1", "Shift_L", "ctrl"]
}
}
await client.display.input.reset();

Send key presses, key combinations, and typed text.

Presses one or more key combinations (e.g. ['ctrl+c', 'Return']).

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
NameTypeRequiredDescription
keysarrayYesKey combinations (e.g. ['ctrl+c', 'Return']). 120 items, each max length 100
windowinteger | stringNoTarget window ID
delayintegerNoDelay between key presses in milliseconds. Range: 05000
clearModifiersbooleanNoClear modifier keys before pressing
{
"success": true,
"action": "key",
"details": {
"keys": ["ctrl+c"],
"window": 1234567
}
}
await client.display.input.keyboardKey({ keys: ["ctrl+c"] });

Holds a key down (without releasing it).

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
NameTypeRequiredDescription
keystringYesKey name (X11 keysym, e.g. Shift_L, ctrl). Max length: 100
windowinteger | stringNoTarget window ID
holdMsintegerNoAuto-release after this many milliseconds. Range: 10060000
{
"success": true,
"action": "key-down",
"details": {
"key": "Shift_L",
"holdMs": 1500
}
}
await client.display.input.keyboardKeyDown({ key: "Shift_L", holdMs: 1500 });

Releases a previously held key.

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
NameTypeRequiredDescription
keystringYesKey name (X11 keysym, e.g. Shift_L). Max length: 100
windowinteger | stringNoTarget window ID
{
"success": true,
"action": "key-up",
"details": {
"key": "Shift_L"
}
}
await client.display.input.keyboardKeyUp({ key: "Shift_L" });

Types a string of text at the current cursor position.

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
NameTypeRequiredDescription
textstringYesText to type. Max length: 10000
windowinteger | stringNoTarget window ID
delayintegerNoInter-keystroke delay in milliseconds. Range: 01000
clearModifiersbooleanNoClear modifier keys before typing
{
"success": true,
"action": "type",
"details": {
"text": "Hello, world!",
"window": 1234567
}
}
await client.display.input.keyboardType({ text: "Hello, world!" });

Move the cursor, click, hold, release, and scroll.

Clicks a mouse button. Supports repeated clicks with a delay between them.

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
NameTypeRequiredDefaultDescription
buttonintegerNo1Mouse button (1=left, 2=middle, 3=right, 47=extra). Range: 17
repeatintegerNo1Number of times to repeat the click. Range: 1100
delayintegerNoDelay between repeats in milliseconds. Range: 05000
windowinteger | stringNoTarget window ID (decimal or hex 0x...)
{
"success": true,
"action": "click",
"details": {
"button": 1,
"repeat": 1
}
}
await client.display.input.mouseClick({ button: 1, repeat: 2, delay: 50 });

Double-clicks a mouse button.

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
NameTypeRequiredDefaultDescription
buttonintegerNo1Mouse button (1=left, 2=middle, 3=right, 47=extra). Range: 17
windowinteger | stringNoTarget window ID
{
"success": true,
"action": "double-click",
"details": {
"button": 1
}
}
await client.display.input.mouseDoubleClick({ button: 1 });

Presses and holds a mouse button.

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
NameTypeRequiredDefaultDescription
buttonintegerNo1Mouse button (1=left, 2=middle, 3=right, 47=extra). Range: 17
windowinteger | stringNoTarget window ID
holdMsintegerNoAuto-release after this many milliseconds. Range: 10060000
{
"success": true,
"action": "mouse-down",
"details": {
"button": 1,
"holdMs": 1000
}
}
await client.display.input.mouseDown({ button: 1, holdMs: 1000 });

Releases a previously held mouse button.

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
NameTypeRequiredDefaultDescription
buttonintegerNo1Mouse button (1=left, 2=middle, 3=right, 47=extra). Range: 17
windowinteger | stringNoTarget window ID
{
"success": true,
"action": "mouse-up",
"details": {
"button": 1
}
}
await client.display.input.mouseUp({ button: 1 });

Moves the cursor to an absolute position on the display.

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
NameTypeRequiredDescription
xintegerYesTarget X coordinate. Range: -6553565535
yintegerYesTarget Y coordinate. Range: -6553565535
windowinteger | stringNoTarget window ID
screenintegerNoTarget screen index. Range: 015
syncbooleanNoWait for the move to complete before returning
{
"success": true,
"action": "mouse-move",
"details": {
"x": 540,
"y": 312,
"screen": 0
}
}
await client.display.input.mouseMove({ x: 540, y: 312, screen: 0 });

Moves the cursor by a relative offset from its current position.

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
NameTypeRequiredDescription
xintegerYesHorizontal offset in pixels
yintegerYesVertical offset in pixels
syncbooleanNoWait for the move to complete before returning
{
"success": true,
"action": "mouse-move-relative",
"details": {
"x": 25,
"y": -10
}
}
await client.display.input.mouseMoveRelative({ x: 25, y: -10 });

Scrolls in one of four directions.

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
NameTypeRequiredDefaultDescription
directionstringYesScroll direction. One of: up, down, left, right
clicksintegerNo5Number of scroll clicks. Range: 1100
{
"success": true,
"action": "scroll",
"details": {
"direction": "down",
"clicks": 5
}
}
await client.display.input.mouseScroll({ direction: "down", clicks: 5 });

Focus, move, resize, close, and search for windows on the display.

Closes a window by ID.

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
NameTypeRequiredDescription
windowIdinteger | stringYesWindow ID (decimal or hex 0x...)
{
"success": true,
"action": "window-close",
"details": {
"windowId": 1234567
}
}
await client.display.input.windowClose({ windowId: 1234567 });

Activates (focuses) a window by ID.

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
NameTypeRequiredDescription
windowIdinteger | stringYesWindow ID (decimal or hex 0x...)
{
"success": true,
"action": "window-focus",
"details": {
"windowId": 1234567
}
}
await client.display.input.windowFocus({ windowId: 1234567 });

Minimizes a window by ID.

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
NameTypeRequiredDescription
windowIdinteger | stringYesWindow ID (decimal or hex 0x...)
{
"success": true,
"action": "window-minimize",
"details": {
"windowId": 1234567
}
}
await client.display.input.windowMinimize({ windowId: 1234567 });

Moves a window to a new position. Supports absolute and relative moves.

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
NameTypeRequiredDescription
windowIdinteger | stringYesWindow ID (decimal or hex 0x...)
xintegerYesTarget X coordinate
yintegerYesTarget Y coordinate
syncbooleanNoWait for the move to complete before returning
relativebooleanNoTreat coordinates as relative to the current position
{
"success": true,
"action": "window-move",
"details": {
"windowId": 1234567,
"x": 100,
"y": 50,
"relative": false
}
}
await client.display.input.windowMove({ windowId: 1234567, x: 100, y: 50 });

Raises a window to the top of the stacking order.

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
NameTypeRequiredDescription
windowIdinteger | stringYesWindow ID (decimal or hex 0x...)
{
"success": true,
"action": "window-raise",
"details": {
"windowId": 1234567
}
}
await client.display.input.windowRaise({ windowId: 1234567 });

Resizes a window to a new width and height.

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
NameTypeRequiredDescription
windowIdinteger | stringYesWindow ID (decimal or hex 0x...)
widthintegerYesNew window width. Minimum: 0
heightintegerYesNew window height. Minimum: 0
syncbooleanNoWait for the resize to complete before returning
useHintsbooleanNoSend resize as a size-hint instead of forcing a new size
{
"success": true,
"action": "window-resize",
"details": {
"windowId": 1234567,
"width": 1280,
"height": 720
}
}
await client.display.input.windowResize({ windowId: 1234567, width: 1280, height: 720 });

Searches for windows matching a regex pattern across name, class, and classname fields.

NameInTypeRequiredDescription
displayIdqueryintegerNoDisplay ID to use (overrides the *-display-N.* hostname pattern). Valid range: 1999999
NameTypeRequiredDescription
patternstringYesSearch pattern (regex). Max length: 200
namebooleanNoSearch by window name/title
classbooleanNoSearch by window class
classnamebooleanNoSearch by window classname
onlyVisiblebooleanNoOnly return visible windows
{
"success": true,
"windows": [1234567, 1234568, 7654321]
}
const matches = await client.display.input.windowSearch({
pattern: "Visual Studio",
name: true,
onlyVisible: true,
});