Visual Interfaces#

A Visual Interface (VI) is an external application that provides visual feedback to participants during experiments. MyoGestic communicates with VIs over UDP and manages their lifecycle.

Each VI integration consists of three components:

  • Setup Interface – Launches/stops the external process and manages communication (e.g., UDP sockets).

  • Recording Interface – Manages per-VI recording settings and ground truth collection.

  • Output System – Routes model predictions to the VI during online sessions.

MyoGestic supports multiple active VIs simultaneously. The main window exposes them via active_visual_interfaces, a dict[str, VisualInterface] keyed by the VI short name (e.g., "VHI").

See also

Part 0: Overview and UI Setup for a tutorial on adding a new VI.

Templates#

VisualInterface(*args, **kwargs)

Base class for visual interfaces in the MyoGestic application.

SetupInterfaceTemplate(*args, **kwargs)

Base class for the setup interface of a visual interface.

RecordingInterfaceTemplate(*args, **kwargs)

Base class for the recording interface of a visual interface.

Output System Template#

OutputSystemTemplate(*args, **kwargs)

Represents a base class handling initialization of main window and prediction type for classification or regression tasks.

Pre-implemented Visual Interfaces#

Virtual Hand Interface (VHI)#

Displays two virtual hands (user pose vs target pose) using a Unity application. Records 9 DOF of hand kinematics at 60 Hz and maps 10 gestures.

VirtualHandInterface_SetupInterface(*args, ...)

Setup interface for the Virtual Hand Interface.

VirtualHandInterface_RecordingInterface(...)

Recording interface for the Virtual Hand Interface.

VirtualHandInterface_OutputSystem(*args, ...)

Output system for the Virtual Hand Interface.

Virtual Cursor Interface (VCI)#

A 2-D cursor tracking task. Uses the Cursor task category (CURSOR_TASK_MAP – 5 directions: rest, up, down, right, left).

VirtualCursorInterface_SetupInterface(*args, ...)

Setup interface for the Virtual Cursor Interface.

VirtualCursorInterface_RecordingInterface(...)

Recording interface for the Virtual Cursor Interface.

VirtualCursorInterface_OutputSystem(*args, ...)

Output system for the Virtual Cursor Interface.