Part 0: Overview and UI Setup

A Visual Interface (VI) is an external graphical application that provides visual feedback to experiment participants (e.g., a virtual hand, a cursor, or a game). MyoGestic communicates with VIs over UDP to stream predictions and receive kinematics.

For example, the Virtual Hand Interface (VHI) displays two virtual hands – one representing the user’s decoded pose and one showing the target – allowing real-time feedback during neural rehabilitation.

Important

A VI is a standalone process that communicates with MyoGestic via network protocols (typically UDP). This ensures that rendering does not block MyoGestic’s real-time processing loop.

Create your standalone VI application before proceeding.

Three Components to Integrate

To integrate a new VI into MyoGestic you must implement three components:

1. Setup Interface (Part 1) – Launches/stops the external process, manages UDP sockets, and handles per-VI configuration in the Setup tab.

2. Recording Interface (Part 2) – Manages per-VI recording settings (e.g., kinematics checkbox, ground truth collection) in the Record tab.

3. Output System (Part 3) – Routes model predictions to the VI during online sessions.

Directory Structure

Each VI lives in its own package under myogestic/gui/widgets/visual_interfaces/:

myogestic/gui/widgets/visual_interfaces/
+-- my_interface/
    +-- __init__.py                # Exports Setup, Recording, Output classes
    +-- setup_interface.py         # SetupInterfaceTemplate subclass
    +-- recording_interface.py     # RecordingInterfaceTemplate subclass
    +-- output_interface.py        # OutputSystemTemplate subclass
    +-- ui/
        +-- my_setup.ui            # Qt Designer file for setup tab
        +-- my_recording.ui        # Qt Designer file for recording tab
        +-- ui_my_setup.py         # Generated by pyside6-uic
        +-- ui_my_recording.py     # Generated by pyside6-uic

Qt Designer Workflow

1. Open Qt Designer and create .ui files for your setup and recording interfaces.

2. Convert them to Python using pyside6-uic:

   pyside6-uic my_setup.ui -o ui_my_setup.py
   pyside6-uic my_recording.ui -o ui_my_recording.py
   

3. Import the generated Ui_* classes in your interface modules.

Required Widget Names

The recording UI must include a widget named groundTruthProgressBar (a QProgressBar). The template base class looks for it during initialisation and will raise an error if it is missing.

Other common widget names (from the VHI recording UI):

• recordRecordingGroupBox – the per-VI QGroupBox

• recordTaskComboBox – per-VI task selector (hidden by default, see Part 2)

• recordUseKinematicsCheckBox – optional kinematics toggle

Registration

After implementing all three components, register them in user_config:

from myogestic.utils.config import CONFIG_REGISTRY
from myogestic.gui.widgets.visual_interfaces.my_interface import (
    MyInterface_SetupInterface,
    MyInterface_RecordingInterface,
)
from myogestic.gui.widgets.visual_interfaces.my_interface.output_interface import (
    MyInterface_OutputSystem,
)

CONFIG_REGISTRY.register_visual_interface(
    name="MYI",  # Short name (used in task maps, output systems, etc.)
    setup_interface_ui=MyInterface_SetupInterface,
    recording_interface_ui=MyInterface_RecordingInterface,
)
CONFIG_REGISTRY.register_output_system("MYI", MyInterface_OutputSystem)