Guides
Integrate the Virtual Hand Interface
The Virtual Hand Interface (VHI) is a separate Godot-based 3-D hand visualization application shipped alongside MyoGestic. It is MyoGestic’s remote target, using LSL for continuous controls and read-backs, and gRPC for discrete controls.
See Drive a remote target for the transport contract, Concepts › Controls for control maps and the control standard for the full rules.
If VHI isn’t installed yet, see Install the Virtual Hand.
RemoteTarget publishes continuous controls over LSL and queues discrete SetControl
commands through RemoteClient. That client queries GetControlManifest for control
capabilities. A separate RecordingClient calls SetRecordingSession to gate recording
and GetRecordingSessionState to read session state and the current movement. The
VHI_Control and VHI_Predict LSL streams return the two hands’ poses.
The one-liner that wires it up
from myogestic.vhi import virtual_hand
vhi = virtual_hand() # resolves install path + gRPC endpointvhi_control = vhi.control_client() # negotiates the control space (v2)recording = vhi.recording_client() # recording session gate + trajectory playbackvirtual_hand() uses an explicit vhi_path or $VHI_PATH first. Otherwise it looks for
an existing install in the writable checkout, then the per-user install root. It reads $VHI_GRPC_HOST / $VHI_GRPC_PORT for the control endpoint
(defaults 127.0.0.1:50051).
The returned InterfaceSpec locates VHI.
It contains no control vocabulary. The running VHI supplies that vocabulary,
so no local table can go stale.
RemoteTarget queries the running VHI for its
controls and names each stream by its address:
from myogestic.controls import ControlLinkfrom myogestic.remote import RemoteTarget
link = ControlLink(CONTROL_MAP, [RemoteTarget(client=vhi_control, interface=vhi)], hz=32)One target drives the whole map, including both hands. You do not name
or count streams yourself.
See Negotiating with the target for what that
resolves and what it refuses, and Binding retries while the target is
unavailable for why this is a
ControlLink. An application with a VHI launch button can bind before VHI starts.
Launching the VHI process
Drop the launcher into your ProcessLauncher panel and the user gets
a Launch/Stop button for VHI:
import sysfrom myogestic.widgets import ProcessLauncher
PROCESSES = [ ( "EMG Generator", [ sys.executable, "-m", "myogestic.tools.emg_generator", "--name", "TestEMG1", "--channels", "8", "--fs", "2048", ], ), *vhi.launcher(),]
launcher = ProcessLauncher(PROCESSES)
@app.uidef ui(ctx): with grid[0, 0]: launcher.ui()vhi.launcher() prefers a packaged binary install when present, unless
godot_bin or $GODOT_BIN requests source mode. It falls back to
godot --path <project> when a source project and Godot are available. Set
$VHI_LAUNCH_MODE=binary or =godot to force one.
If VHI is not installed, launcher() raises FileNotFoundError with the
exact install_vhi command to run. Surface this as a status message:
try: PROCESSES = [*base, *vhi.launcher()]except FileNotFoundError as e: print(f"[demo] {e}", file=sys.stderr) PROCESSES = base # demo still runs; VHI button just absentConnect, or launch
TargetPanel binds to a running VHI or launches it if none answers.
Place it beside the app’s other targets:
from myogestic.controls import ControlLinkConnectorfrom myogestic.widgets import TargetPanel
connector = ControlLinkConnector(link)targets = TargetPanel([vhi], connectors={vhi.name: connector}, title="Hands")
@app.uidef ui(ctx): with grid[0, 0]: targets.ui()Prediction keeps reading link.bus, which is None until VHI answers. When VHI
cannot be launched here, the row says so and its tooltip carries the same
install message launcher() raises.
VHI control names
The running VHI’s manifest is the authority on its controls.
uv run --extra grpc python tools/inspect_control_map.py examples/controls/hand.toml
resolves a map without driving the hand. The address selects the hand:
| address | drives |
|---|---|
vhi.prediction.thumb.flexion, vhi.prediction.thumb.abduction |
the predicted hand’s thumb, two axes |
vhi.prediction.index, .middle, .ring, .little |
the predicted hand’s other four digits, one axis each |
vhi.prediction.wrist.flexion, .abduction, .rotation |
the predicted hand’s wrist |
vhi.control.pose.* |
the same digits on the control hand, the one the operator poses and a recording captures |
vhi.control.gesture |
the control hand’s movement presets, as a discrete control |
vhi.control.gesture’s states are whole-hand poses: Fist, ThumbExtension and the rest of
what that VHI build ships. A preset is a held state and not a number because it commands a
compound shape no single continuous address expresses. VHI supplies the state names, so push
one of those; what you write yourself is debounce_s, a property of your control loop and
not of the hand.
Sliders can pose the operator’s hand while a model drives the predicted one, and a map naming
both is still one RemoteTarget and one bus. Each hand has its own addresses.
examples/start_here/myocontrol.py
uses one map and one bus to drive predicted digits from the model
and cue the participant through vhi.control.gesture on the control hand.
The nine channels of a recorded pose
VHI publishes two read-backs, VHI_Predict and VHI_Control, so a client can see what
actually moved. Both are nine positional float32 channels, whatever the inbound shape. A
recorded session uses this layout, defined by PredictedHandSkeleton and
myogestic.vhi.pose and verified against recorded sessions:
| Index | Joint | Notes |
|---|---|---|
| 0 | Thumb flexion | bones 1/2/3, X axis |
| 1 | Thumb abduction | bones 1/2/3, Z axis |
| 2 | Index flexion | |
| 3 | Middle flexion | |
| 4 | Ring flexion | |
| 5 | Little flexion | |
| 6-8 | Wrist flexion, abduction, rotation | bone 0, which parents every digit |
Measurements established the thumb channel order and wrist behavior.
Channel 0 is thumb flexion; channel 1 is thumb abduction. A recorded fist
has channel 1 at -1.0 because the thumb crosses the fingers. Channels 6-8
drive the wrist. Archived sessions with these channels hardcoded to 0
do not contain recorded wrist movement. Those zeros came from the recorder;
VHI does respond to wrist channels.
A ready-made movement palette
VhiMovementPanel packages “fetch control-hand state in the background, render the movement
chips, dispatch clicks” into one widget. The Hand movements header has a refresh action and
a reachability dot; the chips wrap with the available width and mark the current movement.
A mono count appears below them. When the hand is not running, launch instructions and a
Refresh action replace the list. It reads the recording aid for state and takes the
click handler explicitly. Wire that handler to a control-standard DOF (one degree of
freedom), because dispatching straight at the target would
bypass the debounce:
from myogestic.widgets.vhi.panel import VhiMovementPanel
panel = VhiMovementPanel( recording, lambda state: bus.select("gesture", state),)
@app.uidef ui(ctx): with grid[8, 0]: panel.ui()The handler is where you layer side-effects on a click. Snap a session label, drive a fake generator, whatever the experiment needs:
def _on_movement_click(name: str) -> None: ctrl_outlet.push_sample(...) # e.g. drive the EMG generator bus.select("gesture", name) # deliver + rebase the debouncebus.select
delivers the state immediately and rebases the DOF’s stability gate to prevent
predict ticks from repeating the button action.
Driving the control hand as ground truth
Some workflows want the control hand to move on its own so the recorded kinematics sweep a range. Continuous regression is one, where the control hand is the regression target the model learns. The recording aid is for that:
recording.start_trajectory("Fist", frequency_hz=0.7) # cycles, producing a trajectory...recording.stop_trajectory() # stops and rests the handWhile a trajectory runs it owns the control hand, and discrete DOFs are refused with a reason; nothing silently interrupts it. See Recording is not control for why the sweep lives here and not in the control standard.
Wrap a recording in the session gate so VHI’s local keyboard cannot compete as a movement source:
def _on_record() -> None: app.start_recording() if not recording.set_recording_session(True): app.ctx.log("no VHI recording gate — the keyboard is not blocked")The call returns False without raising when the aid is unavailable.
Decide whether an ungated recording is acceptable for your experiment.
This is an experiment-integrity decision for the caller, so the client does
not make it by raising.
Testing without VHI
print is the cheapest viewer:
@pipeline.predictdef predict(model, features): pose = model.compose_pose(features) print(f"pose: {[f'{v:+.2f}' for v in pose]}") return {"pose": pose}Or point uv run mne-lsl viewer -s vhi.prediction.index at a single DOF
going out, VHI_Predict for all nine coming back. For the gRPC plane, the standard grpcurl
works against the local server when VHI is running. The proto is at
myogestic/remote/_proto/remote_control.proto.
tools/inspect_control.py checks declaration, resolution and wire frames
without launching the Virtual Hand. It also shows target behaviour when VHI is absent.
Common mistakes
See the full Troubleshooting index for symptom-organized debugging, and Controls › Common mistakes for the ones that are about the control standard.
- Forgetting
set_recording_session(False)on session end. VHI keeps ignoring its own keyboard until you toggle it back. - Leaving a recording trajectory running. It keeps the control hand moving and
refuses discrete DOFs.
stop_trajectory()is idempotent, so call it in teardown regardless. - Forgetting
pose_filter.reset()on retrain. The first few smoothed frames blend the new model’s first prediction with the old model’s last; looks like a brief pose drift on every train cycle. See Post-process predictions.
See also
- Install the Virtual Hand - the installer CLI.
- Drive a remote target - the contract VHI serves, and what MyoGestic calls on it.
- Control standard - declaring a map, negotiation, the three layers of smoothing.
- Edge trigger - fire-on-change pattern.
- Examples directory - runnable protocols, including those that wire VHI over LSL and gRPC.
myogestic.vhi.virtual_hand- full signature.myogestic.widgets.vhi.panel.VhiMovementPanel- movement palette API.