EMG regression with the Virtual Hand¶
End-to-end walkthrough of
examples/synthetic/emg_regression.py:
8-channel synthetic EMG → MyoVerse RMS+MAV+WL features →
multi-output CatBoost regressor → 5-DOF kinematics → expanded
9-vec → smoothed pose pushed to the Virtual Hand.
Why regression and not classification? Two reasons it's the next thing
to learn after emg_classification.py:
- The label loop is different. Recording captures the VHI control hand's kinematic value at the end of each movement, not a class index. That changes both the recorder setup and the training-data iterator.
- The dual-plane idiom is unavoidable. The example uses gRPC
(
SetMovement,SetSessionActive) to drive the control hand to static end poses and LSL to read the resulting kinematics and LSL to push the predicted pose back. Three streams, three roles.
If you haven't yet, read Integrate the Virtual Hand first - that page explains the dual-plane architecture in general; this one walks the specific script.
Run it first¶
The process_launcher panel inside the GUI spawns the EMG generator
and the VHI binary. No env vars required if VHI was installed with
python -m myogestic.tools.install_vhi.
The recording loop, narrated¶
The thing that surprises first-time regression users: recording captures the control hand's settled kinematics, not the button click. The flow:
- Click Launch on EMG Generator and VHI Hand.
- Click a gesture button (Rest / Fist). Two things happen:
vhi_client.set_movement(name, cycle=False)- VHI animates the control hand to the end pose of that movement and holds it. Thecycle=Falseis load-bearing: regression needs the hand to reach and hold the target, not sweep through an open/close cycle.ctrl_outlet.push_sample([CTRL_VALUES[i]])- the EMG generator switches to the corresponding amplitude pattern.- Click Record.
vhi_client.set_session_active(True)disables VHI's local keyboard so the only movement source for this session is your gesture buttons. The session captures EMG samples and theVHI_Control9-channel kinematics stream side by side. - Click Stop Rec. The session ends;
set_session_active(False)restores VHI's local control.
That's the loop. Repeat for every gesture you want the model to regress; pick the recorded sessions in the session manager; click Train.
The five DoFs¶
VHI's VHI_Control outlet is the full 9-channel pose (see the table in
Integrate the Virtual Hand).
For a fake-EMG demo you don't want the regressor to wrestle with the
3-DoF wrist - five DoFs (wrist rotation + four fingers, indices 0, 2,
3, 4, 5) is the right starter target.
The expansion back to 9-DoF for vhi_outlet.push() zero-fills the
unselected channels:
pred_9dof = np.zeros(9, dtype=np.float32)
for i, vhi_idx in enumerate(VHI_DOF_INDICES):
pred_9dof[vhi_idx] = -pred_5dof[i]
The sign flip aligns the regressor's [0, 1] magnitude with VHI's
flexion convention (-1 = full flex).
Training: two iterators, one model¶
The training callback handles two kinds of session transparently:
for emg_window, aligned, _ts in iter_aligned_windows(
kin_paths, "emg", ["vhi_control"], WIN_SECONDS, HOP_SECONDS,
align_window_samples=10,
):
kin = np.abs(aligned["vhi_control"][VHI_DOF_INDICES])
all_X.append(extract_features(emg_window))
all_y.append(kin)
iter_aligned_windows walks every EMG window in the session and
time-aligns a slice of the vhi_control stream to it. The kinematics
slice becomes the regression target. This is the primary path -
sessions with both EMG and kinematics.
for emg_window, _ts, ci in iter_labeled_windows(
label_paths, "emg", WIN_SECONDS, HOP_SECONDS,
classes=data.classes if data.classes else None,
):
kin = np.ones(5) if ci == 1 else np.zeros(5)
all_X.append(extract_features(emg_window))
all_y.append(kin)
iter_labeled_windows is the fallback for sessions that were recorded
before VHI was wired up (no vhi_control store). The script
synthesises a 5-vec target from the class index - Fist → all 1s,
Rest → all 0s. Useful for mixing pre-VHI data into a new training set
without re-recording.
Both iterators ignore class chips the user has un-ticked in the session
manager, courtesy of the classes=data.classes argument.
The model is a single
catboost_regressor(loss_function="MultiRMSE") fit
to the stacked (X, y).
Prediction: smoothed, expanded, pushed¶
@pipeline.predict
def predict(model, features):
pred_5dof = model.predict(features.reshape(1, -1))[0]
pred_5dof = np.clip(pred_5dof, 0, 1)
pred_9dof = np.zeros(9, dtype=np.float32)
for i, vhi_idx in enumerate(VHI_DOF_INDICES):
pred_9dof[vhi_idx] = -pred_5dof[i]
pred_9dof = output_filter(pred_9dof).astype(np.float32)
vhi_outlet.push(pred_9dof)
return {"dof": pred_5dof, "hand": pred_9dof}
Three steps:
- Predict 5 DOFs, clamp to
[0, 1]. - Expand to a 9-vec with the sign flip.
- Apply the live-tunable
FilterControl(defaults to one-euro at 32 Hz) and push to the LSL outlet.
The returned dict feeds pipeline.predictions so widgets like
prediction_label (when configured) can render the current 5-vec.
Layout - six rows, three columns¶
LOGO_CELL_W = 300
WORDMARK_ASPECT = 800 / 540
grid = Grid(
6, 3,
row_height=[Px(LOGO_CELL_W / WORDMARK_ASPECT), *[Fr(1)] * 5],
col_width=[Px(LOGO_CELL_W), Fr(1), Fr(1)],
)
A 6×3 grid: a fixed-height top row sized to the wordmark aspect, then
five equal-share rows below. The left column is fixed at 300 px for the
logo + control panels; columns 2 and 3 are Fr(1) and grow with the
window. See Grid layout for the
Px/Fr rules.
The signal viewer spans rows 0-3 across columns 1-2; stream and log panels share the bottom two rows.
Where to go next¶
examples/synthetic/emg_regression_raulnet.py- swap CatBoost for RaulNetV17 (PyTorch Lightning CNN). Same I/O contract, deeper model. UseTrainer(precision="32-true")- the TorchScript backward has hard-coded fp32 checks that fail under mixed-precision.- Record good training data - how many seconds per gesture, how to avoid posture drift.
- Edge trigger - the gating helper used by the classifier example.