human operator

Overview

Human Operator is a wearable augmentation system that actively moves your body based off of AI commands. A pair of camera glasses feeds live video to a vision-language model (VLM) while you speak a command like “play a tune.” The model interprets what it sees, plans a sequence of motor actions, and dispatches those commands to hardware that stimulates the muscles controlling your fingers and wrist.

The goal is not to replace your agency, but to offer brief, guided physical assistance, a step toward on-body intelligence where AI output is movement, not just text.

How it works

• Sensing: A camera captures your point of view; a voice trigger starts each interaction.
• Reasoning: Video and speech are sent to Claude, which returns a timed plan of finger and wrist actions.
• Actuation: A Flask receiver routes commands to an Arduino that switches relays on a TENS/EMS unit, delivering pulses through gel electrodes on the forearm and hand.

My contribution: hardware & electrode placement

I built the hardware and did electrode placement for the system: wiring the relay board, Arduino firmware integration, and the EMS stimulator path that turns software commands into muscle contractions.

Electrode placement was the other critical piece. EMS only works when current passes through the right muscle groups. Drawing on back-of-hand actuation research from the Human Computer Integration Lab at UChicago, I adjusted electrode layout and routing so stimulation felt targeted without blocking palmar dexterity. The white scale-like patches visible on the wearable are covering the electrodes placed.

I also ran calibration to find appropiate amplitude, frequency, and pulse-width ranges per channel before we let the AI take over.