Toggle navigation menu
Contact Us
Human Behaviour → Machine Behaviour

Human Behaviour into Machines

Teaching pipelines—demonstration, teleoperation, and imitation—so human know-how becomes machine-usable data.

Human Behaviour into Machines
RESEARCH THESIS

How people teach machines—not what the hand is, and not the finished skill library.

Programming every motion by hand does not scale. The more useful path is to capture how skilled people move, decide, and correct themselves—then turn that into machine behaviour that can be reused and improved.

This program studies demonstration, imitation learning, teleoperation, and language or gesture interfaces that translate human intent into robot-ready skills. We care about libraries that survive beyond a single task or robot.

Outputs feed RoboWorks, AgentFlow, and embodied research programs that need grounded human skill as a starting point.

Behaviour transfer fails when demos are too clean. We study how people actually move, hesitate, and recover using capture data, teleoperation logs, and simulation, and continue into robot teaching cells with partners when a study needs hardware contact.

Scope: This program owns human→machine transfer methods: demos, teleop logs, retargeting, and interactive imitation. Packaged skill contracts live in Physical Skill Intelligence; contact controllers in Dexterous Robotics; mission use of taught skills in Autonomous Physical Work.

WHAT WE BUILD TOWARD

Durable layers of the program.

Capture, teaching, and transfer methods—so people remain the source of hard physical know-how without overlapping the skill-library or hand programs.

  • Demonstration capture
  • Imitation learning
  • Teleoperation interfaces
  • Language-to-action
  • Gesture-to-action
  • Skill libraries
  • Correction and refinement loops
  • Multi-modal behaviour logging
  • Transfer across embodiments
  • Safety-aware teaching
METHODS & ALGORITHMS

From human traces to policies machines can reuse.

This program sits between motion capture, teleoperation logs, simulation, and robot learning. The scientific question is sample-efficient transfer: how few human demonstrations, under how much domain shift, still yield a skill that is safe to promote toward RoboWorks—after sim gates and, when needed, partner hardware checks.

We compare behavioural cloning, interactive imitation (DAgger-style loops), and preference or reward-model variants—always with a retargeting story from human embodiment to robot morphology.

Algorithms we study and implement

  • Behavioural cloning baselines with explicit train/validation splits by user and scene
  • Dataset-aggregation (DAgger-style) interactive imitation to correct compounding error
  • Motion retargeting from human keypoints / mocap to robot joint spaces
  • Gesture and intent models (HMM / sequence models) for coarse human cues
  • Preference learning and ranking losses when absolute rewards are unreliable
  • Safety filters and residual policies that bound exploratory actions during teaching

Things we measure

  • Demo efficiency: success vs. number of human demonstrations
  • Cross-user and cross-scene transfer gap
  • Interventions required per minute during interactive teaching
  • Retargeting error (joint / end-effector) after mapping humans to robots
  • Qualitative failure taxonomy: hesitation, collision near-miss, mode confusion
HOW WE EVALUATE

Teaching quality in data and sim is the evidence—not a single smooth video.

We report who taught, how many demos, what failed in simulation or replay, and whether a new user can extend the skill. Hardware confirmation, when required, is scheduled with robotics partners.

  1. Per-user demo counts and success curves published with the skill card
  2. Held-out scenes that were never in the teaching set
  3. Intervention logs from interactive imitation sessions
  4. Comparison against a non-learning scripted baseline on the same task
  5. Clear ownership: research skill library vs. RoboWorks production skill pack
Embodied Human Intelligence
Captured behaviour needs embodiment to act in the world.
Dexterous Robotics
Hands and contact make many human skills possible.
WHERE THIS SITS

An open field, not a closed stack.

Robot learning labs, teleoperation startups, and foundation-model robotics groups are active neighbours. We focus on durable skill libraries tied to real products.

Adjacent work

  • Imitation-learning research labs
  • Teleoperation platforms
  • Foundation-model robotics startups
  • Industrial teaching-pendant ecosystems
  • Academic human–robot interaction groups

Partners and complements

  • ROS 2 ecosystems
  • NVIDIA Isaac / Isaac Lab
  • RoboWorks
  • AgentFlow
  • Simulation partners
  • Hardware OEMs
HOW IT CAN GROW

A path that compounds over time.

From single-task demos to libraries that transfer across robots, tools, and workplaces.

  1. Reliable demonstration capture in real environments
  2. Skill libraries with correction and versioning
  3. Language and gesture bridges for non-expert teachers
  4. Transfer across robot embodiments
  5. Product hooks into RoboWorks and AgentFlow workflows
NEXT STEP

Explore a collaboration.

Tell us about the sensing problem, robot behaviour, or long-horizon direction you want to shape with us.