Humanoid robots could be used to take on physically demanding, hazardous, and repetitive work in spaces built for humans, including shipyards, energy infrastructure, construction sites, and factories. However, a useful robot for these spaces must coordinate locomotion, whole body motion, perception, contact, and operator supervision. In this thesis, we present a way of doing this via a behavior authoring and runtime system. We argue that behavior architecture can be a primary enabler of robot capability, task execution speed, and reliability. We also argue that runtime editability of behaviors enables fast behavior creation, adaptation, extension, and combination. This means if the operator can “dream it” using the available behavior nodes, in a short amount of time, they can get the robot to “do it” repeatably, autonomously, and quickly.
We present a robot-local, runtime-editable behavior architecture that is born from Coactive Design principles developed during the DARPA Robotics Challenge. In that regard, our system strives to be maximally observable, predictable, and directable. Our operator interface remains continuously synchronized for runtime authoring, monitoring, and repair. Our behavior structure is inspired by the Affordance Template Framework, using an object-centric approach to manipulation and action. Uniquely, we combine this with a control data structure inspired by Behavior Trees for behavior organization and runtime-editable reactive logic. We also take a runtime-editable approach to perception, implementing a behavior scene and primitive scene actions that live alongside our physical action primitives, which allows the human behavior author to work around context-dependent perception issues.
Our action primitives are built on top of a whole body controller that supports moving the arms while walking. We achieve speed using a novel concurrent action layering algorithm.
Door traversals are used as the main benchmark task because they expose the full coordination problem in a compact and repeatable setting. They require approach walking, body placement, mechanism perception, grasp selection, handle actuation, interaction with a moving obstacle, and a transition back to locomotion.
The behavior library developed during this work covers more than twenty real-robot task variants, including push and pull doors with knob, push-bar, and lever-handle mechanisms, multi-step exploration sequences, obstacle clearing, and reactive table-to-table manipulation tasks. This behavior system has been deployed on many humanoid robots, such as Boston Dynamics’ DRC Atlas, NASA’s Valkyrie, IHMC and Boardwalk Robotics’ Nadia, Unitree’s H1-2, and IHMC’s Alex.
We evaluate our system across capability, speed, reliability, and speed of behavior creation, adaptation, extension, and combination. We executed our fastest door traversal in 14 seconds and, in a manipulation demonstration, sorted 6 balls by color in 42 seconds under human disturbance. We demonstrated 11 successful push door approach-and-opening repetitions in a row and 12 successful pull door approach-and-opening repetitions in a row. Measured authoring sessions show that we can scratch-author new loco-manipulation behaviors such as push door traversal in hours. Our experiments also demonstrate that we can adapt, extend, and combine existing behaviors to create novel loco-manipulation behaviors in minutes or hours. Finally, we compare our results against measured results from the literature and find our approach to be competitive with the latest results from learned systems. Videos of the work presented in this dissertation are available online at https://www.youtube.com/playlist?list=PLJK5CTyotYqsfgfnXb-09YNFeBose6uEY.