Meet the Researcher Who Teaches Robots to High Five Humans

If our silicon cousins are going to find a peaceful, useful place in our bio-based lives, they're going to need to communicate with us in a way we understand.

Dr. Naomi T. Fitter has been focusing on this emerging field, called spHRI (social-physical Human-Robot Interaction). As the author of many academic papers, the thrust of Dr. Fitter's thesis is that by teaching robots to play bi-directional clapping games, we can engender a sense of trust, cooperation, and engagement between us and our future robotic companions, healthcare assistants, and co-workers.

After an internship at Microsoft Research, and visiting research at the Max Planck Institute for Intelligent Systems in Germany, Dr. Fitter accepted a post-doctoral position at University of Southern California (USC). She recently left Los Angeles to become an Assistant Professor of Robotics at Oregon State University. We spoke on the phone when she was home for the holidays in Cincinnati about how to make robots playful. Here are edited and condensed excerpts from our conversation.

Dr. Fitter, when did you first become interested in robotics, and what was the first one you worked on?
During my undergrad—doing a double BS/BA in Mechanical Engineering and Spanish at University of Cincinnati—I worked on unmanned ground vehicles, including the Bearcat "Cub." This was designed for autonomous navigation challenges like the International Ground Vehicle Competition (IGVC). There, I also started learning about sensor fusion.

What was the first robot you saw—IRL or in pop culture?
IRL, I'm guessing something at a science museum during my childhood. My parents are both musicians but were really good about taking us on lots of educational vacations around the country. Pop culture—Lost in Space? I'm pretty sure the show's Class M-3 Model B-9 General Utility Non-Theorizing Environmental Control Robot was the first example I saw on-screen.

How did you first develop your ideas on social-physical HRI [spHRI]?
I always had a passion/interest for civics and science, and how technology could help society. During my undergrad I took a class on emotional intelligence and wanted to explore further. During experiences helping out in my mother's special education classroom and volunteering at local elementary schools, I realized robots could be useful resources in the classroom environment. Teachers have limited time for one-on-one interaction with students, so robots could be a helpful tool for tutoring or getting students in the right mindset for learning. I wanted to examine what sorts of skills robots would need, so humans could trust them and interact in an engaging way. That's what social-physical HRI is all about. Teaching robots in our lab to play hand-clapping games with humans was a first research step toward this vision.

How do you program the robot to play high five and hand-clapping games?
To develop human-robot high fives, we recorded human high-fiving motions using an accelerometer and magnetic motion tracker and generated a high-five motion model. Our Rethink Robotics Baxter Research Robot used this model to move in appropriate ways while high fiving, and the accelerometer in Baxter's wrist gave us a reliable indication of when human-robot hand contact had occurred.

For human-robot hand clapping games, we first recorded human hand motion via a nine-axis Sparkfun MPU 9150 IMU breakout board. From these demonstrations, we recorded a labeled motion library of 10 common hand-clapping movements from 10 participants. Then this dataset was used to train a Support Vector Machine (SVM) classifier to automatically identify hand-clapping motions from previously unseen participants. Baxter used these motion labels to reciprocate human motions and join people in hand-clapping games.

Did you design and manufacture the Baxter end effector hands at your lab?
Yes, we did. We designed the hands in Solidworks, 3D-printed the hands, and went through many different iterations. The final hand had inlaid silicone-based Eco Flex, which made the hand squishy and pleasant to touch. The Baxter hand is available for download online.

What other tech tools did you use in your research?
I'm a mechanical engineer by training but so much of advanced robotic design requires a broad technical knowledge including controls, machine learning, and computer vision. I am excited by the potential of emerging tools such as OpenPose, which can track human motion. SpHRI, in general, is a really exciting field, but challenging, as we're pursuing interdisciplinary research and collaborating with people from many different disciplines.

What human response did you elicit in your research on hand/end-effector gameplay?
Over the course of the hand-clapping game experiment, people felt more understood by [the robot] Baxter and became more willing to follow its example. Users felt uniformly safe interacting with Baxter. They expressed positive opinions of Baxter and reported fun interacting with the robot. Taken together, our results indicated that this robot achieved viable social-physical interaction with humans and that its ability to both lead and follow systematically changed the human partner's experience.

Did the robot always get it right? Or did you build in errors to make it seem more "real"?
We did [Laughs]. In one study, we added in what we call "robot mischief," or robot errors, intentionally, where Baxter's visual display [screen face] would indicate "I'm not malfunctioning, but rather being playful." In that study we found that people usually preferred a non-mischievous robot, but a few participants strongly favored the playful mischievous robot, and robot mischief would make people pay more attention and think harder while playing with the robot.

Recommended by Our Editors

Inside the Online-Only Grocer Where Robots Get the Job Done

Amazon Robot Sprays 54 Workers With Bear Repellent

This Google Research Scientist Helps Robots Make Good Decisions

After your PhD in Mechanical Engineering and Applied Mechanics at the University of Pennsylvania GRASP Lab, you worked as a postdoctoral scholar with Professor Maja Mataric at USC. What did you work on there?
I was working on telepresence robot research, to see if we could use them to support students who need to miss extended periods of school. We identified potential ways to keep students more connected, building more expressive robot capabilities (LED lights and expressive arms) to augment the existing navigation and video conferencing that most telepresence robots offer.

You've used both PR2 and Baxter robots in your research, but as of the end of 2018, both Willow Garage and Rethink Robotics, which manufactured those models, are now defunct. Where will you get your next robot platforms?
My goal for the next few years will be to develop a research program that's uniquely mine. On a practical level I'm still thinking of which robot platforms to use as I take up my Assistant Professor role at Oregon State University. But I'm currently considering soft robotics solutions and hoping to develop collaborations with leading soft robotics researchers.

Like the corpulent squishy Baymax in Big Hero 6?
That's one type of soft robotics form factor, but I'm also interested in things like origami robots. People approach this area in different ways. Some people make continuum robots made of folded pieces of paper, formed into a robot arm or claw-like end effector, and others think about various ways to make prismatic joints (that extend) and revolute joints (that turn) out of paper or other sheets of material. In general, I think soft robotics is a good option for building new actuators which are inherently safe, so I'll be looking for collaborators in this area.

Who has funded your research to date?
Most of my work, to date, has been funded by the US National Science Foundation through the Graduate Research Fellowship Program and Integrative Graduate Education and Research Traineeship. But as I move into different spaces, such as human-robot exercise games, perhaps for older adults, the National Institutes of Health [NIH] might be another source of support.

Finally, what's next for you?
I'm currently working on a workshop proposal for Robot Sciences and Systems, which will be held in Freiberg, Germany, in June 2019. I'll let you know nearer the time if/when it gets accepted!