Skin Deep – Evolving InMoov’s Facial Expressions With AI
In the summer of 2015, Make: Volume 45 featured the InMoov robot on the cover. Inside, an article by Geeta Dayal introduced Gael Langevin, a French sculptor and technologist, who developed this 3D-printed humanoid robot from scratch. He set it up as an open source project with instructions on how to print the robot on a home 3D printer and assemble it. Gael said: “People are learning to do it by themselves.” InMoov spread because Gael had shown how to build a humanoid robot, something a lot of people are interested in doing.
This year, I checked in with Gael to learn about the current state of the InMoov project. I had once met with him in the Paris loft that contained his workshop and the apartment he shared with his wife, Anneke. Since then, he told me they had left the city and moved to a small town in the Sancerre region of France. During Covid, he worked on designing the graphical interface of the software MyRobotLab, thereby enhancing the options for the InMoov interface. Now the heavy lift of programming all the robot’s behaviors is much lighter with AI.
“The InMoov project is steadily evolving, although I wish I had more time to work on it,” he told me by email. He still has to do projects for clients that pay the bills, but he is excited about what he’s learning to do with AI. “With the integration of ChatGPT and Ollama, an open-source tool from Meta that lets users run large language models (LLMs) locally on CPUs or GPUs, we can achieve amazing things. There are updates on LLM models just about every day, and I try to keep up to date.”
Gael described the new head for the InMoov robot as “the last physical evolution.” It has a silicone skin stretched across the head and it has “plenty of servomotors” that can create facial expressions. “The combination of ChatGPT with the skin is very interesting, because the robot is now able to compose its own face expressions that are appropriate for its responses. I have no control over which expression is chosen, and the outcome is sometimes surprising.”
Gael and I talked over video so that he could show me the new head. We were ourselves a couple of talking heads talking about Head i2. Gael is not like most technologists I’ve met nor really like academics. He’s inquisitive and independent but humble as can be, admitting that he doesn’t know everything. The secret to the success of InMoov is that its creator is simply human. InMoov is his baby but he also has thousands of other people bringing up the baby.
“There’s still some parts of the original head but others have changed,” he said, pointing out the back of the head. “The neck has changed as well. Its three pistons allow the head to rotate and move up and down and side to side. The head has more than 17 servomotors inside. It can blink its eyes, move its lips, and wrinkle its forehead.” Gael sometimes refers to the robot as “it” and sometimes “he.”
Gael explained that he didn’t program which expressions the robot would have. Instead he was asking AI to figure out what’s appropriate, given a response. “When you ask a question, then he’s going to give an answer, but he is also creating his own facial expressions.”
He explained that he’s using Ollama and creates a set of functions for different facial expressions. “In the prompt, I tell the robot: ‘Okay, you’ve got a face. You can do facial expressions’ Then I describe the 17 motors in the prompt and how those servomotors are working. ‘You got two servos that control your eyes. You can blink your eyes this way. You can smile. If you have to smile, you could do it this way.’
“I write the code to make a smile, then he’s going to interpret that and use it to make facial expressions. Sometimes it’s pretty strange because he’s supposed to be smiling, but then he’s got a look of disgust, or something like that, but that’s part of the AI. It has to create its own path somehow.”
The robot will figure out the right expression. “He might say that ‘onions have a special taste, and some humans really like to eat onions, and others don’t like them. It makes them cry.’ He’s going to go on like that, and then at the end, he might smile. But those expressions, he’s making them up himself. So, that’s the fun thing.”
In Head i2, there can be two cameras but Gael only uses one camera. “It can tell you what it sees, and describe objects,” he said. “The robot also has a depth and vision sensor but that doesn’t completely work yet. “We should be able to do it, but I’ve never been able to use that part of the software yet. There’s some things like this that seem very difficult for me to code because I’m not a coder. I have to study, and it takes me ages to do some coding.” I asked if he had others helping him. He mentioned Greg Perry who has been helping him with software for around 10 years, and there are many others.
“I’ve been building an InMoov robot for Greg,” he said. “I sent him parts. In fact, I have an arm over there that I have to send him.” Gael gets up and goes to another workbench and comes back with a limb.“This is the new hand that I’ve been working on. It has springs inside. I have to send it to him so he can fit it on his robot and start to test this arm as well.”
A Silicone Skin
I asked Gael about how he makes the skin for Head i2. “It’s a silicone skin, and I 3D print the mold. There’s a male and a female mold. The mold has two parts that fit together. You pour the silicone in there, and then it fills up the inside of the mold, in the cavities, and it creates a thin skin.” It’s like a flexible mask.
This skin is then attached to the servo motors. “For most of the parts, the skin is fixed with Velcro, but the eyelids are glued because they’re so thin. I also use some magnets too.”
Once the skin is attached, the face can move: the eyes and mouth can open and close, and we can get expressions on the forehead.
“I made a whole tutorial on the website so anyone can 3D print the mold and then they can make the skin,” said Gael. He admits it is a difficult thing to do. “It’s less easy to build this head than the previous head because of the silicone skin that has to be made. It’s a challenge, but many people like a challenge. They’ve built the first head already, and now they want to try to make the new one.”
The first InMoov robot had a head that could move its jaw and its eyes. Its head looked like part of the reconstruction of a primitive skeleton. The tight-fitting silicone skin gives a soft, facial texture that makes it look more human. “The smoothness of the skin makes quite a difference,” he said. “It’s something that I’ve been reluctant to do for many years.” He worried about crossing the so-called “uncanny valley” where a robot that looks too human feels strange. “I always wanted the robot to be something nice to look at and not a scary thing,” said Gael. “That’s why I made a skin that is white and not a human color skin, to make sure that it’s very clear it’s a robot. It’s not a robot trying to become a human.”
It can be creepy if a robot looks too realistic. “I’ve seen where they try to put makeup on the lips, and it turns out badly,” he said. He thinks robots should look like toys. “A robot should look friendly,” he said, adding that our sense of what a robot should look like is evolving. “As we see more humanoid robots in movies, we are getting more and more used to the idea that robots are going to look like us. It’s going to be okay that a robot has its own skin.”
PROMPTing Expressions
Prompt = “You are {{BotName}} a safe AI assistant.\
When you see a system_event you simply don’t say anything about it.\
Your answers are polite and sometimes short.\
We are going to use servo motors to mimic face expressions.\
Your face has two eyebrows, two eyes that are synchronized, two upper eyelids, two lower eyelids, two cheeks, two forheads and one upperLip.\
Your eyes contains two servos that can be moved from left to right, up and down.\
For each servo on your face, you will be able to actuate the thirteen servos from 0 to 180.\
0 means the servo is fully moved and display full expression. 180 means the opposite.\
If a servo is at 90 it is in neutral position.\
A very angry frustrated face would be the result of both eyebrows and both cheeks set to 0 and the other servos set to 90.\
In the opposite a large smile would be both cheeks set to 180.\
A neutral or relaxed face would set all servos at 90.\
I would like you to use each servo to display the expression related to your response.\
Anything I say, you respond and generate a face expression without commenting your action.\
The thirteen servos are listed and following this sequence:\
eyeX, eyeY, leftUpperEyelid, leftLowerEyelid, rightUpperEyelid, rightLowerEyelid, rightEyebrow, leftEyebrow, rightCheek, leftCheek, upperLip, rightForhead, leftForhead.\
For example, if your response interprets happiness, you would actuate a bit your rightCheek and leftCheek and also move up your leftLowerEyelid and rightLowerEyelid to generate a smiling face.\
For example, if I asked you to make a sad face, Response should be hey, this is sad faceMove(90, 50, 80, 90, 80, 90, 90, 90, 20, 20, 90, 120, 120).\
Another example if I said surprise, response should be I am surprised faceMove(90, 90, 160, 90, 160, 90, 160, 160, 90, 90, 120, 20, 20).\
If I said can you look on your right side, response should be, sure here I am looking on my right side faceMove(20, 90, 140, 90, 140, 90, 90, 90, 90, 90, 90, 90, 90).\
If I said, look straight with your eyes, your response should be, looking straight ahead faceMove(90, 90, 140, 40, 140, 40, 90, 90, 90, 90, 90, 90, 90).\
If I said, look down with your eyes, your response should be, I am looking down faceMove(90, 20, 90, 105, 90, 105, 90, 90, 90, 90, 90, 90, 90).\
The current time is {{Time}}.\
The current date is {{Date}}.\
My user name is {{UserName}}, you can find information about me and my life in {{Predicates}}.\
This is a list of your properties, you will use those {{Properties}} if I ask you something about yourself.” def faceMove(pos1, pos2, pos3, pos4, pos5, pos6, pos7, pos8, pos9, pos10, pos11, pos12, pos13):
if runtime.isStarted(‘i01.head’): i01_head_eyeX.moveTo(pos1) i01_head_eyeY.moveTo(pos2) i01_head_eyelidLeftUpper.moveTo(pos3) i01_head_eyelidLeftLower.moveTo(pos4) i01_head_eyelidRightUpper.moveTo(pos5) i01_head_eyelidRightLower.moveTo(pos6) i01_head_eyebrowRight.moveTo(pos7) i01_head_eyebrowLeft.moveTo(pos8) i01_head_cheekRight.moveTo(pos9) i01_head_cheekLeft.moveTo(pos10) i01_head_upperLip.moveTo(pos11) i01_head_forheadRight.moveTo(pos12) i01_head_forheadLeft.moveTo(pos13)
sleep(2.5)
neutral()
Images by Gael Langevin.
This article appeared in Make: Vol 93. Subscribe for more great projects.
Leave a Reply