Getting into the uncanny valley
Karl MacDorman is an associate professor in the human-computer interaction program at the School of Informatics at Indiana University and has been working with the uncanny valley hypothesis, using human participants, for the past year. In his own research on the subject, MacDorman has found scientific support for the hypothesis, which has revealed, among other things, that there are too many contributing factors for it to be narrowed down to a single theory. "If you are interacting with an android and the timing of its speech and gestures is off, this will be uncanny for a different reason than if its eyes are too far apart. This in turn is uncanny for a different reason than if part of its body is open, exposing wires and motors. I have identified about 10 possible causes for the uncanny valley, and I am sure there are many more."
"If you are interacting with an android and the timing of its speech and gestures is off, this will be uncanny for a different reason than if its eyes are too far apart." -- Dr. Karl MacDorman, School of Informatics at Indiana University
The sheer number of factors and the precision of detail required to perfectly approximate humans makes the process seem extraordinarily daunting, though MacDorman believes that the most difficult of these problems is not in creating proportionately accurate humans, but in the timing and the interactivity of character movement: "I suspect the hard part will not be in finding the right physical proportions, but making the movements seem natural and well-timed, especially during interaction." This presents an extraordinarily poignant problem for video games, in which the player has control over the characters for the majority of the time. The characters must not only look realistic and animate accurately, but they must also react to control with perfect timing.
This character may look realistic, but there are many factors which make her also seem uncanny.
Click here to enlarge.
What makes this uncanny?
Of course, there are still plenty of problems within controlled situations. One such example of this can be seen in developer Quantic Dream's recent tech demo for its game Heavy Rain (working title), which has come under fire for uncanniness since it was first revealed at E3 2006. The trailer, which depicts an extremely realistic female character performing a monologue for the camera, is technologically impressive, yet many people have responded to it in a negative way, stating that there's something about it that is unsettling.
MacDorman lists several factors behind what makes the principal character of the video, Mary Smith, seem unnatural, although he recognises that the video displays great technique nonetheless. "Presumably to enhance realism or reduce calculation, the game designers or animators exaggerate the depth of field, so that Mary's face is rendered in focus, but her hair and neck are out of focus and blurry. In addition, there is sometimes a lack of synchronisation with her speech and lip movements, which is very disturbing to people. People 'hear' with their eyes as well as their ears. By this, I mean that if you play an identical sound while looking at a person's lips, the lip movements can cause you to hear the sound differently." He further cites her pale complexion and the fact that she behaves as a sociopath as reasons that most people will have trouble relating to her. "In general, sociopaths tend to seem odder than ordinary people -- not only in their behaviour, but in their facial asymmetries, which reflect developmental disorders -- as anyone can tell by surfing Web sites that list photographs of known sexual offenders." This list of factors shows the breadth and depth of the uncanny valley problem, in that it goes far beyond what seems obvious (something like the lip synching) and into the more nuanced (the difference of the face structure of sociopaths).
And as the animators make these characters more realistic, they have already become used to their less realistic predecessors. So they never get to look at their own creations with fresh eyes."
- Dr. Karl MacDorman, School of Informatics at Indiana University
Posing an even deeper problem for developers, MacDorman believes that animators working from inception to completion will have difficulty witnessing uncanniness in their own projects. "They build up characters that start off looking not particularly realistic and, therefore, not particularly uncanny. And as the animators make these characters more realistic, they have already become used to their less realistic predecessors. So they never get to look at their own creations with fresh eyes. As artists, animators must always rely on their own sense of aesthetics. The problem is that they have lost what is 'common sense' to the rest of us." According to Elspeth Tory, the Animation Project Manager on Ubisoft's upcoming game Assassin's Creed, that's why it's absolutely critical that animators not work on their games in isolation. "You certainly start to get used to a character after a while and can occasionally lose a bit of the objectivity that you had at the beginning of the process. Getting feedback from others, especially the artistic director for animation, is an essential part of the pipeline in order to maintain a standard of believability."