How to create a Virtual Character.

What makes a Virtual character 'Real'?

'Hideo Kojima', the Designer behind the 'Metal Gear Solid' games has sparked my interest this week and as it's been a while since my last blog I thought I'd turn this into a larger look at the challenges behind Virtual Characters.

Which Avatars have sparked your imagination - from Film to Science to Games - Use the comments section to let me know examples you can think of and why you think they stand out. :-)

Graphics
Often when asked what makes a virtual character Look real people talk of about graphics or animations. After all it's difficult to consider a character real when they're made of a few hundred polygons with boxed shaped heads. But now characters are can be so detailed that the Uncanny Valley is also mentioned. This  - where we might have a character look 'realistic' yet something doesn't feel right the 'illusion of life' can suddenly drop away (often leaving a very surreal mixed set of emotions with the viewer). Picture for example a character that could look real in still motion but suddenly feels very false when animated. Even a crooked smile can destroy the best of models.

Personally I feel Benjamin Button [VFX by Digital Domain : Directed by David Fincher in 2008] is still one of the best examples of a believable character. If you've not seen it I highly recommend the film. Here's a sneak peak behind the scenes into its effects.

But films can afford massive computations to create each frame - Real-time graphics have to be generated in around 1/30 of a second! Hence different techniques are used. Here's another of my favourites from the same year as Ben' Button [2008] showing Nvidia using multiple texture composites to create a real-time head [a method similar to modern games which use deferred rendering with multiple passes of screen renders]

Computer Games have also advanced with their animation - from marker based motion capture, to phosphor painted or videoed actor faces recording subtle skin movements, ie performance capture.

Here's a good example from 'Janimation' of human facial expression/animation being retargeted onto a CG character using marker based motion capture im 2011. Or take a look at the cutting edge performance capture tech' used behind the game L.A. Noire in 2010


Intelligence
Hold on a minute though. There is a key difference between film and other forms of avatars. It's one thing to watch a character that has been skillfully crafted and presented to us by a whole gaggle ['technical term' ;-)] of artists, but what happens if we can interact with it. The best looking virtual character will seems pretty dull if it responds to us like a 1970's robot.

One way to see how well a computer can mimic human behaviour is the Turing test. Essentially this is measured not by the avatars behaviour but our response to it. ie If 'we' can't tell that a computer is controlling it then it 'must' be behaving like a real person... Hmmm. Personally I don't like this test and find it quite flawed. Nevertheless it has been used many times.

It used to be said that a computer could never beat a Grandmaster at chess. This was because although it was relatively easy to program it to follow the rules of chess, to play at such a high level requires a deeper understanding of the human psyche. So it was quite a shock to many when IBM's
BigBlue defeated Gary Kasparov in May 1997.

I find it interesting to note that this 'test' was measured over a 'game' of chess... So perhaps it is not surprising to find that on the 27th September 2012 a computer games "bot" passed the Turing Test, as it hunted down and killed opponents in a video game was judged to have displayed behaviour that was indistinguishable from a human. Something many had said would be 'impossible'...

If we take this notion a little further we can see computer 'intelligence' [mimicking behaviour] spreading wider afield. For example this summer Google set a new landmark in the field of artificial intelligence with software that learnt how to recognise cats, people, and other things simply by watching YouTube videos . This takes us into the realms of 'learning behaviour' and neural networks... something I'll come back to another day I'm sure. What will it mean when search engines adopt 'intuitive behviour'?

Games are often the first adopters to new technology and being able to 'deliver' realistic and believable character to our living rooms has been a dream goal to many a developer. How though will this effect us - We may accept Tamagotchie's needing feeding, but how would we respond to a computer not wanting to play because it feels a little down..? ;-)
Microsoft have skirted around the concept of realistic avatars several times. In particular
'Project Natal' [June 2009] gave us a glimpse of what may be to come.

Character & Personality
The true beauty of this technology for me is that we are at last able to put to test the subject that philosophers have debated for 1000's of years. What makes us human and how do we define 'the person'. Modern culture refers to us as 'Mind, Body and Spirit'. Perhaps this is why Virtual Avatars are following the same path. ie Body = Graphics, Mind = Artificial Intelligence, and Spirit... hmmm this is a more tricky one.

If we are make a virtual character believable it not only must be visually detailed and 'perform' as if it has an 'awareness', it crucially needs one more thing. 'Spirit' - and how can we recreate that?
Well perhaps this can be represented by the traits which we associate with a persons character of personality. Do we care about the character enough to value their 'life'. This, imo, is the real challenge behind making a Virtual Character believable - sure we need the graphics, animations and A.I. (artificial intelligence), but can we go further? I say yes we can but it may not be quite the way we might imagine it to be. Just as with the Turing test, we may only need to mimic personality in order for us to believe in it. After all how many people name their cars and anthropomorphise them?

Video games have a unique angle upon the person who interacts with them. Not only the 'percieved freedom' but also the length of time a player will invest into a game; long enough to create an emotional bond or attachment to the characters they 'play' with. For me Video Games will take another leap forward when we truly care about the characters in them. Perhaps my favourite examples of this to date have been:-

GTA IV: Grand Theft Auto, a game more usually associated to violence and scale. Yet I would argue this scale/backdrop provides a depth that we need to relate to one another. And the huge range of interactions with the game mean the virtual world does not appear to be linear, as such the player allows themselves the subconscious freedom that 'autonomous' players are always going about their business. This is illustrated beautifully when in GTA IV the player is asked to kill another character, but they are given a choice. This is crucial as in that moment the ground is unclear - we have been taken away from a computers Ones and Zeros. If the player does not shoot, or does, the game [ie your character journey/'life'] will continue. There is no 'play again' option, instead there is a sense of forging ones destiny. You 'choose' your Fate. Simply gaming genius imo.

Heavy Rain: For me, this was one of the most beautiful games ever made. An example, similar in execution to GTA's but forced upon the player early on. Your character has two sons and you are out with them in a simple shopping mall. packed with people... Your son has a balloon and, as must be one of the most frightening nightmares for parents, you suddenly find that your child is missing. Fighting through the crowd, chasing after a balloon in the distance, the camera sweeps as you are buffeted between the shoppers... I found this to be genuinely heart racing - 'careing' about the safety of this vulnerable child. Again the player is given a situation - find your son or not... the game will continue either way. Consequences upon your actions but ALSO upon your emotional attachment to your virtual character. Would you feel guilt playing a character whilst also feeling/knowing that you were responsible for loosing their/your son?

Pushing this even further and returning to where I began. Hideo Kojima's talk was about Metal Gear and a particular character that he introduced, which had traits from a real person he knew in his life. He made this character behave in ways more 'real' than any other (for example, calling when you were busy and talking about their problems, rather then fixating upon yours). Curiously this was the character above all others which people/players rejected... It seemed that being real was not what people were after in an escapists world... I really recommend listening and reflecting upon this fascinating observation.

Conclusion
So what does this mean? Is there a point when 'real' becomes 'too real'? Do we desire 2 dimensional character in our virtual 3 dimensional worlds? Do we seek clarity in quality and avoid ambiguity?

I believe the answer is that, just as in real life,  we cannot separate the character from their context/surroundings. How often have you asked a persons 'how are you' simply out of politeness. We choose when to be intimate and share ourselves with others. In this was so too will our virtual characters of the future need to adapt their virtual behaviour to their surroundings. Then tomorrow's game designers will really have something to play with. Imagine fighting alongside your team mate only to find out their motivations are different to yours, or having to live with the remorse shown in other characters over 100 hours of game play, due to your actions. Consequence and Context will imo define tomorrows game play.

And Finally... what of the individual character. When they look and move as real as us, How will we judge them? To project personality upon a virtual character takes one more special trait imo. Integrity. A belief that the character will react in ways that mimic all forms of human emotion. Based upon their surroundings, history, personality and more. To create a truly believable virtual character will need lots of work, far more than an a SubSurfaceScattering rendering of skin. But just as movement can be synthesised (eg Natural Motion endorphin engine) so to will these other qualities...

It it my belief that the Uncanny Valley is only the beginnings of what we will experience with our virtual characters. hmmm perhaps I should create my own Turing test... one for the 21st Century and our virtual worlds. ;-)

---phew, that's a blog and a half--- If you're still with me, please share your thoughts/comments in the box below. Can you name any cool virtual characters? :-)