How to create a Virtual Character.

What makes a Virtual character 'Real'?

'Hideo Kojima', the Designer behind the 'Metal Gear Solid' games has sparked my interest this week and as it's been a while since my last blog I thought I'd turn this into a larger look at the challenges behind Virtual Characters.

Which Avatars have sparked your imagination - from Film to Science to Games - Use the comments section to let me know examples you can think of and why you think they stand out. :-)

Graphics
Often when asked what makes a virtual character Look real people talk of about graphics or animations. After all it's difficult to consider a character real when they're made of a few hundred polygons with boxed shaped heads. But now characters are can be so detailed that the Uncanny Valley is also mentioned. This  - where we might have a character look 'realistic' yet something doesn't feel right the 'illusion of life' can suddenly drop away (often leaving a very surreal mixed set of emotions with the viewer). Picture for example a character that could look real in still motion but suddenly feels very false when animated. Even a crooked smile can destroy the best of models.

Personally I feel Benjamin Button [VFX by Digital Domain : Directed by David Fincher in 2008] is still one of the best examples of a believable character. If you've not seen it I highly recommend the film. Here's a sneak peak behind the scenes into its effects.

But films can afford massive computations to create each frame - Real-time graphics have to be generated in around 1/30 of a second! Hence different techniques are used. Here's another of my favourites from the same year as Ben' Button [2008] showing Nvidia using multiple texture composites to create a real-time head [a method similar to modern games which use deferred rendering with multiple passes of screen renders]

Computer Games have also advanced with their animation - from marker based motion capture, to phosphor painted or videoed actor faces recording subtle skin movements, ie performance capture.

Here's a good example from 'Janimation' of human facial expression/animation being retargeted onto a CG character using marker based motion capture im 2011. Or take a look at the cutting edge performance capture tech' used behind the game L.A. Noire in 2010


Intelligence
Hold on a minute though. There is a key difference between film and other forms of avatars. It's one thing to watch a character that has been skillfully crafted and presented to us by a whole gaggle ['technical term' ;-)] of artists, but what happens if we can interact with it. The best looking virtual character will seems pretty dull if it responds to us like a 1970's robot.

One way to see how well a computer can mimic human behaviour is the Turing test. Essentially this is measured not by the avatars behaviour but our response to it. ie If 'we' can't tell that a computer is controlling it then it 'must' be behaving like a real person... Hmmm. Personally I don't like this test and find it quite flawed. Nevertheless it has been used many times.

It used to be said that a computer could never beat a Grandmaster at chess. This was because although it was relatively easy to program it to follow the rules of chess, to play at such a high level requires a deeper understanding of the human psyche. So it was quite a shock to many when IBM's
BigBlue defeated Gary Kasparov in May 1997.

I find it interesting to note that this 'test' was measured over a 'game' of chess... So perhaps it is not surprising to find that on the 27th September 2012 a computer games "bot" passed the Turing Test, as it hunted down and killed opponents in a video game was judged to have displayed behaviour that was indistinguishable from a human. Something many had said would be 'impossible'...

If we take this notion a little further we can see computer 'intelligence' [mimicking behaviour] spreading wider afield. For example this summer Google set a new landmark in the field of artificial intelligence with software that learnt how to recognise cats, people, and other things simply by watching YouTube videos . This takes us into the realms of 'learning behaviour' and neural networks... something I'll come back to another day I'm sure. What will it mean when search engines adopt 'intuitive behviour'?

Games are often the first adopters to new technology and being able to 'deliver' realistic and believable character to our living rooms has been a dream goal to many a developer. How though will this effect us - We may accept Tamagotchie's needing feeding, but how would we respond to a computer not wanting to play because it feels a little down..? ;-)
Microsoft have skirted around the concept of realistic avatars several times. In particular
'Project Natal' [June 2009] gave us a glimpse of what may be to come.

Character & Personality
The true beauty of this technology for me is that we are at last able to put to test the subject that philosophers have debated for 1000's of years. What makes us human and how do we define 'the person'. Modern culture refers to us as 'Mind, Body and Spirit'. Perhaps this is why Virtual Avatars are following the same path. ie Body = Graphics, Mind = Artificial Intelligence, and Spirit... hmmm this is a more tricky one.

If we are make a virtual character believable it not only must be visually detailed and 'perform' as if it has an 'awareness', it crucially needs one more thing. 'Spirit' - and how can we recreate that?
Well perhaps this can be represented by the traits which we associate with a persons character of personality. Do we care about the character enough to value their 'life'. This, imo, is the real challenge behind making a Virtual Character believable - sure we need the graphics, animations and A.I. (artificial intelligence), but can we go further? I say yes we can but it may not be quite the way we might imagine it to be. Just as with the Turing test, we may only need to mimic personality in order for us to believe in it. After all how many people name their cars and anthropomorphise them?

Video games have a unique angle upon the person who interacts with them. Not only the 'percieved freedom' but also the length of time a player will invest into a game; long enough to create an emotional bond or attachment to the characters they 'play' with. For me Video Games will take another leap forward when we truly care about the characters in them. Perhaps my favourite examples of this to date have been:-

GTA IV: Grand Theft Auto, a game more usually associated to violence and scale. Yet I would argue this scale/backdrop provides a depth that we need to relate to one another. And the huge range of interactions with the game mean the virtual world does not appear to be linear, as such the player allows themselves the subconscious freedom that 'autonomous' players are always going about their business. This is illustrated beautifully when in GTA IV the player is asked to kill another character, but they are given a choice. This is crucial as in that moment the ground is unclear - we have been taken away from a computers Ones and Zeros. If the player does not shoot, or does, the game [ie your character journey/'life'] will continue. There is no 'play again' option, instead there is a sense of forging ones destiny. You 'choose' your Fate. Simply gaming genius imo.

Heavy Rain: For me, this was one of the most beautiful games ever made. An example, similar in execution to GTA's but forced upon the player early on. Your character has two sons and you are out with them in a simple shopping mall. packed with people... Your son has a balloon and, as must be one of the most frightening nightmares for parents, you suddenly find that your child is missing. Fighting through the crowd, chasing after a balloon in the distance, the camera sweeps as you are buffeted between the shoppers... I found this to be genuinely heart racing - 'careing' about the safety of this vulnerable child. Again the player is given a situation - find your son or not... the game will continue either way. Consequences upon your actions but ALSO upon your emotional attachment to your virtual character. Would you feel guilt playing a character whilst also feeling/knowing that you were responsible for loosing their/your son?

Pushing this even further and returning to where I began. Hideo Kojima's talk was about Metal Gear and a particular character that he introduced, which had traits from a real person he knew in his life. He made this character behave in ways more 'real' than any other (for example, calling when you were busy and talking about their problems, rather then fixating upon yours). Curiously this was the character above all others which people/players rejected... It seemed that being real was not what people were after in an escapists world... I really recommend listening and reflecting upon this fascinating observation.

Conclusion
So what does this mean? Is there a point when 'real' becomes 'too real'? Do we desire 2 dimensional character in our virtual 3 dimensional worlds? Do we seek clarity in quality and avoid ambiguity?

I believe the answer is that, just as in real life,  we cannot separate the character from their context/surroundings. How often have you asked a persons 'how are you' simply out of politeness. We choose when to be intimate and share ourselves with others. In this was so too will our virtual characters of the future need to adapt their virtual behaviour to their surroundings. Then tomorrow's game designers will really have something to play with. Imagine fighting alongside your team mate only to find out their motivations are different to yours, or having to live with the remorse shown in other characters over 100 hours of game play, due to your actions. Consequence and Context will imo define tomorrows game play.

And Finally... what of the individual character. When they look and move as real as us, How will we judge them? To project personality upon a virtual character takes one more special trait imo. Integrity. A belief that the character will react in ways that mimic all forms of human emotion. Based upon their surroundings, history, personality and more. To create a truly believable virtual character will need lots of work, far more than an a SubSurfaceScattering rendering of skin. But just as movement can be synthesised (eg Natural Motion endorphin engine) so to will these other qualities...

It it my belief that the Uncanny Valley is only the beginnings of what we will experience with our virtual characters. hmmm perhaps I should create my own Turing test... one for the 21st Century and our virtual worlds. ;-)

---phew, that's a blog and a half--- If you're still with me, please share your thoughts/comments in the box below. Can you name any cool virtual characters? :-)

Kinect on your wrist!

Microsoft have just made a mini Kinect that fits on your wrist to scan and record your hand/finger positions Pretty nifty if you ask me... To see their tech in action take a look at this video on the  'Microsofresearch' youtube channel

For me, Kinect is very cool tech'. Not only did it take a whole motion capture studio and placed it inside a box, but it's also cheap enough to be bundled with a games console... Sure, I know some will argue that it's not that accurate, but you know what - there are always those wanting to spread doubt.

This new device is why I find Kinect cool - not because of what it can do, but because of what it will do. Remember Kinect is still the first version (2nd if you count the PC version). There are other companies making similar devices - in fact, this tech is not new at all... but it's packaging and delivery to the mass market is a game changer on many levels.

In a couple of years it's likely that Kinect 2 will be out with improved accuracy, finger and face recognition. From this there will be all sorts of applications (and some social questions too - not least Health and Safety) - A revolution of sensor/positioning/gesture/scanning tech is about to happen imo and even more importantly this revolution will combine these multi-disciplines in to one single product.

From mapping our world through LIDER (Light Detection And Ranging) as I mentioned in my 'Mapping the Future blogpost on the 25th September 2012' to Disney's new Augmented Reality Cake patent (!) This can of worm's has only just been pierced. Some-one hand me a can opener! ;-)

3D Printing - Who pays the postage?

For those of you who don't know '3D printing' is a magic process that makes something from nothing.

Ok ok, maybe there's a little more to it than that... Most stereolithography (to give 3D printing its proper name) builds a real tangible object, by solidifying a gel one thin layer upon another. If an object (real or virtual) can be sliced up, then 3D printing can put it back together... But, don't limit your thinking - check my links below and you'll see the sky is much higher.

I've noticed quite a surge of news about the topic recently so thought I'd put together a few news articles too. Before I list them though - I can remember a 'think tank' (hmm - I wonder if we could map/print thoughts one day?) saying that in 40 years printing objects will be as common as sending emails... well I don't know about that, but I do think if we said 40 years ago that we would communicate through emails and text we'd be seen as crazy. So you never know...

03-10-12 Printing Guns... Warning: Red Alert.

http://www.bbc.co.uk/news/technology-19813382

03-10-12 Printing toys and LEDs - Disney Attacks!

http://www.bbc.co.uk/news/technology-19818815

20-09-12 New York opens 3D printing store
http://www.bbc.co.uk/news/technology-19665625
 

13-08-12 Don't want to wait for you model to be delivered - build your own printer!

http://www.digitalartsonline.co.uk/news/printing/build-your-own-3d-printer-with-pwdr/

02-08-12 Virtual Characters come alive - Print your own virtual avatar!

http://www.bbc.co.uk/news/technology-19085980

02-07-12 Feeling Low? Print with Sugar to make a new Liver (Onions still need to be bought)

http://www.bbc.co.uk/news/technology-18677627

06-04-12 Feeling Hungry - Print with Chocolate!
http://www.bbc.co.uk/news/technology-17623424 

So, what will this new printing age mean to us? Will we really be as keen at 3D Printing as we are with emailing? Perhaps cost will be a factor, or maybe acceptance? I find it curious that last week the BBC ran two stories on the subject and chose to use toys and guns? One thing is for sure. The costs are tumbling and there are even machines now that can print themselves - maybe the phrase 'parts and labour' will be replaced with 'Printing and Labour' - although I suspect the use of these machines will be more related to the materials they use rather than cost (what good is a washing machine part made of cheese - other than a tasty sandwich).

Imagine a car showroom where you design your dream car and as you sip a coffee it gets printed next door... 'sounds like science-fiction? We'll see...