Emotionally Adaptive Computer Games - an Intro to my new Blog

 

Hi,

I’m trying to develop a set of guidelines, a starting point for debate if you like, regarding how to develop emotionally adaptive computer games for my PhD. The concept behind such games is very simple; keep the player more entertained, for longer. Should she get bored whilst playing, introduce some new and fun elements to the game to heighten her interest. If she is getting too stressed to enjoy playing the game, make the game calm down a little, allowing her to relax and enjoy the experience. As part of my PhD project I have already developed such a game. It uses physiology in conjunction with gaming context to determine the player’s changing emotional state, and adapts accordingly. The game I’ve developed is called TEDDI (Transient Emotion Detecting Designed Interface), and is a Minesweeper style game with some role playing game elements introduced. Think PuzzleQuest, but with Minesweeper, and two boards rather than just the one.

The reason I’m starting this blog is so that all you gamers out there can help me develop my set of guidelines. I’m really looking for your input, any thoughts you may have regarding the big picture of this, or more specifically on the topics of a particular post. Any thoughts/ideas that end up in my thesis will be referenced appropriately, I’m not out to steal your ideas! However, getting expert gamers thoughts will increase the credibility of my work, so I really need you guys!

This post is really only to introduce my blog, more specifics will come later. I’ll probably update this a few times a week from now on. If you have any questions, put them out there, and I’ll try my best to answer. If I don’t have the answer, perhaps someone else does, or perhaps it’s a point we should discuss!

See you next time!

Irene

6 Comments
7 Comments
Posted by Phewsie

 

Hi,

I’m trying to develop a set of guidelines, a starting point for debate if you like, regarding how to develop emotionally adaptive computer games for my PhD. The concept behind such games is very simple; keep the player more entertained, for longer. Should she get bored whilst playing, introduce some new and fun elements to the game to heighten her interest. If she is getting too stressed to enjoy playing the game, make the game calm down a little, allowing her to relax and enjoy the experience. As part of my PhD project I have already developed such a game. It uses physiology in conjunction with gaming context to determine the player’s changing emotional state, and adapts accordingly. The game I’ve developed is called TEDDI (Transient Emotion Detecting Designed Interface), and is a Minesweeper style game with some role playing game elements introduced. Think PuzzleQuest, but with Minesweeper, and two boards rather than just the one.

The reason I’m starting this blog is so that all you gamers out there can help me develop my set of guidelines. I’m really looking for your input, any thoughts you may have regarding the big picture of this, or more specifically on the topics of a particular post. Any thoughts/ideas that end up in my thesis will be referenced appropriately, I’m not out to steal your ideas! However, getting expert gamers thoughts will increase the credibility of my work, so I really need you guys!

This post is really only to introduce my blog, more specifics will come later. I’ll probably update this a few times a week from now on. If you have any questions, put them out there, and I’ll try my best to answer. If I don’t have the answer, perhaps someone else does, or perhaps it’s a point we should discuss!

See you next time!

Irene

Posted by ahoodedfigure

Sounds like something I've been thinking about, but I guess I didn't know that emotional states were easier to read.  Have you thought about the variance from person to person?  Like, how one person's physiological reaction to an emotion may read differently?  Would there be some sort of calibration involved?  What about the kind of stress that usually comes before a big payoff?  Will there be a way to differentiate between stress which is avertive and stress which brings about a higher emotive state when finally successful?
 
Would you allow players to toggle whether or not certain states would be read, or would the player not have such an option? The more deterministic it is, the more it becomes something new, as games are often about giving the option of controlling an environment, or at least the timing with which a thing may happen, more to the player.
 
What about when the game is interrupted?  Should the player be allowed to take a break without distorting the physical readings?  Would a paused game still read a player's responses?  Might they, say, be able to calm down and solve a problem or would there be timing involved that might warrant a pause function in the first place?
 
That's all I can think of off the top of my head.  Very interesting!

Edited by Phewsie

Hi again,
 
I guess I forgot that most of you out there don't have a clue about what I've been doing or how my system works... Sorry about that! :P
 
Re how physiology varies from person to person: My system is user independent, which means anyone can use it without training the system first (ish, I train the system for 1 minute pr person, that's it..). Emotions are far from easy to read, and the emotional states my system responds to are not labelled as 'happy', 'bored', 'stressed' etc, but rather are measured as a two dimensional state. The two dimensions are arousal (not sexual, but rather how much 'strength' an emotion is felt with) and valence, which is a happiness scale. To explain with examples, a person who is sad would have low valence (unhappy) and low arousal. Someone who was furious would have low valence, and high arousal.
 My system then determines arousal from physiology, and valence from gaming context. So the system uses a threshold to determine a significant change in physiology and then messages the game which checks gaming context and then determines how to adapt.
Still following? 
I can re-explain any of this is this crash course is too dense!
The threshold is determined for each person separately. That's what that minute of training does at the start, determines the initial threshold. The threshold then changes throughout the game session, and is dependent on the player's physiology.
 
In terms of stress that leads to better pay-off the game context should be able to determine whether it is 'good' stress or 'bad' stress. Bad stress I define as when the player no longer feels like she can cope with what's going on. Up until that point you have good stress. However, this specific point is very interesting, and I believe a lot of different developers would have their own take on this. Also, it probably depends on the genre, and game style, not to mention the specific story. My system is very simplistic in the way it deals with this sort of thing, and so it does not plan ahead or know what's about to come. If the game has been designed to arouse a certain level of stress only to majestically resolve it in some climax, then that system should probably allow the player to be stressed through that phase of the game...
 
Whether the player would have the option to turn on/off certain state recognition would depend on the system design as well as the developers and what they want to achieve. With my system this is not possible, as it only detects significant changes in physiology. I suppose it would have been possible to give the player some options on which adaptive rules could be triggered, e.g. don't change my avatar's speed, but the system would still detect the changes that would normally trigger this change. A different system design could allow for it though, a system using neural networks for instance.
 
Wow, the next question is a REALLY interesting one! What happens during pause? Well, obviously they have to be allowed to pause the game. Whether the system keeps reading whilst during pause I think would be entirely up to the developers. They could not however assume that the player stays connected to the sensors during the pause, and so could not use these readings once the game starts up again. The system would also have to allow for drastic changes in physiology after a pause, which may or may not be related to a new emotional state. It could be something so simple as the player washed their hands in hot/cold water, or that they had to pop outside for something, or any number of things really. On the other hand perhaps the pause came about because someone came in and started yelling at the player. After an argument the physiology would be quite different, as is the emotional state. So, how should a game deal with this? I have to admit this is something I have never thought about before, and I don't think I have a good answer...not yet anyway! I suspect it would be somewhat determined by the style of game, if there is no storyline it may not matter much. My system would simply calculate a new 'initial' threshold and take it from there, sort of do a re-start of the recogniser. But for most games I think it would matter and my solution would be far from adequate. Do you have any ideas?
 
And now for my next blog post.... :)

Posted by JJWeatherman

This sounds really ambitious. I'm too tired right now to contribute anything of use but I'll follow your blog.    :P

Posted by ahoodedfigure
@Phewsie: I'm following, but I'm not sure how valence is derived from gaming context.  There's a cynic in all of us who may not feel happy or sad at the proper moments, because unlike films, where it's more difficult to build up a barrier between us and what we see, games have an extra level of abstraction, and a level of interactivity and sabotage, that lets us thwart the expected emotional outcome.  It would help to know exactly how valence is defined.  Maybe you could refer me to a good definition off-site so you don't have to waste your time explaining it.  If valence is just a subjective, game-specific situation, like what you do to determine ranges in "lie" detectors, then I think I understand.
 
Are there studies that, say, have one's reaction to a movie that uses the same data collection method?  Movies are a lot easier to control because the experience is virtually the same for all viewers.  Games allow some sort of player feedback, so you have that feedback along with the physiological feedback running concurrently.  Do you have a playback feature for each player's performance, so that you can match your readings to what they were doing at the time?
 
Gamers are also notorious snobs, so often their reaction to a certain situation will be relative to their liking a certain style of game.  Many refuse to respond in any meaningful way if the graphics are considered to be out of date, or the gameplay style is irritating to them.  At least with movies, it plays no matter how you interact with it, and can measure the reaction that way.  With a game, you have at least three things going on, not just valence and physiology.  You also have player behavior.  That could actually be broken up into following the game's path, fiddling with options (if possible), trying to break the game through combinatoric testing, trying alternate paths that the game is expressly telling you not to do or not mentioning, loitering... If the game is straightforward enough, and there's some sort of implicit tester-testee relationship that has developed, maybe the rebellious behavior will be limited, although I'm not sure you'll get a fully accurate reading of just how the potential complexities of player-game interaction can be.  
 
As far as the reaction of the game to the pause, I think it's complex.  If we're talking about a theoretical game that actually will, some day, react to a player's emotional state, I can think of a few solutions.  I think it's best to define what's being discussed here, as there are two ways to take your hypothetical example of a person stepping out.  
 
One is to say that this game is a future game that is in the home, in an uncontrolled environment, which reads physiological data and overlays it with contextual data to form an emotional matrix for the given player.  Should a pause be allowed in such a situation, and for home use it seems necessary lest it be quite inconvenient, limiting repeat performance, you would HAVE to re-initialize the system.  Any number of different situations could occur during the pause, and the only way for the system to understand this is to reset it.  It may be a pain to recalibrate every time, but this minute-long calibration is the only way to reaffirm the relationship between player and game.  You might be able to gather data about gamer habits and behavior over time, and adjust for that (player A often plays in short fits, and usually begins playing the system with a lot higher physical activity readings than how they read later on during the course of play, while player B plays nonstop for several hours), but that would be a different relationship with the data and would still have to be based on your system for reading particular game sessions.
 
The problem also comes to mind that, beyond pausing, a player's own physical state may change over the course of extended play.  Even if you don't pause, there can be situations where one's state changes through an external stimulus, say food or conversation.  It would help to know just what the specific demands are on a player at any given moment.
 
Should this game be in a controlled environment, a laboratory say, where distractions are are minimalized, this changes things.  I would say in a laboratory setting, the game simply shouldn't have a pause at all, to allow for a more accurate reading.  Then again, if the player needs to pee, but is too embarrassed to say, it might change the readings a bit! :)
Posted by Phewsie

@ahoodedfigure: 
The wiki definition is a pretty good one actually. It doesn't contain heeps of information, but I don't want to drown you on this particular topic.
http://en.wikipedia.org/wiki/Valence_(psychology)

As a gamer myself, I understand what you're saying about snobbery, though it's not the word I'd choose to use. ;-)
People have different preferences and as such will be affected by situations within the game differently. I suppose I've always thought of this process in a different direction than what you're doing. Rather than the game determining how you should be reacting/feeling, it simply reads what changes occur and adapts to that. Of course designers could choose to attempt to induce a certain emotion at certain times, and they may or may not succeed. But as a general rule the system should attempt to adapt to you, not the other way around.
With regards to valence this means that, for my game in specific, if you are about to win and arousal is going up, the system will assume that that is a positive reaction. With machine learning it could learn differently, should the player insist on feeling stressed at this stage. And if the player wishes for the system to never work properly they could then proceed to randomise they're reactions, providing they are in that good control of they're bodily functions. This is entirely possible, just like it's possible to beat a lie detector, but it's certainly not easy without practise.

Unfortunately I don't know of any Movie experiments that are similar to my experiments, I have semi been on the look out for a couple of years now. I thought about using the drama curve of films in my project, but as my game lacks a storyline, and without one it's tricky to achieve, I shelfed this idea. Still think that would have been really interesting though!

I agree that after a pause the system would have to reset somehow, though I think in a lot of games this could be achieved behind the scenes, whilst play continued. It could mean that the game would delay the appearance of an event which might trigger an emotional response until after recalibration, or something similar. If this was not an option, perhaps the game could get away with not being adaptive during this phase. But if there was no other option, I suppose the boring reset would have to be utilised.
The game I'm talking about will always be one to be used in uncontrolled environments. Controlled environments is a struggle in the sense that people are constantly aware of their surroundings, not being as comfortable and relaxed as they normally would be, leading to considerably less activity in physiology during game play. Having said this, most of my experiments were run in a laboratory setting. :(    

Posted by RagingLion

Ah.  I should have read this blog post first before posting on your latest entry.  Anyway, at least I have an idea of the way in which you're looking at things now.  I didn't realise you would actually be measuring physiological responses - that changes a lot.  Anyway, I'll follow your blog with interest and say if anything else springs to mind.