Playtesting session at Microsoft

One of the advantages of living the the great Pacific Northwest is being so close to the Microsoft and Nintendo headquarters. I can't be more grateful for the opportunities I've had to get Microsoft-published games when they came out for $25. But today I experienced another convenient situation; a playtesting session for Kinect. After being called on the phone about a week ago and answering some basic questions I was asked to come in as a tester. Microsoft seemed to keep the process very secretive. They didn't announce the location I'd be playing at until yesterday and they only announced what I would be playing when I got there.  
 
When I arrived at the Microsoft building where I'd be testing I was escorted into a very sparse and ordinary room. The only items in the room were an Xbox 360 and Kinect add-on, a flatscreen TV, and a desk with laptop to answer  survey questions between 15 minute gaming sessions. Mounted on the TV was a camera which other employees used to moniter me.  I felt privileged to be trying something out at my own pace and independently that others waited hours for just recently at E3. The testing assistant got me situated and calibrated the Kinect camera. I saw a wire-frame, fuzzy, gray version of myself staring back at me on screen as I was told to hold my arms up. It was unique to see my joints have a 1:1 (albeit delayed) feedback with the avatar represented on-screen.  
 
The assistant told me I'd be playing three different sports in Kinect Sports. I started off with bowling. Unlike the other sports I played later, this one didn't play a short tutorial video before I got started. At first I tried walking forward about 4-5 steps as I would bowling a real ball before releasing my throw---however, I soon realized that the Kinect system prefers that you stand in the same spot about 8 feet away from the camera. The camera was able to track my motions left and right to line up a throw, but it wasn't always able to keep my shot lined up as I was swung my arm back and then forward for a throw. Kinect is good at detecting broader, slower movements but isn't very good at detecting movements that require quicker reaction times or movements which are more subtle. For example, as I was playing, I was only able to throw straight because the camera only rarely recognized the twisting of my wrist. In my game I was able to get one strike and a few spares, but I wasn't as satisfied with my level of control compared to the bowling in Wii Sports. I was still able to defeat my opponent, recorded my score on a sheet of paper, and moved on to answer some survey questions on the laptop.  
 
Next up was table tennis. This time I did get a tutorial video showing how to play and I had gotten more adjusted to how the camera expects me to play. The simple game required basic swatting movements to hit the ball back to the opponent. I could swat both backhand and forehand and apparently could also put spin on the ball and have "power smash" shots. This game was much harder than bowling. I didn't really feel like I had enough control on the ball to decide where I wanted it to hit the ping pong table. Also, about half of my backhand hits wouldn't register at the speeds that the ball was coming back with me or my arm would not be detected (presumably because it involved overlap with my torso). The opponent constantly kept getting power smashes on me which I moved quickly enough for but weren't being detected in enough time to hit back. I was unable to get any power shots regardless of how fast or hard I swatted my arm. With scores of 2-11 for my two matches, I continued onto pentathlon less enthused than I was during bowling.  
 
The last game was pentathlon which consisted of dash, long jump, javelin, discus, and hurdles. A new aspect of these events was the usage of the lower half of my body. To sprint I had to run in place, lifting my knees high, which looked quite ridiculous. I could imagine the employees watching me on the camera laughing on the other end. Dash, hurdles, and long jump all handled the running and jumping aspects quite well. I was able to get a hang of the timing and moving my legs up and down faster made my avatar run faster. I managed to actually break a sweat as I was running.  Discus  and the throw for javelin seemed pretty buggy. Oftentimes my fairly decent throws would hardly be half the distance of my competitors.  Other times my throw wouldn't register and the discus or javelin would clumsily drop onto the ground. On one of my discus throws, it didn't release as I swung my arm around, but as I put my arm back down along the side of my body it shot out at high speeds and broke the in-game world record and put me really far in first for that event. Overall I was pleased with the pentathlon game but I still had issues especially concerning the amount of control I had with my hands.  
 
After I finished the game, the assistant came in to escort me out. I received a gratuity reward (a game of my choice from a list of Microsoft published games) and walked out. Overall, I had a good experience with Kinect although there are still several wrinkles to iron out. The camera didn't quite capture me as quickly or as accurately as some of the games expected me to play. Kinect Sports played very similarly to Wii Sports despite the hands-free nature of it. While I felt more immersed in the game with Kinect, I didn't feel like I had as much control compared to Wii Sports. If Microsoft is going to sell Kinect for $150 later this Fall, they are going to have to fix several crucial issues. 

7 Comments
7 Comments Refresh
Posted by OneEyedNinja7

One of the advantages of living the the great Pacific Northwest is being so close to the Microsoft and Nintendo headquarters. I can't be more grateful for the opportunities I've had to get Microsoft-published games when they came out for $25. But today I experienced another convenient situation; a playtesting session for Kinect. After being called on the phone about a week ago and answering some basic questions I was asked to come in as a tester. Microsoft seemed to keep the process very secretive. They didn't announce the location I'd be playing at until yesterday and they only announced what I would be playing when I got there.  
 
When I arrived at the Microsoft building where I'd be testing I was escorted into a very sparse and ordinary room. The only items in the room were an Xbox 360 and Kinect add-on, a flatscreen TV, and a desk with laptop to answer  survey questions between 15 minute gaming sessions. Mounted on the TV was a camera which other employees used to moniter me.  I felt privileged to be trying something out at my own pace and independently that others waited hours for just recently at E3. The testing assistant got me situated and calibrated the Kinect camera. I saw a wire-frame, fuzzy, gray version of myself staring back at me on screen as I was told to hold my arms up. It was unique to see my joints have a 1:1 (albeit delayed) feedback with the avatar represented on-screen.  
 
The assistant told me I'd be playing three different sports in Kinect Sports. I started off with bowling. Unlike the other sports I played later, this one didn't play a short tutorial video before I got started. At first I tried walking forward about 4-5 steps as I would bowling a real ball before releasing my throw---however, I soon realized that the Kinect system prefers that you stand in the same spot about 8 feet away from the camera. The camera was able to track my motions left and right to line up a throw, but it wasn't always able to keep my shot lined up as I was swung my arm back and then forward for a throw. Kinect is good at detecting broader, slower movements but isn't very good at detecting movements that require quicker reaction times or movements which are more subtle. For example, as I was playing, I was only able to throw straight because the camera only rarely recognized the twisting of my wrist. In my game I was able to get one strike and a few spares, but I wasn't as satisfied with my level of control compared to the bowling in Wii Sports. I was still able to defeat my opponent, recorded my score on a sheet of paper, and moved on to answer some survey questions on the laptop.  
 
Next up was table tennis. This time I did get a tutorial video showing how to play and I had gotten more adjusted to how the camera expects me to play. The simple game required basic swatting movements to hit the ball back to the opponent. I could swat both backhand and forehand and apparently could also put spin on the ball and have "power smash" shots. This game was much harder than bowling. I didn't really feel like I had enough control on the ball to decide where I wanted it to hit the ping pong table. Also, about half of my backhand hits wouldn't register at the speeds that the ball was coming back with me or my arm would not be detected (presumably because it involved overlap with my torso). The opponent constantly kept getting power smashes on me which I moved quickly enough for but weren't being detected in enough time to hit back. I was unable to get any power shots regardless of how fast or hard I swatted my arm. With scores of 2-11 for my two matches, I continued onto pentathlon less enthused than I was during bowling.  
 
The last game was pentathlon which consisted of dash, long jump, javelin, discus, and hurdles. A new aspect of these events was the usage of the lower half of my body. To sprint I had to run in place, lifting my knees high, which looked quite ridiculous. I could imagine the employees watching me on the camera laughing on the other end. Dash, hurdles, and long jump all handled the running and jumping aspects quite well. I was able to get a hang of the timing and moving my legs up and down faster made my avatar run faster. I managed to actually break a sweat as I was running.  Discus  and the throw for javelin seemed pretty buggy. Oftentimes my fairly decent throws would hardly be half the distance of my competitors.  Other times my throw wouldn't register and the discus or javelin would clumsily drop onto the ground. On one of my discus throws, it didn't release as I swung my arm around, but as I put my arm back down along the side of my body it shot out at high speeds and broke the in-game world record and put me really far in first for that event. Overall I was pleased with the pentathlon game but I still had issues especially concerning the amount of control I had with my hands.  
 
After I finished the game, the assistant came in to escort me out. I received a gratuity reward (a game of my choice from a list of Microsoft published games) and walked out. Overall, I had a good experience with Kinect although there are still several wrinkles to iron out. The camera didn't quite capture me as quickly or as accurately as some of the games expected me to play. Kinect Sports played very similarly to Wii Sports despite the hands-free nature of it. While I felt more immersed in the game with Kinect, I didn't feel like I had as much control compared to Wii Sports. If Microsoft is going to sell Kinect for $150 later this Fall, they are going to have to fix several crucial issues. 

Posted by ajamafalous

Sounds about how I'd expect.
 
Cool to have an opportunity to playtest something that (presumably) people are looking forward to that isn't out yet, though. Have you done any other playtesting in the past?

Edited by SolemnOaf

 
If you, or any members of the press ever get the chance, I would like one question answered about Kinect. 
 
Where was the resistance to this disaster before it occured? 
 
That's all I really want to know about Kinect at this point.  So first I'll probably have to wait until the potential disaster becomes actual disaster, then maybe someone in a position to ask this question will for me.  Was it some Executive who had too much power and called all the shots and pushed this thing through come hell or high water?  Were they mass-hypnotized by gypsies?  Have aliens assimilated into Microsoft's upper management positions and are trying to gain domination over our planet through nefarious motion-controlled schemes?  That's the only thing that interests me about Kinect now.   
The Aftermath.

Posted by OneEyedNinja7
@ajamafalous said:
" Sounds about how I'd expect.  Cool to have an opportunity to playtest something that (presumably) people are looking forward to that isn't out yet, though. Have you done any other playtesting in the past? "
I've been called by Microsoft and market research groups before in the past for gaming-related things, but I didn't qualify after they asked me questions.
Posted by SSully

The way you describe it actually sounds kind of bad. Responsiveness is one of the most important things in motion controls and it sounds very unresponsive from the way you describe it. 

Edited by Sjupp
@OneEyedNinja7: Were there only like cheap old games that you got to choose from?
Posted by OneEyedNinja7
@Sjupp said:
" @OneEyedNinja7: Where there only like cheap old games that you got to choose from? "
They had a lot of older games like Viva Pinata 2, Banjo Kazooie, Fable II, Mass Effect, and Gears of War 1/2 but they also had newer games like Alan Wake, ODST, and Halo Wars.