Kinect Died in the Uncanny Valley

@rokkaku: MS, as we know through the leaked 56 page document a few years ago, is working on AR glasses. Which have, even according to someone like John Carmack, greater commercial potential than VR goggles. MS will be fine in that respect.

I think that regardless of the promise of Kinect and whether it managed to live up to that promise (I don't think it did at all), that this was also caused by Microsoft's hubris. Besides touting stuff like Skype calls and voice commands, and how it was involved in their TV plans, I don't think there was ever some kind of unified message saying here's why people who play games should care about Kinect. It seemed like Microsoft mostly just showed up and said, "hey we're the Xbox people, we've been on top for the last 8 years, and we want one of these in every home so you're gonna pay for it, because you really have no other choice" Which seems like the worst way to get people to adopt something.

Whether this is the end of the Kinect, who knows? When they announced the new SKU I saw someone say, "there's nothing to worry about, if they want Kinect games they can just buy the one with kinect," wait you mean the one that they're having trouble selling already? I think there's a very real chance that the Kinect will suffer from Wii U syndrome when the new SKU comes out. Also known as, "no one is buying this thing, and no developers will make a game for it because no one is buying it."

Actually, to be honest, this probably would have been a horrible business decision, but you know what? If Microsoft cared so much about getting a Kinect in every home, then they should have covered the cost of it themselves. No consumer should have to pay an extra hundred dollars for a product that already failed once, and is mostly being sold on the promise of seeming futuristic.

Sure there's a chance that the Kinect add on by itself might succeed, but I just don't see Microsoft getting behind it, at least not for games, and especially not like they did back in 2010, with weirdo Cirque Du Soleil elephants and tons of games.

yep, but you forgot one excellent example...

And the star segment of the quicklook...

No, what he means is that the analogue between actions in the real world and actions in the game need to be better designed. The mistake that way too many developers make with motion control is they try to make the player pantomime the action their onscreen character is supposed to do, which creates an immediate disconnect the second those two things fail to line up correctly.

The reason Wii Sports works as well as it does has nothing to do with fidelity. If anything, Wii Sports benefits from having a much lower fidelity than most motion controlled games, it also does a good job of hiding the fact that it's just barely reading your movements and handling most of the action without telling you. Reduced fidelity and that slight layer of abstraction makes for a game that's considerably more playable, even if the realism is broken the minute you start thinking too hard about what you're doing.

Basically, he's saying that a lot of the problems people run into with Kinect could be solved by having it take in less complex inputs overall, and designing around not just the hardware limitations but also the real life limitations that the player is going to have to deal with in trying to use the thing in the first place. (keep in mind that this is coming from a man who designed a game where you hold a controller still, and other people try to make you shake your controller while also holding theirs still)

Unless they can make a Kinect that perfectly reads your actions 1-1 exactly 100% of the time, then making games that require precise pantomime on the part of the player are never going to work right.

Imagine a world where steel battalion had worked as advertised. Everything they were having it do the kinect should have been able to handle and it used the controller to for the mech controls that kinect could not handle. I wish I knew the full story on how they allowed that game to ship that broken. Even as a gimmick kinect plus controller had potential. No one ever tried it again.

It never hit me or my wallet, as 3D, Wii waste, or any other gimmick that has been tossed down the line over the last 10 years. All i want are good games and gimmicks are for kids. It also wont die as the wii balance board showed, it will be wrapped with a nice bow every year with a new work out video.

While i highly enjoyed Vinny and Jeff in their QL of the new Fitness sports, its the Legend RD and anytime he walked in front of a Kinect. Nothing can bring a smile to my face then or now as easy as those vids. Just a few below.

One of Kinect's most promising moments on Xbox 360 was Double Fine's Happy Action Theater. You know what Happy Action Theater does? It doesn't give a shit about Kinect's inability to properly track you. Instead, the designers incorporated the fuzzy nature of the technology into the aesthetic, and encouraged players to be subversive through design. Happy Action Theater relishes and indulges in Kinect's quirks.

It should have signaled a new way forward with designing Kinect games on Xbox One. Embrace what the device is, rather than pretending it's something else.

I'm not sure if you fully considered your point here Patrick. Happy Action Theater worked because it wasn't a "game", in the sense that it didn't have specific goals. It was a toy, which is absolutely fine, but there was no success or failure element to it. So when the detection screws up, who cares, because it's just a fun toy.

As soon as you introduce an end goal, which is intrinsic to most games, and a player feels impeded from accomplishing that goal because of the fidelity of the detection and not their own ability, that's where a Kinect game breaks down.

So by "embrace what the device is", you're saying embrace it being a control method so imprecise that you can't actually build entertaining games with any competitive aspect or end goal around it. So that leaves us with what? Basically "activity" collections, like Happy Action Theater or it means combining the Kinect with more traditional input methods in a manner that makes it largely ancillary to the gameplay. Exactly how many similar titles like that do you think the market would support? That's not a future for the device either.

Those tweets from Doug Wilson are spot on.

Erm... i think this is a bit generous viewpoint of Kinect's design, true capabilities, and the games that were actually developed for it.

There are a couple things it can do okay, but the potential of those capabilities is smaller than I think you are suggesting. No matter what sort of game you are trying to make, you will eventually run in to the control fidelity problem which severely limits what you can do with your game. It's the lack thereof that ultimately sunk the Kinect for me. I never enjoy games with loosey goosey controls, no matter the input method. Just because there are certain types of games that can fudge shit controls behind forgiving input window does not excuse the shit control in the first place.

I agree entirely. I will say that Kinect itself is an interesting technology with lots of potential - but not for anything particularly mass-market, and especially not remotely as a gaming device, for many (simple and obvious) reasons. Trying to sell it as a gaming device was always a non-starter.

Why has JS Joust guy not made a great must-have game for it, if he likes it so much and is bitter about people celebrating its demise? The reason is that it's a terrible gaming control method, that simply never worked properly.

And 'physical play'? Just play a sport. Far more fun and satisfying and exhilirating and rewarding than any weird indie game designed around a bad control system can ever be.

In unbundling the Kinect from the XBox One, Microsoft has acted in a way that will make any future vocal support for the device ring hollow. That undoubtedly changes the future prospects of Kinect 2.0 and kills any shot it had to be a game changer for the foreseeable future.

If Microsoft had stuck to its guns, or if the XB1 had dominated console sales, the Kinect most likely would have continued to receive the development and financial resources that it needed to reach its potential. I dont see how that happens now and its a shame.

Ill admit. Microsoft turned me off with their tough talk and multiple reversals last year. It led to me choosing a PS4 at launch and to be honest, i still have no plans to get an XBone......BUT, if they had showed the guts to stick with their vision for Kinect, and it came into its own, down the road, it may have been the reason i eventually broke down and picked up their console. Instead, Microsoft just reinforced all my biggest peeves against them. They once again spoke with the conviction of a preacher and acted with the impotence of a politician. They look dishonest or at the least, unreliable. You just cannot count on anything the company claims.

So, i am disappointed to see the Kinect be relegated to the background. It was the only part of the Xbox One that made it stand apart.....i guess we live in a world where its easier to fall in line, thats too bad.

No Patrick ... motion gaming has only one place and that is the grave.

Good read. I haven't thought much about Kinect, but now that you made me think about it - yes, I was always fascinated by the tech demo stuff the Kinect's enthusiast crowd came up with on PC. Indeed Microsoft has failed to transport that fascination with Kinect to their Xbox consumerbase.

You encouraged me to dream about a world where-in Microsoft got the Kinect right, and I agree, that'd have been a dream worth living. Developers jamming their hearts out, and players interacting with their creations, with little interference by the big bad Xbox Man. Now I'm sad. Broken dreams. Bummer.

I can't fathom how this isn't Kinect's fault. The technology just straight-up isn't good enough for anything that interests the market that's still around and willing to buy those games. Natal was introduced in 2009 and we've been through a second iteration of this tech now, and it still isn't satisfactory.

People can hold up the voice commands as the beacon of "the future" all you like, but the fact remains that you don't really need a giant shitty camera to accomplish that. The PS4 allows you to use voice commands by plugging any old microphone into the audio jack on the controller, and presto, you have "the future." Nothing fancy required. Presumably Microsoft could allow the same flexibility to their commands for those who wished to use them.

And seeing developers on twitter subtly lash out at people for being happy they no longer have to spend a lot more money on an inherently flawed device they never wanted in the first place? I just think that's childish. It shouldn't be my responsibility to prop up that market just so people can keep trying to release games year after year that turn out to still not be that interesting. The fact that consumers have basically been asked to beta test these devices for several years, over a second iteration, at our expense, and we still have so little to show for it, is bananas. Average consumers have done way, way more for that device than any developer has thusfar.

The thing Dance Central did right was to not show any representation of you on the screen which is the thing that leads to awkward glitches and twitchy avatars (i.e. the thing that makes Kinect quick look entertaining.)

Their characters on screen do the moves correctly, looking all cool and hip with only the high-lighting limbs signifying that you're actually looking like a buffoon. If When Kinect completely breaks during a song, you might not even notice it, you might just think "oh well, I guess my right arm wasn't quite at the angle it was supposed to."

Finally, I'd like to say, the reason why DDR is fun, is because it's buttons that fucking work when you hit them. We've given them years to prove Kinect can work reliably, yet the recent Kinect Sports QL and Patricks own voice navigation video (btw, I seen no reason why voice recognition is in any way tied to Kinect, it's a technology that has existed for years) show me it's still not at a point where I would trust it to do what I want it to.

I agree kinect games could be fun, but microsoft cant even get the voice to work in their own menus. so how can they expect developers to get it to work? maybe next time or skip the next generation and comeback as 3in3ct and then you have a cool logo and you also get voice command to work, at least as well as my internet browser running google chrome.

Kinect is good technology, but I just don't need it.

Buttons work fine and Kinect doesn't offer enough improvement for me to want to do more than occasional voice commands (if even that).

@grantheaslip: It's analogous enough that taking issue with it feels pedantic.

Kinect games just aren't fun. I think that the designers compensate players way too much for the games to be any real fun because they don't want you to feel punished for making wrong moves. As a result, they overcompensate by making tasks too simple, or easy to accomplish. Of course with Kinect, it also can and does swing the otherway where you feel cheated by the system because you're "totally doing this right, the dumb thing just isn't picking up my movements right!" That's the problem. With a controller, the input works the same every time. With Kinect, it feels janky because you never know if it's going to properly register your movements and as a result, designers have to work around this huge flaw, best they can, by making terrible hand-holdy games.

I wish Xbox would (could) stick to their guns on the Xbox One at least for the first year. I personally love the 2.0 for navigating the interface and fantasia looks cool. I hold out hope that they are going to view this as a new challenge to prove the kinect is a worthwhile purchase (as they should anyway) instead of defeat.

Great article Patrick! I think it's important to note this bizarre and gaping gulf between Microsoft's expectations for the device and what it was actually good at. It's a shame the Kinect found itself helplessly trapped in the gulf between these two points. A Kinect in every box might have been something the public wanted if the messaging for the device had ever managed to re-focus on what it was good at doing. But, of course, they didn't do that, and here we are with Microsoft responding to the market's desire for a box without the $100 useless lump in it. Which is what happens when a perfectly serviceable piece of tech is incapable of doing the things you promised it would do.

I just gotta say, saying "Xbox open Super Time Force" and it open the game is pretty fucking rad. It's like I'm starting up a time machine. Like others have stated, it's really the little things like that, that Kinect is cool for. Thanks for the article Patrick!

[1/2] Also, if you're looking for "higher fidelity tech" to rescue motion control gaming, you fundamentally misunderstand physical play.

— Douglas Wilson (@doougle) May 14, 2014

This statement sounds like it is saying things are supposed to be wonky and the action on screen is supposed to be disjointed from the actions in the real world. If that is the case then I'll be glad when the Kinect is completely dead and buried. I have no interest in lowering my expectations for video game controls to a point where it doesn't matter if the game does what I want it to.

Oddly enough, the Kinect is what brought me to Giant Bomb in the first place. Someone gave me a link to the livestream they were doing when it launched. Watching it reminded me of when I was a kid and I would fall in a hole in a mario game, and then turn to whoever was watching me and say "BUT I PRESSED THE JUMP BUTTON" while holding up the controller and making exaggerated button presses. Only now everyone could see that actions were being performed correctly, but the game just wasn't listening.

And I don't think you can just blame developers for Kinect games being bad. If that were the case, there would be a good Kinect game out there you could point to that got it right, and everyone who played it would agree that the controls totally worked and it wasn't crap. But I've never heard such a thing. I don't think I've even seen a piece of footage of a Kinect being used that doesn't have someone going "Xbox.... Xbox..... Xbox".

I'm glad my Xbox One came with one. I never would have bought it without it being included. The mic functions are a game changer for me. I still try to talk to my phone to make it play music or call someone without hitting a button. I really like game show games, and I really want to see a Wheel of Fortune or Jeopardy game that I can yell answers at.

The Kinect and Kinect 2 was an interesting and potentially useful piece of technology, just not for video games. If Microsoft wasn't so insistent on crowbarring it into the X-Box platform and instead marketed it for other uses (see here: http://news.dice.com/2014/02/03/south-koreans-using-kinect-to-monitor-dmz/ for hacked kinects being used to monitor the South Korean DMZ) there would be a lot less heartache over it. As is there were a not insignificant amount of people for whom (perhaps incorrectly) figured that the Kinect was just a window for the NSA to look into their lives and that it was always going to be on and listening to you even after Microsoft's "Kinect always has to be plugged in" 180 last year .

Having said that, I can't get behind Patrick in part blaming the developers for the lack of Kinect appeal. In order for any controller (be it a physical controller or kinect/move/wiimote motion sensor) to work and not be frustrating it has to be responsive to your input and do so very quickly. The Kinect failed on both counts; See Jeff's TV control video back when the Xbox one was first released for the example of that. If you have voice commands and you have to tell it to do something more than once to have it understand you regularly, it's frustrating to deal with. I've got Microsoft Sync in my car and I get super frustrated with it because it has huge problems parsing albums and song names and doesn't do well with any sort of natural voice recognition.

Developers didn't fail the Kinect. Customers didn't fail the Kinect. Microsoft absolutely failed the Kinect. If this was suppose to be such a revolutionary step above the Kinect 1, then the whole X-Box package got released a year too early because the software didn't support it in the ways it should of. For example: Why does the Kinect need to be an IR blaster for your TV when every TV and cable box made in the last 5 years supports HDMI CEC and could receive commands from the XBox over the HDMI line and vice versa?

They never supported that fuckin thing. Making you buy it isn't the same as supporting it. You want the kinect to catch on? Well you have to make fun games for it.

Lots of fun games.

Not 2 fun games, lots of games.

Navigating on the xbox 360 with my Kinect was never fun or an improvement over the controller so why would I use that? The only cool thing I used my Kinect for was pausing/playing movies on Netflix. "XBOX PAUSE"

Also I don't blame Microsoft for making sound changes to the way they are selling the xbox one. It is full of spiders and the spiders are on fire, plus it's more expensive than it's competitor. Make those changes, sell more units.

Have you guys seen that Hero Cat video? That shit is bananas!

I think the Kinect could be cool.....if it didn't have horrible 'terms & conditions', disgusting future ad techniques, a huge part in the next presidential campaigning on xbox, and it worked as advertised.

The last think Microsoft cared about when it was bundled was using it for gaming. That's the problem with Kinect.

I don't think you pilot steering wheels, Patrick.

latency killed the kinect

even the latency in the voice processing is god awful

the actual skeletal tracking was pretty cool in the right conditions. and you know, if you didn't care about it lagging behind your actual movements substantially.

Almost unrelated to the article, but wow I forgotten how much I hate families in those new technology trailers. Nobody behaves like a human being in them.

I think the Kinect is a bit like the Virtual Boy, solid idea but shitty execution. Maybe in a few years it will be resurracted like they with the Oculus Rift and everybody will love it.

Great article. In my opinion Kinect will either live or die on June 9th. Maybe a VR solution is coming. Maybe they will show how the world can turn their Xboxes into Dev Kits and open up Kinect. Or maybe they will really just let it die.

The "uncanny valley" thing applies. Jumping up and down in your living room miming running in an effort to convey to the device "I am running" is definitely "uncanny valley". A bunch of what you are doing is completely fake and the sensation that it is all phoney is increased by interacting with it. Their attempt and striving to become "more accurate" just enhanced how fake it was.

Another great piece, Patrick. Your points are dead on.

Really good read Patrick, while I was never a fan of Kinect as a "hardcore" platform because it wasn't, they few times where it was just friends/family and I messing around in front of it was a lot of fun. It probably should've been something more than what it turned out to be but it really hurt Microsoft to include the Kinect with the XBONE.

It actually does, in that it it the same type of phenomenon applied to kinesthetic and spatial perceptions rather than purely visual perceptions. Here's a better explanation: Extra Credits: Kinect Disconnect

Kinect still makes the best quicklooks. I thank it for that.

I've always thought and held the belief that Kinect was misguided as technology for a console. Instead it should have been technology for things like TV. Imagine if your TV showed you "here are the three active inputs" where you could either point or give an audio cue and it would switch to it. Or it notices no one is actually in front of the TV for 15 minutes starts the process of turning itself off. Or it notices that you had a phone call that issues "mute" and "pause" (if it can). Simple tasks with direct results.

Trying to make it a complex interface into a complex environment and do complex things is what lead to it never working quite correctly in many common situations.

No. It would recognize thick Engrish pronunciation of things like song titles for SingStar and do the most basic short voice commands. It's just that only Sony used it and SingStar's mainly a Euro/Japan thing.

(Nnnn brainfart, that was the SingStar mic, came along with the PS2 Eye. Still, that was voice recognition and commants. PS3 Eye had a four-mic array to make ranged detection easier and circuits for it but who used it? Sony did..)

I tried Kinect. What it ultimately came down to is that the games weren't fun. Even the Wii games were fun to dick around with. The only Kinect game to hold my interest for more than a day was the Double Fine one, and that lasted a weekend.

June 1st 2014 shall forever be known as "Well, BAM! There it is."

Wait....Kinect died?

When?

If anything not making the Kinect a lazily forced bundle will give the device more life. Now it is required to fight to show its relevance as opposed to getting a free ride. It can now prove its worth rather than lie that it's already worthy. It should have never been bundled in the first place. Especially in 2013 (no one in their right mind wanted to put a camera in their living room last summer).

ID@Xbox is where Kinect will shine (if Microsoft lets it) and those games will never have the burden of being expected to be a best seller. The need for everyone to have one is nonexistent. The focus needs to be on creative thinking and not reaching the biggest audience. This decision to CORRECTLY call the Kinect an accessory forces that focus.

The Kinect was doomed when it was bundled. It was being relegated to just a microphone. People that like the Kinect should be happy that it actually needs to compete now. Nothing great comes from anything that doesn't require a little elbow grease.

I say good riddance to the idea that Kinect is integral to the Xbox One and part of the machine. It never was. That is what you call bullshit. Why anyone believed them is mind blowing. Pushing that kind of messaging was killing it faster than unbundling it ever could.

If you believe that developers won't make games for this now that it isn't in every single home than you have absolutely no faith in the device or the people making your games.

People will buy peripherals when they see gameplay that is appealing enough to them. Unbundling Kinect from the Xbox One doesn't spell doom for either. Dance & fitness games weren't appealing to me so I never bought a Kinect, but on the other hand I did go out and buy Guitar Hero with the bundled Guitar, and millions of others spent hundreds of dollars to buy the complete set of peripherals for Rock Band, etc. And as you point out, 24 million people did go out and buy Kinect for the 360. Sony can't keep their PS4 camera in stock because they underestimated the popularity of using it for live streaming.

People who are fans of dance and fitness games will continue to buy Kinect, as will people who want try their hand at competing with Spookin-with Scoops with their own live-streams on Twitch, and people who want to Skype with friends and family using the living room TV. And if developers come up with additional types of "killer-apps" for Kinect, it's popularity will continue to increase beyond that base.

excellent article, really makes you think about microsoft's problems considering they are leading technology innovator for the past 35 years.

The kinect in theory is a very cool peice of tech. The problem that it's always had is latency. The time it takes to track a person's movement is just too high. This is a battle the Oculus guys are dealing with, but in the Kinect case, MS has to use video processing software instead of a chip. In which case the chip is much faster, and all of the issues are mainly dealing with hardware or low level optimization.

Then compounded upon this issue, the consoles are dealing with very limited technology and processing power compared to the PC. This doesn't help the kinect at all. For the kinect to be what MS originally fantasized it as, it would need a significant boost to processing power. Kinect 2.0 is a step in the right direction, and I'm still curious to see new games that are designed for it. I'm just not sure the XBONE has to capability to give the kinect tech the processing power that it requires. And yes, I'm sure the kinect does some processing inside the kinect itself, but I'm positive it's offloading a ton of the calculations to the GPU.

Everyone should have known those kinect commercials back in the day were complete dribble, but I think MS is completely responsible for showcasing trailers that misrepresented what the technology was capable of.

tl:dr: Great tech, Wrong company.

I very much agree with Patrick's view of the Kinect. I had a great time playing games like Fruit Ninja and Happy Action Theater on my 360 because they embraced the wonkiness of motion control. Heck, I even had fun laughing at Rise of Nightmares and the inept ways that they tried to make motion controls more "realistic." But positioning the Kinect as a mandatory pack-in with the Xbox One was a death sentence from day one if Microsoft didn't seriously change their approach to the peripheral, and look what's happened now.

You'd think that sometime over the last year, Microsoft would have realized that they needed to open Kinect as a platform to more creators. I mean, nobody's using the thing in AAA development, even internally at Microsoft. Watching that shadow puppet video just made me frustrated that nobody was able to take that (or any of those other community-developed PC projects) and develop it out into something bigger. Imagine implementing tech like that into a game like The Puppeteer, and being able to influence the stage environment around you in real time. Or heck, even think about a free app that lets you stage stories for your kids. And that's just one of a hundred possible implementations.

Like pretty much every other Microsoft product over the last year or two, the Kinect has suffered more from confused messaging and poor implementation that from outright bad design, and it's just frustrating to see.

Great write up, Patrick.

I really don't get why people are saying Microsoft abandoned Kinect. Kinect is being sold as of June 7 EXACTLY the same way like any other accessory, and this way it's better for the customer if they don't want that accessory. I'm sure that when a new Dance Central comes along for the obviously improved Kinect 2, people will get it, perhaps in a bundle with that game and others in the future.

I understand the viewpoint that since the Kinect can now be purchased separately there will be a lack of user base for the accessory. But it's not like they're getting rid of all Kinect bundles - I know for a fact that when I get an Xbox One, I will get the Kinect bundle. Not really for games at all, but for everything else. I find the voice controls to work really great, maybe it's just how I talk or something. And the recognition stuff is pretty cool. I still think it makes the console unique and just because the console can be purchased without it doesn't detract from that.

People saying that the Xbox One lost it's defining factor are utterly and completely wrong because you can still buy the damn thing. People now have more options to buy the console, which is what they complained about in the first place, and now they complain about how MS is apparently setting Kinect off to die after they are provided with options for buying the console. So dumb.

Losing the kinect is sad and thank you for writing the article. There are so many cool things that could have happened with it and augmenting a lot of games in the future if it was built into every xbox. Good luck trying to convince anyone to spend the resources to add features for a peripheral now.

Can't wait to see an Occulus + Kinect on the PC do some amazing stuff in a couple years and have people get excited.