Skyrim w/ Kinect and Voice Control

Avatar image for kinectfaast
#1 Edited by KinectFAAST (5 posts) -

What I am posting here is essentially a response to Cliff Bleszinski's (of Epic Games) recent comments that he would like to see a Skyrim utilizing the Kinect where you could "cast my spells using my hand". (Cliff's interview covered by OXM:

What I have done is tried to create that exact experience he is describing. Using the Kinect and Skyrim on PC, I have made it so you can use both gestures and voice to control your character.

That means swinging your arm to slash your sword, casting spells with your hands, and thrusting your arm forward to block with your shield.

This also means Dragon Shouts.

Of course there is going to be those who simply do not like the idea of motion gaming. There always will be. But for those, who like Cliff, want to see Kinect functionality taken to the next level. This may be the starting point.



Loading Video...
Avatar image for mikey87144
#2 Posted by mikey87144 (2091 posts) -

Wow that sounds like the exact opposite of fun. While it would be interesting to say "Fire Spell" to put it in your hand instead of pulling up the menu issues come after you start acquiring more spells. Does it know which fire spell I want? Will it work when I say it? Kinect doesn't really work 1-1 even now. Count me as one of those people who don't care for this option.

Avatar image for relentlessknight
#3 Posted by RelentlessKnight (1132 posts) -

this looks fake

Avatar image for nekusakuraba
#4 Posted by NekuSakuraba (7807 posts) -


Avatar image for the_nubster
#5 Posted by The_Nubster (3045 posts) -

Wow, I have to say, wonderful job! While I'm against such involved motion control myself, you know, whatever floats your boat. It's amazing that you've worked this all in, and it looks like it's as responsive as Kinect is known to be.

The question I have is, do you think you could find a happy mid-point with Kinect controls and an involved, massive game like Skyrim? I would love to play with a controller, but have the ability to use the dragon shouts like an idiot and wake everyone up, or speak the dialogue (since your character is silent).

Again, fantastic work, very impressive.

Avatar image for thekramer89
#6 Posted by TheKramer89 (474 posts) -

solid attempt, but it's looks terrible.

Avatar image for shiftymagician
#7 Posted by ShiftyMagician (2179 posts) -

Definitely a cool implementation of the Kinect, however the thing you definitely have to understand about some more rational nay-sayers is that performing physical interactions over a long period of time just to do something people have already done over a more comfortable control scheme will always feel like a step backwards, even though the idea by itself is definitely something to look more into.

If anything I would rather a game like Myst but built specifically for Kinect, where you traverse the world at a slower pace and use your own hands to interact with some unique puzzles that require precise motions. It wouldn't require the player to rely on a control scheme that is not really as reactive as one would like and it would also take way more advantage of how free-form the kinect controls can be.

Other than that, this is still a really great effort and I hope you continue to improve on it or re-purpose it in other games to demonstrate different ways to attempt to increase the potential level of immersion that can be achieved. Good luck on it mate!

Avatar image for akeldama
#8 Posted by Akeldama (4356 posts) -

Why must people insist on making games harder to play, less responsive and more broken than they already are? Pick up the controller and stop trying to break everything.

Avatar image for canucks23
#9 Edited by canucks23 (1080 posts) -

If that's real then that's pretty cool that you got it working, but who in the fuck would want to play a hundred hours of skyrim using kinect, let alone 5 minutes.

Edit: It looks fake as hell though...

Avatar image for john1912
#10 Posted by John1912 (2362 posts) -

@RelentlessKnight said:

this looks fake

Avatar image for jasonr86
#11 Posted by JasonR86 (10153 posts) -


It does look kind of fake. But, if it is real, that's pretty impressive. I just wouldn't want to play a long game that way.

Avatar image for kinectfaast
#12 Posted by KinectFAAST (5 posts) -

@The_Nubster - Appreciate the kudos! I think that a large world such as Skyrim presents a challenge, and a full voice and gesture setup may be more suited to a slightly smaller world. That said, I think that these both these elements have worked well so far. As for your second question question, yes you can definitely use just the voice controls in combination with traditional controls. This hybrid style is totally possible, and I actually use it when I am not using the Kinect. Check out my youtube description for more details on how this works.

@RelentlessKnight - it would probably be more effort for me to coordinate and fake this entire video than it was developing the gesture and voice control. If you check out a few of my other videos, they better demonstrate the technology behind what you are seeing.

@TheKramer89 - Hey, a guy has to try right.

@ShiftyMagician - Hey, thanks for the constructive feedback! I totally agree that the physical aspect isn't for everyone, especially those who have a long history of controller based gameplay. It is not as though I play only with the Kinect, it is usually in short bursts, just to get up and do something different. Something slower paced would certainly be more appropriate, as there the Kinect lacks a certain degree of precision. I have been considering doing something like GTA4, where there is some autoaiming available. Again, thanks for the support, I look forward to kinecting and sharing more games in the future.

Avatar image for mac_n_nina
#13 Posted by mac_n_nina (293 posts) -

@KinectFAAST: Looks good and really impressive, but it also looks like it's a chore to play.

Avatar image for xyzygy
#14 Posted by xyzygy (10595 posts) -

I think it would be more useful simply for the voice commands. Things like: 
- Changing weapons 
- Changings spells 
- Opening map, skills, magic and items inventory 
The actual scrolling through the inventory and map would be done better on a controller, but having the voice implementation would essentially rid the game of the D-pads use (the favorites system) and it would get rid of the default B button action (open the menu where you can go to skills, map, etc). Which would be awesome.

Avatar image for the_laughing_man
#15 Posted by The_Laughing_Man (13807 posts) -

Looks clunky as hell. And something about all the movement looks fake. And the movement he makes to move from side to side is the same he uses to turn around...

Avatar image for doublekr
#16 Posted by DoubleKR (120 posts) -

Why didn't you just use your hand to scroll in the menus?

That makes the most sense, or is that just not possible?

How long did this take to do?

Do you really want to stand for a 200+hr game like this?

Also, did you work in every name of every shout?

Very impressive, how would someone learn to do this?

Avatar image for mordeaniischaos
#17 Posted by MordeaniisChaos (5904 posts) -

@Akeldama said:

Why must people insist on making games harder to play, less responsive and more broken than they already are? Pick up the controller and stop trying to break everything.

It'd be really cool if actually implemented correctly. Especially if it got into full motion, real 1:1 controls in a good game. Add a few props like guns or melee weapons to hold and use to create more real physical interaction, maybe some VR goggles, and this idea could really go places. You have to remember that if this is real, it's jerry rigged and it'd be hard to really show off the true potential of something like this. That said I think you'd need some sort of giant open building that recreated the terrain of the game some how or a 360 degree ball thing to walk on or something to really get this stuff right, and that'd be hard with a more complex environment with lots of hills and cliffs and all that.

Also, the voice control stuff, if it worked better with menus, would be really awesome. Or being able to just say "drink health potion" instead of having to have it in your favorites and breaking the flow of combat every time you need to use a potion. This looks a bit odd, with the not actually saying the propper names of some things ( I don't think there is any weapon in the game just called "longsword" and he said "double heal" and got double fast heal. I dunno if this is legitimate but it's pretty impressive if it is.

Avatar image for akeldama
#18 Posted by Akeldama (4356 posts) -

@MordeaniisChaos: So what you're telling me is you want VR? Nobody wants to play Skyrim like this for 20 minutes. How would one even attempt to play it for 100+ hours?

Avatar image for kinectfaast
#19 Posted by KinectFAAST (5 posts) -

@DoubleKR said:

Why didn't you just use your hand to scroll in the menus?

That makes the most sense, or is that just not possible?

How long did this take to do?

Do you really want to stand for a 200+hr game like this?

Also, did you work in every name of every shout?

Very impressive, how would someone learn to do this?

Well, I'm glad you enjoyed it! To answer a few of your questions.....

- I had a setup where I used my hands to scroll in the menu, but unfortunately it caused bugs with the rest of gameplay. Really, under my current setup there are two alternative ways to scroll in the menu that i do not use. One is to use my right foot forward or to lean back, the other method is to use my left arm to control the mouse. I am going to continue to experiment and see if I can get the hand swiping system to work effectively.

- This has been an ongoing project that has spanned over multiple games. I first started off with Morrowind, and since I have done videos for Oblivion, Dead Island, and Bulletstorm. You should check them out. They are simpler in a sense, and do not have voice control, but are pretty fun. Bulletstorm is one of my favorites.

- I'm not sure if i want to stand for 200+ hours and play like this, no. It's actually pretty physical so prefer it in short bursts, just to mix things up.

- I think I have about 6 shouts unlocked at this point and have 3 or 4 programmed in. I really don't use more than 3 on a regular basis.

- The tools and programs behind what is done in this video can be found in my Youtube description. The pages that are linked there give a fairly detailed explanation on how their middlware operates.

Avatar image for marlowefire
#20 Posted by Marlowefire (168 posts) -

This is very cool. I could see playing like this in short bursts and calling it "exercise." the voice stuff would be very awesome to switch spells even if it was only limited to your favorites. That way you could say flame spell and it would only pull from your favorites. If you want 15 different flame spells in your favs then that's just wrong. I've seen favorite lists in there that are 30 items long. Seriously? Maybe it is just conducive to how I play 2 daggers a bow a healing spell and a sword.

Avatar image for bombhills
#21 Posted by bombHills (650 posts) -

Impressive, but I wouldn't want to play the game this way. Just give me voice commands to switch spells/weapons and I'll be happy.

Avatar image for sn1per
#22 Posted by Sn1PeR (153 posts) -

Amazing job! /followed

Avatar image for krakn3dfx
#23 Posted by Krakn3Dfx (2722 posts) -

I just flew in from Marketh, and boy are my arms tired.

Avatar image for seppli
#24 Edited by Seppli (11232 posts) -

That was awesome. Though, I'd really want a combination of a gamepad (broken in half) and touchscreen/motion controls/voice commands. Could be amazing. Looking foward to seeing what the next console generations standard human interface device will be. Hoping they'll not go for 'simplified'.

From the looks of it, I can imagine we're in for a treat.

Avatar image for mr_skeleton
#25 Posted by Mr_Skeleton (5195 posts) -

Someone hire this guy!

This edit will also create new pages on Giant Bomb for:

Beware, you are proposing to add brand new pages to the wiki along with your edits. Make sure this is what you intended. This will likely increase the time it takes for your changes to go live.

Comment and Save

Until you earn 1000 points all your submissions need to be vetted by other Giant Bomb users. This process takes no more than a few hours and we'll send you an email once approved.