This piece was originally written as an explain-er for a school newspaper on how videogame streaming works. I was told to pick a topic related to the future of gaming or tech and explain it to a dad or someone with the barest minimum of knowledge about games.
Feedback welcome, especially from those that understand serves and streaming better, this is an education piece for people with no knowledge and I don’t claim to have much more so educate me, make my analogies better etc.!
The pitfalls of live streaming videogames and the growing pains of cool tech.
Streaming is the future, everyone can see it. Like horses charging the horizon it’s the next Battlefield storming towards us. Similarly to the physical vs download debate there’ll always be a place for physical media, once internet speeds increase enough in built up areas it will become the primary outlet for gaming. So let’s look at it, it’s pitfalls and how the major competitors plan to tackle it.
What is videogame streaming?
It is the streaming of videogame gameplay not videogame livestreaming(I promise not to mention Ninja in this piece…).
As things stand now you have a box in your living room. This box contains a graphics card, motherboard, cooling components and all the other computer parts required to run a videogame. However the idea of streaming is that that box no longer sits in your living room. This box sits as one of many in a server farm somewhere else. This box and potentially several others (Xbox originally had planned on using cloud computing to allow multiple Xboxes to share the brunt of demanding games at the launch of the Xbox One, but we’re getting way ahead of ourselves with that) could play games and you would watch the results many, many miles away. You enter controller inputs at home, these inputs would be streamed to the serve farm and the results of your inputs would be sent back to you in milliseconds. This is why internet speed and latency is what’s been holding steaming back.
How does it work?
Think of it like the difference between a DVD player and Netflix. When you put a DVD into a player the disc is scanned next to your TV, the information on the disc is interpreted and displayed on your TV. Hitting ‘pause’ on your remote sends a message to your box saying ‘Stop the video,’ the box says ‘ok’ and the video on screen stops. When you watch something on Netflix imagine there is a computer with a hard drive loaded with films and TV shows which can similarly run(interpret) information (an episode of the office) instead of scanning it next to your box in the same room. The information is interpreted but now it needs a screen to display on. The footage makes its way through the internet and onto your screen. If your internet is slow the quality of the footage requested is reduced, if it’s fast it can handle more information, therefore the footage is higher quality.
When you hit pause this time there’s no box to tell the video to stop. Instead your command has to be sent over the internet to the farm of servers, one of which is computing the footage you requested. When you hit the pause button it’s told to stop. This is where input delay comes from and why sometimes it feels like it takes a beat between you hitting fast forward on Netflix and anything actually ‘fast-ing forward’.
What are the key problems with streaming?
Latency. When the box is in your room, the signal must only travel from the remote to the box. When the footage is being pulled from a far-off server, the message to pause is first sent to your TV and then your TV has to send a request through the internet to the server asking it to stop sending footage. Any footage that has already been sent can’t be stopped. This is why it can feel slower to pause on Netflix than on a DVD, you’re playing Chinese whispers… over the phone.
Why does this matter for gaming?
That DVD player I told you about just became a hard drive full of episodes of the office. Apply that to an Xbox. No longer does it sit in your living room but in a server farm far away or it could even be several idling Xboxes daisy chained together running all the calculations in the background when they’re not currently using their power to play a game for their owner. Interestingly, this was the original plan for games like Crackdown 3’s multiplayer. It was to have massive destructible environments that would see several Xbox Ones connecting to run one instance of a game allowing each one to compute an aspect of the game rather than one doing all the heavy lifting.
The computing power required for videogames to happen doesn’t take place next to your TV. You’re being sent a (almost) live video of that gameplay to your TV. This is only happening now because 10, 5 or even 1 year ago internet speeds were too slow.
If your connection isn’t fast enough it would be like trying to control a puppet if its limbs only moved two seconds after you pulled the string, nauseating. Or imagine turning your steering wheel and your car only deciding the wheels should follow suit a second later; not only would you have to be going through your normal driving, reactionary, thought process, you’d also have to keep in mind what you just did and remember it until it happens while at the same time trying to decide how much more you should turn the wheel to get around a corner. Like I said Nauseating.
Some games are better suited to this style of interaction between player and experience than others. It is ok for games where you are essentially making decisions and watching their impact play out then reacting. Everything from choice based adventure games to action games, with long canned animations you can’t cancel once they’ve begun, work great. These game have been designed so that you decide an action, you commit to it, you wait to see the games reaction and you continue. Either you say a line of dialogue and a second later you hear it and a character’s response(like a TellTale Game) or you run behind an enemy and choke him out while no one is looking (like in uncharted or Tomb Raider) – you ostensibly ask the game a question and it answers you. Hit one or two buttons and then the game will show you something cool. Things get troublesome when games begin asking you questions, that need a fast reaction. Like, think fast!
When a game asks you to react, is where streaming has fallendown for years. Let’s look at one of the most commonly cited examples; fighting games. In a fighting game, like Street Fighter, with two characters on screen you see what move your opponent is doing, the game asks you what are you gonna do about it, and you react; dodge, block or try to get out a quicker attack to interrupt. This has always worked best in person rather than over the internet. When online, one player is often deemed the ‘host’ (the more accurate gameplay, the instance the game is happening in) and as things happen on their screen the results are sent over the internet to their competitor, giving the host an advantage as the game is logic-ing out what is happening live for them, while sending that information to the other player after it happens.
Confused? Here’s an example; I punch the host, the host takes damage, on the host’s screen they see this and go to block, if we’re looking at the same monitor I’d see the animation start and back off. When we play online the game can’t predict, on my end, what reaction the host was going to have, so it shows them doing nothing, only for a few frames, but long enough that I commit to another attack. This attack doesn’t connect because when that message reaches the host, his characters hands are up and I’m left wide open to be countered. This can be somewhat mitigated with dedicated servers hosting the game, basically a halfway information hub between the two players to make the delay less noticeable because both player are connecting to the same midway point. This helps, especially for multiplayer games with many players. In general however, this is why fighting game tournaments can’t be held online and have to be in person. Any delay is too much of a delay.
How does latency affect streaming?
These same latency problems are inherent to streaming, it’s like adding more people to your game of Chinese Whispers, even if the message stays accurate, it still takes longer to pass along, more things(servers, consoles, TVs, controllers etc.) have to be pinged, checked and confirmed. This impacts any game where the game relies on your time to react to it rather than you making the game simply show you something. When streaming poor latency will make it feel like there is a delay between what you tell your character to do and when they do it.
So why now?
Basically our internet is fast enough now, with things like 5G on the horizon so that the time it takes these messages to travel between points where decisions are made is so short that the new limiting factor won’t be the delay from the internet’s end but the computing speed. The internet is getting fast enough that it’s beginning to no longer feel like we’re waiting for answers but like we’re having a conversation in person. We’ve gone from texting as fast as we can to having a phone call.
So how do the billion dollar mega-corporations plan on taking advantage of this infrastructure?
With millions of equally powerful computers(Xboxes) already in peoples homes, more server farms than you can shake a stick at and the sheer amount of people running Windows 10, Microsoft is an good example of a company that has high hopes and lofty ambitions to unite as many of your devices through one account, to quietly become ubiquitous. Only the other night at their most recent Inside Xbox event they showed off project xCloud for the first time in public.
If xCloud works as advertised it could be a game changer, allowing you to stream any Xbox game to anything with a screen and Xbox app, meaning you’d no longer need to even own a physical Xbox to play Halo on your TV. You would simply require access to fast enough internet and a bluetooth controller to allow you to play, theoretically, games at 1080p or 4k and 60fps or 30fps depending how demanding the game was on the offsite hardware. Their key feature would be that you could change devices and the games progression would follow you, play 10 minutes of Gears of War at home, come to a break, sign into your account on your phone and continue where you’d left off on your way to work.
From all the industry scuttlebutt we’ve heard this seems likely to be both Amazon’s and Google’s goal too. Convince people, ‘hey you have a google/amazon account already connecting your phone, laptop and TV so you can check your email on all of them or continue to seamlessly watch videos using Amazon Prime what if you could do the same for games? Buy this USB stick or download this app and pay a monthly fee and… Hey presto! Videogame-Netflix!’
With Mircosoft(Xbox), Amazon and Google going all in on streaming, competing for the game anything anywhere market what will the rest of the industry do? Nintendo will continue to do Nintendo things and make enough money to fuel a small country purely off of Pokemon-hardware-bundle sales. But personally, I think Sony will be in an interesting position. Back in 2015 it was announced Sony was buying the failing OnLive, one of the early game streaming services. OnLive were trying to do exactly what Microsoft showed off at Inside Xbox the other night, but at the time the infrastructure just wasn’t there. So Sony sat on the tech for a while only to launch PlayStation Live; allowing you to play PS3 games on your PS4 by (very basically) running them on PS3 hardware and streaming the results to you PS4. The service has changed massively since then to something more akin to Microsoft’s offering in Gamepass (basically you can download many of the non PS3 titles, I’m not getting into the complications of emulation right now! But it is basically a subscription service for videogames).But Sony still have the streaming tech. Personally, I believe Sony will offer a streaming service but it’ll be less robust than the other options being focused on streaming to a console or TV while still selling a box(mid-tier PC) for premium gaming experiences.
Basically Microsoft, Amazon and Google all want to become the interconnected Netflix/Hulu/etc. entertainment hub for games, Sony wants to be the entire box office film industry turning a profit (just no Super Normal Profits) on AAA/Blockbusters and indie experiences, saying that ‘this is the best way to experience them’, as we currently do. They’ll still sell a console and physical/downloadable games and advertise it as the true way to experience games with minimal lag or delay with the option to stream them.
And Nintendo? Well in this analogy they’ll keep selling Labos with Homemade covers out of the back of a sedan at a flee market. They won’t change, and their market will be there. But for everyone else? The race to fill the void of the morning commute is on and it’ll be down to who ever can offer the best service.
Log in to comment