Most of the comments in my blog about Sly Cooper and the Thievius Raccoonus were from people who wanted to talk about Sly 2. I'm playing through that game and will have thoughts when I finish, but one thing that has really stood out to me is the sound. The game has fantastic music, but it also has really impressive use of surround sound. I bought a surround sound set up a little over a year ago, mostly for gaming, and I've found it pretty disappointing to be honest. There are a few games that have good separation and make use of the rear speakers, but the vast majority don't do anything special with it. Most games are set by default to surround sound (when I used TV speakers I remember having to go in and change the setting on almost every game until that became a console wide thing) but they don't do anything of note with it. Sometimes you'll hear sound effects like gunfire from the rear speakers or what have you, but it's often indistinct in the mix.

Sony is now pushing 3D sound where it's separated vertically with their new headset, and sound coming out of a speaker in your controller (Destruction Allstars was traumatic) but I feel like those things are gimmicks and can't be trusted when even basic surround sound is used so sparingly.

In Sly 2 (and I've seen some buzz that this may have been altered for the PS3 version, though the PS2 version had surround sound too as far as I can tell) the surround sound stuff is clear and notable. Not only do ambient noises come from the rear speakers, which helps to immerse you in the game, but sound effects often come from them too, and there's even an in game use because enemies and collectables make lots of noise and you can locate them using surround sound. I was able to find a hidden guard with a key in one level by tracking him with my surrounds. The bottles you pick up clink when you're near them, and you can use your surrounds to track them down.

This is awesome!

Not everyone has surround sound so of course you can't put key game functions into it, but you can add little extras like these that really add to a game. In addition, the vast majority of people do have sound of some sort, so adding gameplay sound cues like collectables that make noise is great. Some people are hearing impaired, or just play with sound off, but you can add accessibility options to add visual cues for those players.

Sound in games seems much less developed than graphics. There are a number of reasons for this but I feel like there's a lot of opportunity to innovate there. I know that there have been sound only games, and plenty of games that used sound in interesting ways, but to play this 2004 platformer and be consistently blown away by how it uses sound is just wild to me. It really brings into focus how neglected sound has been in so many games. Lots of games don't even bother with decent sound mixing; making dialogue impossible to hear because they don't turn down music and sound effects when people are talking. I remember that Dirt 5 tried to do something interesting with surround sound audio by having the music play out of virtual speakers on the track when you drove by them, and it just made it impossible to hear the music. Close but no cigar.

Sound can be even more immersive than visuals, especially with surround sound, because the fidelity is closer to life. You always know you're looking at a game on a screen, but with the right sound mix and positional audio you can actually mistake game sound for real life sound. We've all heard the phone ringing in a movie and thought it was in real life. Sound has reached the level of near perfect replication, and yet it's so neglected.

Sly 2 has aged badly in a lot of ways but its sound design not only stands up to modern games but exceeds most of them. As game graphics get more and more expensive and reach a point of diminishing returns there's lots of opportunity to improve sound, and Sly 2 shows that you don't need cutting edge tech to do it, just a little thoughtfulness and effort.

Given how many games I have to turn subtitles on in just to understand the dialog, there's a long way to go.

Sound design is something that will always take extra effort that developers either can't afford or be bothered. Many videos have been made to show how Thief's sound design in 1998 is leagues beyond most stealth games released since. In the Thief games, you can play by sound alone knowing where guards are located. In modern stealth games, you often have to rely on some gimmick where you see through walls instead.

I wonder if there's any data on how many people use headphones, how many use TV speakers or a basic pair of stereo speakers, and how many actually take advantage of a full on home theater setup? Devs might not really have time for a feature only a handful of people will use, or might not even notice if they're all PC gamers who game on a desk in a corner with expensive headphones.

Anyway, I remember the PC version of Skyrim worked correctly when I had it plugged into my setup. Things happening behind me came through the rear speakers and sounds sort of panned through the correct speakers when spinning around to face another direction. I seem to recall Minecraft doing the same thing. And Naughty Dog's games have a bunch of audio settings to fiddle with.

@lonelyspacepanda: I doubt it's a "can't afford" thing considering how much money is spent on big budget games these days. Given the budgets of games like Thief or Sly 2, and how well they did with it, it can't be that expensive in comparison to all the graphical flourishes.

My guess is that there are a few things at play.

The first is promotion. Most games are promoted on Twitch and Youtube these days and those don't tend to encode with fancy positional audio so it wouldn't come through for the viewers (especially not with voice over and obnoxious Twitch sound effects that make those streams unwatchable.) So just like audio couldn't be sold in a screenshot it can't be sold on Twitch.

The second is a split userbase. Surround sound set ups are relatively rare (even though it's the default for most games) and there are people who are hearing impaired or playing with sound off etc... It's neat that you COULD play Thief by sound alone but you can't actually design a mass market game that way. It needs accessibility.

The third is just fitting into people's lives. If you have a family then just like VR you are unlikely to be playing in the right sound environment to enjoy or use good surround sound. It just requires a much more specific environment.

That being said, it's awesome as a supplementary thing for those who do have the opportunity, and with both Sony and Microsoft selling fancy headphones you'd think it would be more of a priority but it just seems like it's not.

@justin258: Considering that modern set ups gather data on virtually everything, I'd be shocked if they didn't have that data in some form. The PS5 tells you how long you played each PS4 game you owned so they were collecting that data from us behind the scenes. They probably know what every user who is connected to the net's settings are. Heck there's even a good reason to collect that data when we send error reports, since some crashes might be triggered by certain set ups and audio settings etc...

That being said, there are fancy 7.1 headphones out there so I'd bet that the number isn't miniscule, even if based on my anecdotal observations it's probably less than half. Someone should do a Giant Bomb poll.

There are definitely some games that do positional audio well. There are others that try and do it poorly (I am STILL mad at Dirt 5 for this.) I was just taken aback because I noticed it so clearly when playing this incredibly old game and I was thinking "wait, I play much newer games all the time and rarely notice them using sound like this."

@bigsocrates: Nothing in game dev is free. It takes time, resources and money to both record the foley work and program that into the game (enemy on carpet at x length from player, enemy on concrete above player).

Was interesting watching Overwatch 2 preview at Blizzcon and they had a whole segment promoting these features, which should tell you the above.

Now what is interesting is that there may be baked solutions to this stuff in the near future that make 3D audio and the logic around believable environments easier to achieve. But really the payoff is rarely there unless it's very atmospheric (Demon's Souls) or audio queues are a key feature (stealth games).

@lonelyspacepanda: Nothing is free, but the foley work is not relevant here (they're already making the sound effects) and we know that the positional audio programming isn't that expensive because games like Thief the Dark Project did it well and that game's budget was $3 million, which even adjusted for inflation is lower than some Kickstarter games. It's somewhat more expensive to do it now because of larger and more complicated worlds with more moving parts, but it's nowhere near as expensive as building the graphics are (and again we're not talking about generating the audio assets, just making them work within the game world.)

A lot of this is just not thinking about it and the same kind of developer laziness that leads to issues like not being able to pause during a cut scene (this ALWAYS infuriates me; it's like game developers have never actually played a game in a real world scenario) or the awful sound mixing so many games have. Decent sound mixing is not expensive but so many games just don't bother to even try to make it good.

And even those that have the tech for positional audio don't try to incorporate it into games in a meaningful way like with Thief's guards or even Sly 2's little collectable sound (which you can do even without positional audio by just making it louder or softer as you get close to it.)

I'm not saying that every low budget game needs perfect 3D audio or anything, or even that every high budget game needs an audio focus, just that it's a very underexplored space and one where there has been a surprising lack of innovation and focus over the last 20 or so years.

And the sound mixing stuff is inexcusable. That should be as basic to game development as making sure you don't have blatant graphical bugs, but again I think the focus on Twitch and Youtube as places where games are discussed makes it a very low priority because good sound mixing doesn't come through when someone is screaming over the sound.

@bigsocrates: I don't doubt that there are developers who take presentation on streaming platforms into consideration when designing games, but I do doubt that it's a large percentage of them. What I suspect is driving a reduction in truly high quality surround sound mixes in games is the transition to flat-screen televisions (which wouldn't have been nearly as common when Sly 2 came out in 2006) and the associated push toward things like soundbars as the most prevalent external sound source. Keep in mind that in 2006 you were much more likely to see "home theater in a box" setups being marketed, which included a receiver (often a DVD player combo) with some cheap 5.1 speakers, so more people had dedicated surround systems. Those made more sense when the TV was a large box unable to be hung on a wall, so there would more likely be space in a TV stand for a receiver that could process surround signals. With a push toward less intrusive entertainment installations, there is naturally less space for that kind of thing. Soundbars that can be wall-mounted make more aesthetic sense, and are good enough for most people.

An additional factor is that both Xbox and Playstation controllers now have built-in headphone jacks, so the frequency of players using headphones instead of dedicated sound systems or TV speakers is likely much higher. I almost exclusively use headphones these days since I don't want to disturb other people in my house, despite having a very capable dedicated surround setup. Virtual surround sound codecs, while not great, are substantially better than they used to be, so convincing surround sound is possible with headphones even with a stereo signal.

Sound will just never be as big a focus as visuals, because it's not as obvious and true surround sound is too cumbersome for mass-market saturation. Most people will default to the simplest option that is good enough, and that is what's going to drive content development.

@ralphmoustaccio:I don't agree that developers don't take presentation of marketing materials into account when making these kinds of decisions. We know from lots of evidence that many developers focused on visuals that looked pretty in screenshots over things like performance back when those were how games were sold. Now that games are sold via Youtube and Twitch and trailers (often viewed on phones) I am sure that those media drive a lot of development decisions.

The fact that virtual surround sound codecs have improved and many people are using headphones is actually a reason why more developers should consider putting effort into something like positional audio. Both Xbox and PlayStation are pushing fancy headphones with virtual positional audio drivers so lots of people are playing with some ability to hear that stuff.

I don't actually think surround sound was ever particularly common (at least from my experience.) Most people have always relied on TV speakers or just a soundbar or couple of speakers rather than a full 5.1 (or 7.1) set up. I think that with fancy headphones you might actually have a better sound system on average now than you did in 2004.

I think one big difference is that there's more room to go really hog wild with visuals now, while on older machines you might have gotten more bang for your buck with impressive sound because visuals were more limited. The OG XBox could push very sophisticated soundscapes, and of course sound quality was excellent even back then, while visually it was primitive. Now visuals are much more impressive, while sound hasn't improved technically much over the OG Xbox capabilities (though it has a little.)

That doesn't mean that sound should be ignored as much as it is, though.

@bigsocrates: With regard to the virtual surround codecs, the fact that they are improving would necessitate less effort being put into a traditional positional sound mix, since the codec does the work to synthesize the positional nature of the sound from the stereo mix. Taking the time to hard-code a fully positional 5.1 or 7.1 mix wouldn't provide a substantial enough benefit to those relying on virtual surround to make it worth it, based on my admittedly limited understanding of how these things work. And if there aren't as many people with dedicated 5.1/7.1 setups, the time and effort to properly code those likely is something that can reduce an already tight budget of both time and money on game development.

I will disagree about the prevalence of surround sound systems in the early to mid-2000s. As I said in my original reply, those things were everywhere. You couldn't go into Best Buy, Target, Walmart, etc, without almost literally tripping over them. I knew so many people who bought them because they were only slightly more expensive than a standalone DVD player, and, hey, surround sound! Never mind that they were almost universally horrible and probably did more harm than good in terms of making a surround sound setup in the home seem worthwhile. If the sound is barely better than a soundbar, why mess with all those speakers, wires, etc?

I absolutely agree with you that sound is critical component of an immersive experience, and I am a bit bummed that my expensive, acquired-piecemeal-over-several-years surround setup doesn't get more of a workout. But again, I end up playing wearing headphones 80%+ of the time, so I guess I'm content with good enough, too. Unless it starts impacting sales, I don't think we'll see much of a change.

Honestly I'd like to see an actual engineer or programmer who has worked on these things try to answer this. I doubt this is as simple or easy as you seem to think it is. Thief: The Dark Project is impressive in a lot of ways, but are you playing that through a 5.1 system? It's a PC exclusive, so you're probably playing that with headphones or a pair of stereo speakers. It would have been intended for a stereo setup back then anyway - three fewer channels and no subwoofer to consider when programming where sounds are supposed to be coming from.

Anyway, I'd hazard an uneducated guess that a lot of audio design in games is targeting TV speakers, headphones, and soundbars, not an expensive and complicated home theater setup which few will actually take advantage of. Can you include that? Sure, but you've gotta program it and bug-fix it and make sure it all works right and such. If great surround audio were such an easy thing to include, you'd see it a lot more often. In any case, this subject calls for more research and I don't know where to point you on that one.