During Mark Cerny's presentation in the middle of March, people thought the 'ear stuff' was a bit funny. Certainly nobody scoffs at better audio, but on the other hand not many people without nice audios setups get to hear audio that has directionality without a home theater set up.

So, I have been thinking about real in-game world uses for what Cerny was presenting. Concepts that people could understand that went beyond "You heard a gun fired, you can know if it was fired from in front." Gun fire is a nice example of sounds directionality that SOUNDS useful. However, that is not the only thing it would be useful for. There is a bit more that can be done “Tempest” 3D AudioTech using nearly any stereo speakers or headphone if they have a bit of audio frequency range.

Examples:

Currently, when in a cut scene; some game allow you to walk around and it always humorous when an NPC is STILL talking when you turn away. But with "Tempest" when you turn, its will sound like their voice is indeed now behind you or to the side. Moreover, because more sound will have 'directionality', the developers will have tools for an interesting soundscape. In the example of talking with an NPC the developers could add other sounds and discussion around you. So that when you walk to and NPC that voice is louder and directed at you, but you could hear other lower voices behind you. Basically, giving you the feeling of hearing that one NPC out of a crowd. In the case of a game like Yakuza, you often follow an NPC while they give a monologue. In a game with 3D audios their voice would actually be positioned in the 3D space; so as you weave from behind them during and following their voice it changes direction. If you stop their voice sounds further away or if you turn around to look at a street sign their voice come from behind. In addition if you pass other NPCs in discussion their voices would pass by you as well.

In a space combat game, imagine the crew is surrounding you. So not only would each voice be district in voice timber/dialect/and personality, but they would be directionally based on their position of the bridge. In combat, you'd hear your weapons officer behind you, you communications officer to you rear right, and your engineering officer to you left. The helmsman is heard from the front with his distinctive 'Russian accent'. That is cool, right? If someone computer-station blows up you can hear it 'sizzle' from the direction it should come from. Even the elevator door opening to your rear left would "swoosh" from the correct direction.

I'm not sure about this but I think 3D audios can fake Doppler too. Meaning if someone is talking way out in front of you as you approach them the sound will feel like it is moving closer. It not just "this sound" is in front it this sound is in front and far away. Cerny mention the system can account for a hundreds separate sound sources. That tells me that there CAN be more sounds layered in every scenes. Even if sounds just arrives from 8 cardinal directions, which still means developers will be encouraged to layer sounds in more interesting ways. The example of sidewalk sound would mean SW, W, and NW could be street sounds. Meanwhile, from NE, E, and SE, while the dialogue from the pedestrians are heard. In addition the NPC you are talking to come from North, while the people sound of an assassin running up behind come from the South. That is a very dynamic sound scape especially when you turn around that sound scene 'rotates' naturally to your actual point of view.

So, while we will have to wait months to hear this sort of soundscape in a game using just simple speakers, do not discounts that is will be useful in a game. If nothing else, even in it simples/roughest usage it open up a 'home theater' sound to all players. Players that are playing on a $400 HDTV without a sound bar can still get some decent audio.

Yes, as I understand it it's not some fancy new surround sound system, but a new way of processing the audio within the game, similar to the old EAX system. What it sounds like is that it'll attempt to actually calculate the sound waves, how they bounce against different materials and within confined spaces, to get natural sounding echoes, muffling, delays and as you said, doppler effects. This will work just as well on a stereo output, a soundbar as well surround sound system since it's about the audio processed within the game world rather than some fancy buzzword for handling audio output.

Just think of it as the audio equivalent to visual raytracing. You don't need a 3D monitor, HDR monitor, VR or for that matter be worried about the placement of lamps in your home in order to see the difference between baked lighting and real-time traced lighting. Just like raytraced graphics attempt to model the light bouncing through the game world, this sound system attempts to do the same with soundwaves. It won't be a huge difference for all games (Valve and DICE have had decent methods of simulating this for years, which is why Half-life and Battlefield have always sounded so good), but hopefully it'll mean that future Call of Duty games will no longer sound like absolute garbage.

I'd like to expand the example you've given. Imagine you're in a forest, there's a guy to your 50 meters north shouting towards you, 200 meters to your south there's a cliff. You'll first hear the guys direct shout, followed shortly thereafter by the echo from the cliff.

Now imagine that there's a small hill between you and the guy to the north, and you're prone behind the hill, blocking/muffling the direct sound from the north, but you still hear the echoes from the taller cliff to the south.

There are lots of differences between how different materials and shapes make sounds bounce, which is why sound baffling like what they have in the GB studio, irregular shapes disperses the sound and removes echoes, whereas domes focus the sound towards the center.

Shooting a gun next to a metal barrier results in different echoes than if it would've been dirt or wood, and those kinds of differences seem to be what they're attempting to model.

I think the Wikipedia articles on Architectural Acoustics and Sound Reflection explain some of the concepts.

I, for one, want to hear some impressions of how this 3D audio works when PS 5 comes out.

I am actually looking forward to this aspects because it a useful feature for most owners at least the middle off the bell curve owners. For people with really weak tv speakers it probably won't help. For the folks with $3000 A/V systems it probably won't help - they have 6 or 8 speakers in their "viewing rooms" already. But, for most PS5 owners it should make a noticeable difference. Moreover, if our TVs are shitty the solution for PS5 owners will be fairly inexpensive - buy a sound bar or some decent headphones to improve what we have a lot.

All the same, I really can't wait to know how well it will work?

So this would express itself through a plain TV's speakers is what you're saying? Because I've been curious for a while if I'm going to need to upgrade to the PS5's proprietary headphones to get the most out of this system or if last gen's model will get the job done? I seem to remember the current headphones being advertised as having surround/3D audio so my assumption was always that they would work just fine, but showing off the new headphones alongside the announcement of the console's price worried me a bit they're for more than mere capitalism.

@nodima: There will be some improvement on TV audio, but the big benefits will come from surround sound or headphone setups. However, from what Sony is saying you’ll be able to realize the benefits from a decent set of cans rather than just going Sony branded.

So it's a step up from Windows Sonic, Dolby Access, and the DTS App on Xbox and PC? Cause those three sound like shit.

Worse, they somehow mess up the dynamics of close and far, up and down, to where I was losing Apex games because I heard someone right up on me that was actually medium range away and on a level above my head with the higher ground.

I had never really thought about the impact on games sound can really have until after I bought my new headphones. They are surround and dolby atmos approved. I played some Siege with them and with my old regular ass stereo headphones to compare and I found it surprising how positional audio made such a huge difference. With a set of stereo headphones there were a sense of "to my left or right" but with these new ones it was "it's to the left and below slightly behind me". The nuance made all the difference. Even in something like CoD I noticed a huge difference in situational awareness between the two headphones or the headphones and just tv-speakers. Taking that experience (which basically only boils down to "gun sound over there") and extrapolating into the examples you're giving is all very exciting and I was very excited after Cerny's presentation about audio as well.

VR headsets have been making use of some pretty crazy-wild audio stuff for a good long while. In Wolves in the Walls, for example, being able to look around and actually feel the distance and position of the girl speaking to you really helps bring the world together. Or even stuff like firing a bow with your head turned, and hearing the different ways the thwips and thocks travel to your ear. it's really cool stuff, and I'm looking forward to a wider implementation of it in the coming generation.

Yeah, I'm curious to see if this is any different from PSVR's positional audio (which... man, 3D audio really makes VR shine in games that use it well), or if it's the same tech reused for flat games? Which I'd be fine with, since, again, it's awesome.

I've been using the PS5 for over a week now and have been trying out the much advertised 3D Audio on three different headsets (Arctis 9X, Sony XM4, Audio-Technica M50) in a few games. It does sound pretty good, but I'm finding it hard to notice much if any difference to the Dolby Atmos for Headphones stuff on my Xbox.

How about the rest of you? Have you noticed a big difference compared to other virtual surround technologies, or should we wait for more games to come out which might take more advantage of the technology? Also, when and how can I send photos of my ear to Cerny?

@isomeri: I’ve been struggling to hear the difference between stereo and 3D audio other than when 3D audio is turned on for games that don’t support it everything sounds like muffled shit.

I’m not someone who is like “audio is audio”, I like a FLAC over an MP3, but for some reason I haven’t experienced any positional audio on PS5 that has tricked me into thinking sound is coming from behind or above etc. It all still pretty much sounds like left and right channels to me. Even in games like the RE8 demo where people say they really notice it it does nothing for me.

Maybe it’s my headphones? They’re great for music but maybe just not equipped for the new tech? I have them plugged into the controller, I assume that’s how Sony expect you to experience it? I dunno.

Existing virtual surround solutions are very hit or miss because everyone perceives sound differently depending on their earlobe and canal shape, and hrtf profiles are built using one shape-fits-all fashion which may or may not work for you. It's like a T-shirt that comes in only one size. If you're one of the lucky few that happens to be the right size - perfect. For everyone else - too bad.

I'm still curious to experience it myself, but usually good 3D audio mixing is something that shouldn't pop right out, but be very subtle and only noticeable if you're focusing on it.

@goosemunch: Ah, my ears my be busted then.

I wish there was a toggle in the menu bar for 3D audio do you could switch back and forth without digging into the system settings. Hard to compared game audio directly when you’re coming in and out like that.