The Graphics: Episode 1 - Shadow Maps

I've been meaning to do this for a while. Everytime I hear one of the crew say something about a games graphics that I understand, I feel like explaining what is going on. Now I'm not the most erudite member of the community, however this is my little attempt a explaining a gripe that Vinny brings up in the quick look for Resident Evil : Revelations on the 3DS.

Now it'll probably be completely indecipherable to most, but hey, watch it, see what you think, and leave critique either here or on the vid itself.

If there is anything graphical that irks you in modern games, chuck that in a comment and I'll see if there's any common themes. I've already got a few things on the list, and this is one of the first vids I've done, so you'd expect at least a slight raise in quality in time. Plus you make a game out of counting the amount of times I scratch my beard...

cheers for taking the time :)

I've been meaning to do this for a while. Everytime I hear one of the crew say something about a games graphics that I understand, I feel like explaining what is going on. Now I'm not the most erudite member of the community, however this is my little attempt a explaining a gripe that Vinny brings up in the quick look for Resident Evil : Revelations on the 3DS.

Now it'll probably be completely indecipherable to most, but hey, watch it, see what you think, and leave critique either here or on the vid itself.

If there is anything graphical that irks you in modern games, chuck that in a comment and I'll see if there's any common themes. I've already got a few things on the list, and this is one of the first vids I've done, so you'd expect at least a slight raise in quality in time. Plus you make a game out of counting the amount of times I scratch my beard...

cheers for taking the time :)

Entertaining and enlightening.  My favorite. 
  
Oh and something that bothers me is, well, best explained using Skyrim on consoles. The load screens have items or characters in them, and when it's something like a sword, there is this weird effect of the "jaggies" on the linear regions (edge of sword) "cycle" from a specific point. It looks like the effects of anti-aliasing happening in time my eye can notice. This is hard for me to explain, since I have no idea what is doing that - maybe something to do with the resolution or the rendering cycle? Any help would really clear up my poor brain. 

sounds like you said, aliasing happens when you try and condense something like a smooth line into specific discreet points, like a grid of pixels(tv/monitor), lately they use different filtering and sampling techniques to minimise this, It's been a real curse of computer graphics that things are rendered sharp. And there's a real push for the softening of edges in cg. When i'm near a computer next (currently on my phone) i'll write a more detailed explanation for you :)

I always enjoy hearing explanations of CG concepts, even if I already know how a lot of these things work, having studied graphics in school and on my own time. This was pretty good, and I think as you develop your video editing skills these can only get better. Having the z-depth rendering as an example was also pretty neat.

@SirOptimusPrime I think what is probably happening in that case is an edge that isn't perfectly aligned with the grid of pixels is going to tick over from one pixel to the next in a sort of rippling fashion rather than all at once. Hold on, I'll make a quick diagram.

Pretend each grid above is a frame rendered in a game, with left to right indicating the passage of time. The white parallelogram without the grid is just to show what the pixels are attempting to approximate (though ironically, it is itself an approximation of a parallelogram, just with a much finer grid of pixels). Alright, see how the white parallelogram (representing the sword in your example), is moving across a grid (the pixels of your television) from left to right, but less than a full pixel for each "frame"? For each frame that is rendered, the renderer tests to see what color is at the center of each pixel and applies that color to the whole pixel. That is what causes the "jaggies", technically referred to as "aliasing".

You can see here, that the whole white polygon moves across, but the pixels don't all switch color evenly. Here the bottom pixels which are farther forward switch first, with the changes rippling upwards. So that sort of crawling aliasing is based on sub-pixel movements. Keep in mind this is for rendering without anti-aliasing. Anti-aliasing, which increases the sampling so we don't just apply the center color of a pixel to the whole thing, can minimize that effect by registering sub-pixel movements.

Whoops, that explanation was a little less concise than I'd hoped it would be. Did I just make things more confusing? Does that diagram even make sense?

Brilliant, I've always been a huge nerd about what makes games tick, and stuff like this is awesome. I keep telling myself I want to be an artist, maybe a modeler/animator, but engines and the way they are engineered always gets my brain excited. Definitely want to see more, particularly about lighting, which is something that A) I think is furthest behind everything else in many games (it's starting to get better but I can't remember any game that rendered shadows with the umbra, penumbra and antumbra in effect, for example) and B) I just think the way light interacts with the world (and there'fore may interact with a game engine) is damned interesting. I also think excellent lighting is the best way to make a game look great, especially in motion.

My only critique of the video is that your not super clear, I dunno if it's just the way your voice translates or your mic but it might help to have a closer mic? It wasn't a big deal but you sounded a bit quiet and maybe a little muffled, especially compared to the little intro.

@MordeaniisChaos Thanks for the crit, it's pretty much what everyone says. It's due to me mumbling in general (thanks parents!) and my in built microphone (Canon 550d/t2i if anyone is interested) although being pretty good , doesn't help too much with my voice in particular. Also I thought that going without a script would be easier... it's not.

The problem with lights using all the umbra penumbra etc, and also with lights that aren't a single point or a directional light (like sunlight that has only 1 distinct direction (you're going to get soft shadows then rather than a simpler "this is lit, this is in shadow" solution) is that you have to usually resort to raytracing of sorts, where rays of light are sent out from the light (I'm a bit rusty here but in practice I think it's usually toward the light) and tested along the way for if they hit an object. this can be really computationally expensive, and to get something that doesn't look blotchy or spotty you need to do it many many times, we do have Shadow Volumes, which are shapes that literally occupy the "shade" as a solid that we can test against, but for this we'd use a smaller detail model on the character/props; once again to speed things up. may do some stuff later on those.

@SirOptimusPrime@Scrawnto Yeah that's about it, the one thing I was going to go deeper into was the sampling, in your example you've sampled only lines that cover the centre, and like you said you can do sampling which stops it just being the same colour. for other people, if you sampled the whole thing 4 times (for example) if one of those sampled points was covered by the polygon edge (line), then the colour that is rendered would be a 1/4 of that of the actual pixel. keep in mind that scene are (supposed to be anyway) rendered from the back forward. so you're constantly adding different colours onto those already rendered. the points sampled are usually even (like for four it'd be like the circles on a Lego brick), however these are sometimes moved around a little as it's still a little too square.

again, Cheers for the time taken to watch it.

Ah thankyou! I'd been wondering about this from GT5..

That was brill, very informative, you have one of those voices perfectly suited for documentaries or lectures which makes this very easy to listen to as well.

Keep it up.

@Vegetable_Side_Dish , @SerHulse cheers guys, glad you got something from it :)

Thanks for the video, that was incredibly informative.

Thanks for sharing duder.

That was amazingly informative, keep doing this and keep posting them! 

@Robster: I understand the mumbling thing. I skype a lot with people and get shit for it a lot. And I don't have a silly accent :3

I kind of wonder if Raytracing will ever be something we see used in any significant way in games. It's not terribly efficient and the computing needed for every frame goes up really fast the more quality you want to pump out. I dick around with blender (again, planning on eventually getting around to being a 3d artist), and even considering that the render process isn't designed for pure efficiency or speed, the render time gets real long once I turn up the quality. Perhaps there will be a sort of branched off bit of technology that is optimized for real time (I know stuff like that exists already but there's a reason it's not really in games yet). I would love to see if, as it opens up the way for a lot of other cool shit like real caustics, indirect lighting (which I wish more games would do Mirror's edge style, even if that stuff looked pretty rough on the 360 version, it still added to the look of that game).

On thing I've been struggling with lately as my PC is far from impressive is lag from post processing effects. Been playing The Witcher 2 on a (failing) 5770 and everything else in my rig is probably worse still than that, and I can't turn on all the SSAO, Motion Blur, Gameplay DoF effects without having about half a second+ of lag. I think I sort of understand how that stuff works, but it might be a cool thing to go over. I'm a fan of post processing (it's pretty much the reason Crysis 2 looks as good as it does, and Witcher 2 is greatly helped by them as well), but they can also be a pain to implement properly, judging by how many games suffer from lag on less powerful systems. Hell, I have enough issues hooking up nodes in blender right just to get DoF working at all.

@mordeaniischaos i've actually planned a big video on post processing in realtime as i wrote a paper on it in uni. As for mirrors edge, it was amazing how much of that lighting was baked before hand (for people who dn't know, baking means to light a scene, and capture how the lights look directly into the textures). I think a vid on maps, then shaders, the post processing/render to texture is gonna be the menu , i'll reply better after work tomorrow.

Hmmm. You learn something every day! Now to show off my new knowledge and make my friends look like half-wits!

This was great! Thanks so much. The illustrations and graphics really helped a lot.

More videos like this please!

I love seeing video stuff like this. Makes me feel lazy and want to jump back into some 3ds max

Made a lot of sense until you starting talking about that Zed guy, not sure what he had to do with anything. If Z doesn't rhyme with D, G, P, T and V the whole song is ruined!

Love it! Keep at it please!

This is awesome! I was actually wondering this myself earlier today. Can't wait to see more!

I got way more excited than I should have upon seeing the welsh flag :/

Good stuff man. The more you know! *

edit - A second brick was shat upon how spookily similar you look like me with a beard when side on, I feel weird now...

@SirOptimusPrime said:

Entertaining and enlightening. My favorite.

I agree completely. Brilliant work, Robster. The graphics layered on top of the second part of the video absolutely made this a great watch. Slick work.

This is terrific, and I haven't even watched it yet!

Shadows are always the first thing I dial down or turn off in PC games, always the heaviest performance hit that I can live without.

EDIT: Obligatory rebuttal to obligatory beard scratch:

Great stuff, I assume your beard is the source of your knowledge.

Nice video, learned something new. I hate bad shadows in games, but I know it's not easy to make good shadows that is easy to render.

Can't wait until the new consoles arrives so more PC games will start using DX11 and we will get better shadows in more games. Here's an example:

This was cool. I liked it.

@Rerejo28 Thanks, making people feel stupid wasn't entirely the point of this, but hey :D

@Vinny Cheers Vinny, I'll have some more up hopefully by the end of the week, my channel is just random stuff that I feel like making, but I'll aim to do one of these for sunday every week.

@blackbird415 I wanted to make these videos non-software-specific, 3ds is something I need to get back into :)

@endaround Haha, I hadn't even considered that, I'll make sure I use ZEE next time then,. it's funny cos I do say Zee Brush (Zbrush), ah well.

@Akrid Cheers, keep watching them please! :)

@NinjaHunter i'll hopefully be covering alot of things that people are interested enough to watch the video, but yet not interested enough to go out and research themselves :)

@Freshbandito a) ~Yes on the welsh flag, it was there already and is now awol due to rugby matches et al. and b) we need proof of this doppelgangeriness

@Little_Socrates The second part was pretty difficult to get right, my computer isn't the fastest, so rendering all the stuff was tricky. I also aciddentally drowned out all the everything with Black (originally it was only toned down a bit), will be doing alot more of this stuff, for my own practice more than anything

@Brackynews I can't view that vid at the moment, when I get home I will :)

@CaptainCody yes it is, I'm like an upside down .. whatshisface, the one with the power in his hair... damnit... to the googlemobile!

@Xyber I'm an openGL man myself... The APIs are very similar, and there's some exciting stuff being down with the "programmable pipeline" using programmable shaders. tasty beans

@dantey cheers mate, I'll post up more when it's done

Good stuff dude, well explained and informative.

Keep it up.

Nice one duder. The next evolution of this technology will be Beard Maps.

@Robster: Hey, are you planing to do these only for game graphics or graphics in general?

Great video.

That was great. Wasn't expecting the use of overlaid diagrams and graphics but they really helped things.

Can the next video be on Why Everything Made Using The Unreal Engine Looks Like It Was Shat Out By The Lubricant Monster?

EDIT: And it needs to be that title. I'm not joking.

I know nothing about this, but that's why I'm watching. because I'm curious.

thanks, dude.

You're a hottie! I love Welsh men. Please post more videos so I can listen to your sexy accent, oh right and learn... about graphics... and such... :-)

I often thought of doing a tech of games series. Always too much going on, glad someone got around to it! Good job!

@Sweep said:

Can the next video be on Why Everything Made Using The Unreal Engine Looks Like It Was Shat Out By The Lubricant Monster?

EDIT: And it needs to be that title. I'm not joking.

My porn name is The Lubricant Monster

Great video! Thanks for the insight!

Cool video, and as someone who is in the video game industry what you said is mostly correct, but I'd like to mention one more crucial item about real time shadows. Resolution. Real time shadows still use the basic principles of other textures. It's an image being projected onto a surface. True there are lots of extra steps involved to figure out where to actually put the shadow, like the z-depth render you mentioned, or ray-casting, but it still comes to resolution of that shadow texture aka shadow map. The 3DS is not a powerful system, and so would be using relatively small texture maps for all the models. You take a texture, and add to the fact that it needs to be rendered in real-time, you'll need to keep the resolution even lower, otherwise performance goes down.

Now that shadow in the Quick Look there did look pretty bad, and in my opinion should have just been left out. If it looked bad enough to be called out like that, they should have just not done real-time shadows, and saved those render calls for something else.

Anyway, just my two cents. Keep up the videos man. The more people who understand how hard this stuff all is, maybe they's start giving the developers an easier time... ah who am I kidding...

I just came here to say that that is one fantastic beard. Well done sir

great video, thanks man!

Finally! One of the great mysteries (for us lowly knuckle draggers at least) of the HD generation has been solved! Thank you!

... So beard writing? What's up with that!?

@Swoxx ,@Oginam,@corganmurray,@xseedx thanks, glad you enjoyed it. like I said previous, I'm going to put more effort into organising the video a bit better next time so it should be shorter and more succinct

@selbie They already have those!... well... Hair Shaders.. that's another topic for another video

@dantey I'm concentrating on realtime stuff at the moment, so games, and also the stuff is all quite backward facing, so after I've explained some of the current trends/quirks in graphics, I may go on to be more forward facing, explaining some of the really up to date stuff

@RagingLion The diagrams and graphics and that are pretty much the only thing that makes me more than a muppet. I like describing things in different ways to different people, because by doing that, you force yourself to get the facts right (or almost right)

@Sweep the next video will cover this (slightly), but maybe it's a video we can do when I eventually come and visit you (it's been too long), I have a vat of lube here with your name on it... also I think we should further our experiments as to whether drinking helps your reflexes on games, I think our super meat boy run was a success, maybe vvvvvv next? everytime you die, you drink...Never not running

@damnable_fiend Well hopefully it described it without going into too much techy stuff, I hate that shit.

@BALE Welsh people, not just the men, as a nation, look squashed, I think it's all the pressure down the mines.

@astonish I know what you mean, hopefully things can be relatively un-busy so I can complete these vids

@TurboMan I had the same dream

@Mcubed Yeah, I know what you mean about not mentioning resolution, it was a point I tried to imply with the rendering a few times thing and the thing. This video was mostly adlibbed, which was a mistake. Hopefully the next one will have less mistakes/ommisions in. Cheers for pointing that out though :)

I also made a mistake that I put in the comments below the youtube video in that when testing the map against an object at a different angle, if you don't give it the right value of bias (how much leeway you give it), you get like a lined look, like slats on a blind. Also... thinking about it more, I forgot to say about the depth map having a small resolution the other way too, in that you can only have from 0 to 1 and that's reliant on the viewing projection and that. I MAY go into that at a different point as it's relevent to shaders and stuff too... hmmm.

@deathstriker666 why thank you. but the thing is now... where do I go from here?, It's on my todo list to get a top hat and monacle, maybe even a cape, but then... I don't know. I was actually considering trimming it down number by number... taking photos at each point, then turning it into a flash app where you can shave stuff into my face... interesting...

@DeF yeah, the beard writing thing was originally at the end and after it I originally said "that probably isn't going to happen", but as I exported it something went wrong, and as it took SO long to export and upload, I didn't bother fixing it. I want to try and get away from the beardiness in the vids to some extent

@Robster: ironically, every time I drink I die

Nice job! Very informative.

Way cool. Today I learned...

I'm in love with your eyebrows :)

Great video. If it were an Ebay seller, you'd get an A++++++++++++++ Very informative video. Would watch again! Also, I kept getting Euwan McGreggor Obi Wan vibes.

I'd love to see a video on field of vision. It's not really a graphics thing, but... sort of is?

I'm constantly trying to explain FOV to my friends and I seem to do a rubbish job.