You can see a sharp shadow near the door frame. So yeah, ambient occlusion is not going to simulate that, but it was never supposed to?
It's supposed to make 3D shapes look good without the need to place lights or do more complex ray tracing. It roughly approximates what you could perhaps see inside a photographic lightbox. Light is coming from every direction and every surface is matte.
The reason it's used in computer games is not that it's photorealistic under all lighting conditions. It's that it makes geometries look better than before with less computational overhead than the alternatives. Harsh lights often look nasty and diffuse lighting is expensive.
I agree with his overall point, but I would point out: realism is not usually the point, the point is to look good. The problem with realism is once you get past "wow the computer can do that now", it's not very interesting -- most of reality is pretty dull looking!
This is extremely untrue. It may be true that you spend most of your time in a dull-looking office or apartment, but most of reality -- at least here on Earth -- is quite visually interesting, and games are usually placed in visually interesting environments rather than set entirely within an office or apartment.
Beyond that, I would argue that the entire point of realism is not looking good (although it can look good), but rather immersion. The intent is to transport you to another world and make you feel like you're actually inside the scene. If your goal isn't to do that, then there's no need to bother with realism at all, and you can instead opt for highly stylized graphics that are intentionally designed to maximize aesthetically pleasing appearance and gameplay readability.
I think there’s a value in realism, ARC Raiders is one of my favorite games right now and it reminds of of movies like Stalker or The Shape of Water, movies that show old run down buildings and industrial complexes in their full glory, paint flaking, water in the wrong place, it’s just fantastic. It really sells me on being in a once busy space that’s been abandoned and fallen from its former glory.
I’m not on the ARC Raiders art team, nor have I even played the game. But I can say with some confidence that what people react to in computer graphics isn’t realism per se, it’s verisimilitude.
Not only look good, but also to be informative. His photos demonstrate the exact problem: Without the lighting hack you can have a 90° angle with almost no visible line, if the two surfaces are evenly lit. That's important visual information you've just left out.
As a player, I'd rather compromise realistic rendering in order to be able to see the contours of things.
I don't think this is true at all. I've been experimenting with a physical sky / atmosphere modeling, and it really seems that the most physically accurate sky is the most beautiful. I think this applies more generally to all light and shadows, and all natural effects which people perceive as beautiful, and at the same time difficult / expensive to recreate on a computer.
It's common for games to have an intentional designed aesthetic which uses cultural and social and graphical "languages" to create specific emotional impacts and effects. The lower the budget the more this is leaned into for expedient reasons but that in turn creates genre expectations! Low-poly and 8-bit are fine examples of what were once a product of technical limitations becoming their own target.
Subject matters also, and the overall aesthetic of the game; triple-A games often push towards what you're describing, that the most impactful and compelling may also be the most real...
...or at least, contain elements of it. Lots of games mess with color shading and saturation etc for the impact-reasons...
I do look forward to the era of more "photorealistic" rendering as you prefer, especially for natural landscapes like those I enjoy IRL. Some of which can have very "unnatural" feeling swings in tone and all sorts of things! I was just up in the mountains this week and saw a lot more variation in lighting and its impact on a consistent landscape (mountains across a lake) than I've ever seen in a game engine so far :)
> It's common for games to have an intentional designed aesthetic ... to create specific emotional impacts and effects.
And before games this was true of movies and before that photos and paintings. A master cinematographer can spend hours lighting a shot that will later be praised for its 'gritty realism' :-).
Tangential, but sometimes real images can also seem unreal. I remember seeing a photo of Earth and the Moon with the Moon on the foreground that looked like a bad CGI. But it was actually a photograph taken by a space probe. Or, one can think about those photos of the Apollo lunar module that looked "wrong" (and probably helped some conspiracy theories spread), but were obviously correct - just there's no atmosphere to speak of on the Moon, so no light scattering etc. That said often, space images actually use "false colors" [1]
Condensed rant: We are blind to the most common gasses on our planet (N2, O2) and sometimes have difficulty perceiving its most common liquid (H20). This is sensible, since otherwise we'd wander off cliffs in the fog and be eaten by giant carnivorous bats. We must be blind to one thing in order to detect another.
If some aliens arrived at our planet and said: "Wow, your planet is the most-boring we've ever seen, it's all just one N2-colored ball", we'd be a little offended: Clearly, they're looking at it wrong!
Flip that in reverse, and we have an artistic obligation to choose "good colors" for planets that maximizes their interesting features.
Yeah, something the article doesn't mention is that videogames have to overcompensate with things that provide the illusion of depth because the viewer is missing their usual stereoscopic vision. The game is already at a disadvantage, and players need to be able to read scenes quickly to play the game effectively.
And yet somehow we can understand videos and photos just fine.
Videogames have to overcompensate when they're missing lots of other important cues, but the closer they get to realism the less they have to overcompensate.
I think you may be underestimating how much photographers and cinematographers also compensate for this by choosing subjects and lighting carefully to re-convey a sense of depth lost by stereoscopic vision.
This is, I think one of the biggest differences between a skilled and amateur photographer. Amateur photo and video work looks flat because they are pointing the camera at a scene that looks fine to them in reality, but the resulting 2D image is flat and they don't know why or what to do about it.
I find a particularly stark example is forests. Walk into a random forest and you will find your sense of being surrounded and in an immersive, evocative, deep space to be very strong. Then fire off a quick photo and the result will be a chaotic uninspiring flat mess.
Great point. I've taken so many grey-blob photos of forests.
Any course on landscape photography will teach that landscapes should (usually) have a foreground, middle ground, and background. It seems kind of too obvious and too restrictive at the same time. But most of the time when I take a landscape photo and feel dissatisfied, it either breaks this rule or has very bad lighting.
SSAO really is awful but it’s the best performing AO we could get for a long time afaik. It’s such a hallmark of 00s and 10s rendering that I laugh every time I see it in a modern release. There are some new solutions hitting the space, I believe RTGI/PT inherently handles AO, and there is even a new method called ‘FidelityFX CACAO’ which does look considerably more realistic. Things are improving… slowly.
SSAO is cheap, doesn't work that well and looks terrible if overdone, as many games did 10-15 years ago. The simple workaround is to dial down the SSAO so that it's maybe weaker than ambient occlusion in reality, but also not noticeably wrong or overdone. Fortunately, the settings often allow that. Problem mostly solved!
It's a common pattern that newly possible (usually due to hardware) effects are overdone for the wow / up-to-date look effect.
I find this very interesting. I have implemented AO myself and understood that its physically inaccurate (it should really be radiosity presence, not ambient occlusion), but have believed the results to look realistic. Or rather; better than flatly lit, which makes geometry hard to understand.
Thank you for these photographs! It would be awesome to have a CG benchmark like this (geometry + light properties) that we could compare to. I'm sure this exists already but. Very neat!
Classic SSAO is very similar to performing an "unsharp mask" based on the depth buffer. This has been proposed "a simple and efficient method to enhance the perceptual quality of images that contain depth information" in the paper "Image Enhancement by Unsharp Masking the Depth Buffer" which came out a year after SSAO was published by Crytek.
The thing that’s always bugged me about SSAO is the “jitter” visible when the player character or camera moves, which I would imagine is due to it being calculated in screen space. In some cases it’s so distracting it breaks immersion.
Given that accuracy isn’t the goal anyway I’ve often wondered if it wouldn’t be better in many cases to apply a sort of pre-calculated AO that uses a relatively small atlas of shading textures rendered on flat surfaces as decals (disclaimer: I am not a graphics programmer). It’d apply only to angular surfaces (so for example human faces wouldn’t be included) but I’d say that 80-90% of the benefit of AO lies in things like corners of rooms anyway — it’s really just manmade objects that look bad without AO, organic shapes don’t suffer nearly as badly.
The jitter is because it's usually rendered at 50% resolution - you can render it at full framebuffer size and it doesn't jitter but it's kind of hard to justify.
That might be part of it, but the effect I’m describing includes the shading changing shape. So for example if a bust on a pillar is sitting in a corner, as the camera pans around the bust, AO behind the bust shifts to “wrap” around the edges of the bust which looks strange because no shadow in the real world acts like that.
"Baked ambient occlusion" does exist. It only works well in games where most objects are fixed. If there's a lot of flexibility in where objects are placed, then movable objects will stick out too much (they will lack the AO shading). It sounds like you're proposing using it only on static walls, which seems like it'd be totally viable.
Splinter Cell Conviction did this well, ambient occlusion got baked for pretty much all models, even dynamic. The dynamic occlusion for characters was solved by approximating them as a bunch of ellipsoids.
In 20 years of archviz, I've only encountered ambient occlusion as a shortcut to save on compute; such as real-time viewport effects or to constrain resources on large render jobs. I've never heard anyone argue that it was physically realistic, realism was always the domain of ray tracing.
I think part of this is the mach bands illusion. We perceive some darkening at the transition between shades and think we have to replicate it in rendering. But we really don't because the brain creates the effect, so artificially adding it makes it really exaggerated.
Literally nothing real looks like CG. Shadows and lighting don't look like that. Optical effects don't look like that. People don't move like they do in games and CG animations. Physics IRL doesn't work like it does in games. Real world doesn't look like filtered JPEGs stretched over wireframes with LODs.
From the photos they mostly look a bit like light SSOA. Obviously some devs overdo it, but that's not SSOA's fault. And we know SSAO isn't physically based. Of course it's not going to perfectly match reality.
I don't think the author makes a persuasive argument. Most of the photos they're analyzing are obviously lit by various point light sources, e.g.:
https://nothings.org/gamedev/ssao/IMG_0064.JPG
You can see a sharp shadow near the door frame. So yeah, ambient occlusion is not going to simulate that, but it was never supposed to?
It's supposed to make 3D shapes look good without the need to place lights or do more complex ray tracing. It roughly approximates what you could perhaps see inside a photographic lightbox. Light is coming from every direction and every surface is matte.
The reason it's used in computer games is not that it's photorealistic under all lighting conditions. It's that it makes geometries look better than before with less computational overhead than the alternatives. Harsh lights often look nasty and diffuse lighting is expensive.
one, the article is 14 years old. two, the point of ambient occlusion was not about making geometry look good, it was about making lighting look good.
The main issue it solved was normal lighting at odds with the world, it was for fast general approximation of global illumination.
I agree with his overall point, but I would point out: realism is not usually the point, the point is to look good. The problem with realism is once you get past "wow the computer can do that now", it's not very interesting -- most of reality is pretty dull looking!
> most of reality is pretty dull looking!
This is extremely untrue. It may be true that you spend most of your time in a dull-looking office or apartment, but most of reality -- at least here on Earth -- is quite visually interesting, and games are usually placed in visually interesting environments rather than set entirely within an office or apartment.
Beyond that, I would argue that the entire point of realism is not looking good (although it can look good), but rather immersion. The intent is to transport you to another world and make you feel like you're actually inside the scene. If your goal isn't to do that, then there's no need to bother with realism at all, and you can instead opt for highly stylized graphics that are intentionally designed to maximize aesthetically pleasing appearance and gameplay readability.
I think there’s a value in realism, ARC Raiders is one of my favorite games right now and it reminds of of movies like Stalker or The Shape of Water, movies that show old run down buildings and industrial complexes in their full glory, paint flaking, water in the wrong place, it’s just fantastic. It really sells me on being in a once busy space that’s been abandoned and fallen from its former glory.
I’m not on the ARC Raiders art team, nor have I even played the game. But I can say with some confidence that what people react to in computer graphics isn’t realism per se, it’s verisimilitude.
Not only look good, but also to be informative. His photos demonstrate the exact problem: Without the lighting hack you can have a 90° angle with almost no visible line, if the two surfaces are evenly lit. That's important visual information you've just left out.
As a player, I'd rather compromise realistic rendering in order to be able to see the contours of things.
I don't think this is true at all. I've been experimenting with a physical sky / atmosphere modeling, and it really seems that the most physically accurate sky is the most beautiful. I think this applies more generally to all light and shadows, and all natural effects which people perceive as beautiful, and at the same time difficult / expensive to recreate on a computer.
I would suggest that both are true!
It's common for games to have an intentional designed aesthetic which uses cultural and social and graphical "languages" to create specific emotional impacts and effects. The lower the budget the more this is leaned into for expedient reasons but that in turn creates genre expectations! Low-poly and 8-bit are fine examples of what were once a product of technical limitations becoming their own target.
Subject matters also, and the overall aesthetic of the game; triple-A games often push towards what you're describing, that the most impactful and compelling may also be the most real...
...or at least, contain elements of it. Lots of games mess with color shading and saturation etc for the impact-reasons...
I do look forward to the era of more "photorealistic" rendering as you prefer, especially for natural landscapes like those I enjoy IRL. Some of which can have very "unnatural" feeling swings in tone and all sorts of things! I was just up in the mountains this week and saw a lot more variation in lighting and its impact on a consistent landscape (mountains across a lake) than I've ever seen in a game engine so far :)
> It's common for games to have an intentional designed aesthetic ... to create specific emotional impacts and effects.
And before games this was true of movies and before that photos and paintings. A master cinematographer can spend hours lighting a shot that will later be praised for its 'gritty realism' :-).
> I think this applies more generally
I would say the sky and a private eye's dimly lit, smoke-filled 1920s office are pretty different, actually.
Tangential, but sometimes real images can also seem unreal. I remember seeing a photo of Earth and the Moon with the Moon on the foreground that looked like a bad CGI. But it was actually a photograph taken by a space probe. Or, one can think about those photos of the Apollo lunar module that looked "wrong" (and probably helped some conspiracy theories spread), but were obviously correct - just there's no atmosphere to speak of on the Moon, so no light scattering etc. That said often, space images actually use "false colors" [1]
[1] https://www.scientificamerican.com/article/are-the-colors-in...
Condensed rant: We are blind to the most common gasses on our planet (N2, O2) and sometimes have difficulty perceiving its most common liquid (H20). This is sensible, since otherwise we'd wander off cliffs in the fog and be eaten by giant carnivorous bats. We must be blind to one thing in order to detect another.
If some aliens arrived at our planet and said: "Wow, your planet is the most-boring we've ever seen, it's all just one N2-colored ball", we'd be a little offended: Clearly, they're looking at it wrong!
Flip that in reverse, and we have an artistic obligation to choose "good colors" for planets that maximizes their interesting features.
> https://nothings.org/gamedev/ssao/IMG_0059.JPG
i wonder if this is on purpose to avoid this illusion where i cant tell if im looking at a convex or concave just from three lines
Yeah, something the article doesn't mention is that videogames have to overcompensate with things that provide the illusion of depth because the viewer is missing their usual stereoscopic vision. The game is already at a disadvantage, and players need to be able to read scenes quickly to play the game effectively.
I'd also wonder about the average display equipment in use, the environment they're viewing in, and versus what our eyes are capable of.
> videogames have to overcompensate
And yet somehow we can understand videos and photos just fine.
Videogames have to overcompensate when they're missing lots of other important cues, but the closer they get to realism the less they have to overcompensate.
And that’s fine, if you can get close enough to realism at 60 fps or better. If you can’t, fake it, even if it’s technically less realistic.
I think you may be underestimating how much photographers and cinematographers also compensate for this by choosing subjects and lighting carefully to re-convey a sense of depth lost by stereoscopic vision.
This is, I think one of the biggest differences between a skilled and amateur photographer. Amateur photo and video work looks flat because they are pointing the camera at a scene that looks fine to them in reality, but the resulting 2D image is flat and they don't know why or what to do about it.
I find a particularly stark example is forests. Walk into a random forest and you will find your sense of being surrounded and in an immersive, evocative, deep space to be very strong. Then fire off a quick photo and the result will be a chaotic uninspiring flat mess.
Great point. I've taken so many grey-blob photos of forests.
Any course on landscape photography will teach that landscapes should (usually) have a foreground, middle ground, and background. It seems kind of too obvious and too restrictive at the same time. But most of the time when I take a landscape photo and feel dissatisfied, it either breaks this rule or has very bad lighting.
SSAO really is awful but it’s the best performing AO we could get for a long time afaik. It’s such a hallmark of 00s and 10s rendering that I laugh every time I see it in a modern release. There are some new solutions hitting the space, I believe RTGI/PT inherently handles AO, and there is even a new method called ‘FidelityFX CACAO’ which does look considerably more realistic. Things are improving… slowly.
SSAO is cheap, doesn't work that well and looks terrible if overdone, as many games did 10-15 years ago. The simple workaround is to dial down the SSAO so that it's maybe weaker than ambient occlusion in reality, but also not noticeably wrong or overdone. Fortunately, the settings often allow that. Problem mostly solved!
It's a common pattern that newly possible (usually due to hardware) effects are overdone for the wow / up-to-date look effect.
I find this very interesting. I have implemented AO myself and understood that its physically inaccurate (it should really be radiosity presence, not ambient occlusion), but have believed the results to look realistic. Or rather; better than flatly lit, which makes geometry hard to understand.
Thank you for these photographs! It would be awesome to have a CG benchmark like this (geometry + light properties) that we could compare to. I'm sure this exists already but. Very neat!
Classic SSAO is very similar to performing an "unsharp mask" based on the depth buffer. This has been proposed "a simple and efficient method to enhance the perceptual quality of images that contain depth information" in the paper "Image Enhancement by Unsharp Masking the Depth Buffer" which came out a year after SSAO was published by Crytek.
https://www.uni-konstanz.de/mmsp/pubsys/publishedFiles/LuCoD...
For a benchmark you can just use a "vfx/cinema grade" path-tracing renderer of the same game scene?
The thing that’s always bugged me about SSAO is the “jitter” visible when the player character or camera moves, which I would imagine is due to it being calculated in screen space. In some cases it’s so distracting it breaks immersion.
Given that accuracy isn’t the goal anyway I’ve often wondered if it wouldn’t be better in many cases to apply a sort of pre-calculated AO that uses a relatively small atlas of shading textures rendered on flat surfaces as decals (disclaimer: I am not a graphics programmer). It’d apply only to angular surfaces (so for example human faces wouldn’t be included) but I’d say that 80-90% of the benefit of AO lies in things like corners of rooms anyway — it’s really just manmade objects that look bad without AO, organic shapes don’t suffer nearly as badly.
The jitter is because it's usually rendered at 50% resolution - you can render it at full framebuffer size and it doesn't jitter but it's kind of hard to justify.
That might be part of it, but the effect I’m describing includes the shading changing shape. So for example if a bust on a pillar is sitting in a corner, as the camera pans around the bust, AO behind the bust shifts to “wrap” around the edges of the bust which looks strange because no shadow in the real world acts like that.
"Baked ambient occlusion" does exist. It only works well in games where most objects are fixed. If there's a lot of flexibility in where objects are placed, then movable objects will stick out too much (they will lack the AO shading). It sounds like you're proposing using it only on static walls, which seems like it'd be totally viable.
Splinter Cell Conviction did this well, ambient occlusion got baked for pretty much all models, even dynamic. The dynamic occlusion for characters was solved by approximating them as a bunch of ellipsoids.
https://www.gdcvault.com/play/1012326/The-Rendering-Tools-an...
Ambient occlusion is not only about realistic indirect lighting, but also about giving objects more depth. Great example here: https://developer.playcanvas.com/user-manual/graphics/lighti...
Also, looking just at inner corners of the room is too specific, SSAO affects creases everywhere, not just at the inner corners of the box.
In 20 years of archviz, I've only encountered ambient occlusion as a shortcut to save on compute; such as real-time viewport effects or to constrain resources on large render jobs. I've never heard anyone argue that it was physically realistic, realism was always the domain of ray tracing.
I think part of this is the mach bands illusion. We perceive some darkening at the transition between shades and think we have to replicate it in rendering. But we really don't because the brain creates the effect, so artificially adding it makes it really exaggerated.
But corners do look better in video games like that.
Some Captain Obvious stuff.
Literally nothing real looks like CG. Shadows and lighting don't look like that. Optical effects don't look like that. People don't move like they do in games and CG animations. Physics IRL doesn't work like it does in games. Real world doesn't look like filtered JPEGs stretched over wireframes with LODs.
From the photos they mostly look a bit like light SSOA. Obviously some devs overdo it, but that's not SSOA's fault. And we know SSAO isn't physically based. Of course it's not going to perfectly match reality.
Author took his example photos in the backrooms lol
Regardless of how right or wrong this is, I woulda walked around with a vacuum first to get all those cobwebs before trying to make my point.
I'm far from a neatfreak but if those are there can you imagine what the inside of bro's computer looks like? Poor thing is probably so hot