Lately I've been playing around with PhysicalBasedRendering (yes, I'm late to the party). It took me a while to get it working because I kept getting weird results that looked more like caustics than surface roughness. I checked the math again and again, looked up as many source materials I could find to compare them to my implementation (just in case there where some errors in some articles), but couldn't see what could cause these weird artifacts. The thing is, when I rendered the light on a sphere it looked perfect, but when I rendered it from the inside of a box it looked completely wrong! So I started to check all my inputs and they all seemed fine ... until I discovered that my lighting position was inverted. Wut? How the hell? It turned out that the wrong light position caused my half way vector to be, well, something else entitely. And that combination made it all look plausible most of the time and completely threw me off course.
I'm currently using the GGX microfacet distribution, from the Microfacet Models for Refraction through Rough Surfaces paper. Here's what it looks like with some random textures and lightwave models borrowed from Doom 3 (don't sue me!) for testing purposes.
Some random thoughts about gloss / roughness maps & normal maps
While I was working on all this I realized that while roughness maps describe surface detail at the micro level, normal maps describe surface detail at the macro level. This made me think that, when generating normal map mip-maps, that lost normal details at lower resolution could be put in the gloss map instead. Of course this probably would work better with an anisotropic BRDF (directional-roughness) compared to an isotropic BRDF (uniform-roughness). Since both of them seem to be connected, it may imply that there might be a way to combine the two in a single thing. ("anisotropic normal maps"? lol)
This would probably work a lot better (especially with isotropic BRDF's) if you generate normal your maps from heightmaps, and create the gloss and normal maps mip maps from the mip-maps of the heightfield. The roughness would be calculated from all the height variation within a lower resolution mip pixel (compared to the highest resolution mip), while the normal would simply be calculated from the lower resolution mip directly. Maybe some sort of min-max mipmap approach could be used for the roughness. Of course you'd still have to take into account that the further away you are from a gloss map, the smoother it'll look. (disclaimer; I haven't actually tried any of this)
The last couple of months I was entangled in some stuff that I can't really talk about, so I didn't have time to post on my blog, or work on any of my tech experiments. (Not to mention that so many interesting tech stuff happened in those months, Rage was released, Doom 3 source code was released etc.) However now that's all behind me, it's now my intention to blog more regularly again now that I can.
In the meantime, here are some things that I've been messing around with lately.
I created a simple Minecraft renderer, which unfortunately slowed down to a crawl due to the recording software but which is silky smooth without it. (I really need to upgrade my aging machine!!) The light is simply interpolated between the tiles which creates an interesting GI kind of effect. It doesn't correctly take occlusion into account, so that's an area of improvement. Also it's very close to brute force at the moment, with chunks only frustum culling (and of course back-face culling) being used right now to decrease the number of polygons. Transparent blocks are sorted on a chunk level, but are currently incorrectly sorted within the center chunk. The only way to do that is to split the center chunk into 8 pieces and render those separately. In fact, all chunks that lie in a straight line with respect to the center chunk would need splitting. That said, I'm not so sure if I'm going to spend much more time on this though.
Of course like everybody else and their cat, I created my own Doom 3 source code branch, with GLSL shader support, where I'm busy replacing the Doom 3 lighting model. Right now it uses a physical correct lighting model, although the lighting isn't yet properly normalized. I also wrote a small C# library to load LWO files, which is still very rough in it's current form, but my intention is to improve on it as time goes by (any help improving this lib is appreciated!)
I had thought about a deferred rendering / virtual texturing approach before where instead of storing the normal, albedo etc. in a g-buffer you'd only store the texture coordinate + texture derivatives (+ z position & surface normal, unless you're storing normals in worldspace which is possible for static environments with virtual texturing). But I quickly dropped that idea back then because you'd store roughly the same amount of data so it wouldn't gain you much. But this morning I was thinking... If you have a lot of overdraw and/or if you render lots of small triangles (especially if a lot of them are smaller than 2x2 pixels) it might pay to have a really simple shader at the g-buffer phase, and then do the actual texture lookups later on, when you're just rendering a single full-screen triangle. It might work, it might make no difference, it might be worse. I should definitely try it some day.
So I saw the QuakeCon 2011 keynote by John Carmack..
... and I figured I'd jot down some notes:
80 degree field of view in Rage, pages might be loaded for entire 360 degree view?
Loading full pages that occupied 1 pixel on screen hurt a lot in Rage.
Single 4k x 4k texture wasn't enough.
Ended up using multiple 4kx4k textures on consoles.
Second buffer of cache, uses just about all memory on console, shared with audio.
Transcode pipeline comes after cache, so pages are not cached uncompressed.
Blu-ray is worse in latency than DVD.
Pages are cached on HD on consoles.
Carmack said better solution would've been split page cache in low-detail and high-detail pages.
1 thread reading from DVD
1 thread reading from HD
"additional threads (plural!) doing analysis of the feedback buffer"
"a bunch of threads doing transcoding"
HD-photo derivative compression, not using dct based compression.
30% better compression, requires twice the processing power.
Rage is running on intel hardware?!? (but not at 60hz)
Rage blurry close up, newer titles have more detail up close.
Biggest levels are 128kx128k, dynamic characters are 64kx64k.
Wasteland are 14 pieces, various sizes from 32kx32k to 128kx128k.
He mentions that "the wasteland doesn't fit into one 256 x 256",
Did he mean 256k x 256k?
Considering he specifically said that number I'm assuming that's some sort of maximum.
256k / 128 = 2k. GPU side page texture is 4k x 4k.
Are they using more page-tables?
Are they swapping page-tables (pieces) in and out according to the area/level you're in?
or maybe it's just not a hard limit, but rather a soft limit.
About 300 - 500mb per 'level'.
(My Q4 level conversion is 1.5gb, DXT compressed)
Source data giant 256000*256000 texture
64 gigatexels of source data.
Terrabyte of source art.
100 gigabytes of texture data (uncompressed) used in Rage.
Large ammount of profiling, removing pieces that aren't visible (roofs etc.)
(Clearly works better in outdoor levels, compared to my experiments with Q4)
This made me think of .. (not mentioned by carmack)
OnLive/Gaikai
The advantage of making a game purely for OnLive/Gakai is that the whole game runs on their servers and the images & audio are send to the player over the network, the player only sends their controller inputs. Obvious latency issues aside, which can be helped somewhat by putting servers all over the world, this has a lot of advantages for publishers and developers. Piracy and cheating being impossible being one example, not being constrained by disk space or hardware is another.
A game like rage could run with full detail, maybe even with dynamic lighting, if it had a good server using an OnLive/Gakai approach. In fact, gameplay could be run on separate severs compared to the actual rendering, which could be done in a more render-farm sort of way.
The render servers could hold a large chunk of the texture data in memory and keep the rest on SSD. Oddly enough multi-player / MMO's become more attractive than single player games because if all players play in the same world lighting can be cached into the pages for all players, allowing the render farm to separate rendering lighting from rendering camera views. The same goes for anything else that's shared between the players, like physics etc.
Scary to think about scaling this up to world of warcraft like sizes though, from a technical POV.
So I had a little bit of time between things to try out discovering which of my pages are potentially visible at any given time when moving around in my test scene (imported from quake 4).
At first I was amazed, my 1.5gb virtual texture file had reduced to about 50mb! I wanted to tell the world!
But no, that was a bug. Crap.
(Good thing I double checked!!)
When I got it working properly, it only removed 9 pages :(
It took about 5 minutes to process the entire Quake 4 level, 2 minutes to create the virtual texture file.
That said, in my current implementation I'm reusing the pages across surfaces, although they're still unique on the GPU side of things (so decals can still be added in the virtual texture), so it might be a bigger improvement when all pages are unique. Besides that I used the gathered information to sort all the pages according to locality and it helped reduce latency when loading the pages.
Considering that I'm reusing pages (hell, even the texture coordinates are created carefully as to maximize reusing the pages) I'm actually surprised that I only managed to get rid of 9 pages. I guess a lot of the 'reusing the pages' idea gets wasted when pages are combined into higher MIPs, which automatically makes them less unique. It would be better to store the lower resolution pages, that make up the lower resolution MIPs, separately. That way the virtual texture file could never be larger than the combined source textures. It would also allow more fine sorting (of sub-pages) and allow us to downscale higher resolution pages into a MIP, and hopefully removing the necessity to loading some pages. But it all gets awfully complicated real quick.
And I'm not sure how much that would help with true unique texturing.
The quake 4 levels are really worst cases for virtual texturing considering that the texture placement is made to minimize the number of textures in the level, not so much minimizing texture area and not worrying about the source of the texture. So many tiny pieces of geometry are rendered with unique pages, which could all be combined into single surfaces. So I really should try to either combine all the surfaces and re-project the textures onto them, or create geometry from scratch with a more virtual texture friendly texture layout.
For some strange reason I was blocked on all projects I was working on, so I had a little time to work on something else. I've encountered many situations where a simple graph editor would've made my life a lot easier, yet I couldn't find any good and simple libraries to help me out. So I figured I'd make a simple graph library myself and make it open source to (hopefully) help whomever who needs similar tech in the future.
There are still a couple of things I want to add to this such as support for different colors of nodes/connections and maybe drop-down thingies. Anyone is welcome to contribute. Anything is open for discussion. I should mention, however, that I'm not interested in making this a super uber-leet library that includes everything including the kitchen sink. I'd rather keep it small, simple & flexible.
