Adventures in virtual texture space, part 16

The last couple of days I've been cleaning up some parts of my code and preparing for when I can add variable sized compressed pages, which may or may not include normal and specular maps.

Unfortunately my code wasn't build with that in mind, and now it'll take some serious refactoring work to make my mip generating code work with it.

What really annoys me is that if only one tiny sub-page in a higher mip page has a normal map, then the entire mip page will need a normal map too.
Sure, it'll compress nicely, but I can imagine that this will complicate rendering, that you may or may not have a normal map at a texel basis.
I suppose this will have to be handled either at a per mesh level, or just give everything a default normal map.
I'm probably going for option no. 2 because it'll be safer and simpler.

Besides, I need to reserve the texture for it anyway.

Also tweaked my algorithm to combine polygons a bit, and I'm now using fewer pages for the same geometry.

Adventures in virtual texture space, part 15

For the last couple of weeks I've been working at a client in Utrecht, a city in the middle of the Netherlands, but I live in Capelle aan den Ijssel, which is a small city which has grown into the much larger city of Rotterdam.
The problem is that traffic between those two city is horrible.
If there wasn't any traffic it would take me 45 minutes, but it actually takes me 2 to 3 hours.
Well, it would take me 2 to 3 hours if I didn't get up at 6, eat my breakfast in the car, and drive like mad to avoid the rush!
So for the last couple of weeks I've been in a constant state of tiredness.

Anyway.
Since my last update I've written a idtech-4 shader parser so I can get more correct textures.
Before I would just take the name of the shader and let the importer kinda guess which textures it would need.
Fortunately most of the time the shaders and their textures had predictable names, but not always.
Parsing the shaders also allows me to get back the normal and specular maps, which should be useful when I start to implement lighting.

Other than that I downloaded the beta of Visual Studio 2010, and man, those profiling tools rock!
I haven't tried them on C++ code yet, so I'm not sure how or if they're any good there, but for managed code they're awesome!
Can't wait for it to come out of beta.

I also optimized my code a bit, and now it oscillates between 3-5ms per frame (300-200fps).
It takes about 2ms for the whole readback part, which is more than I would like.
(And I'm pretty sure this could be done a lot faster in C/C++ where you can do all kinds of smart pointer tricks etc.)
This makes me think that the analytical solution that I've been thinking about could be a lot faster.

Other than that it also takes about 2ms to render everything, but I expected that since I'm not doing anything remotely smart, at the moment, when it comes to rendering geometry.. I'm basically just drawing the entire level in one draw call right now.
The fact that the vertex count is way above 200k certainly doesn't help ;)

Adventures in virtual texture space, part 14

This is just an update to show my progress.

Texture coordinates are now generated correctly, and popping of pages is extremely rare now and basically only happens if you travel trough a wall.

Note that I haven't put the texture compression code in yet, so IO performance will actually get better in the future.

Also, the frame rate in the video is rather chaotic, this is due to the screen recorder competing with the application for resources.

Adventures in virtual texture space, part 13

Since my last post I've been working on cleaning up my tool set and breaking it up into several different pieces instead of one huge monolithic tool which did all the processing.
I really needed to do this because my processing time jumped up from 10 seconds per level to more than 20 minutes, which is clearly unacceptable.

Splitting it up into multiple pieces allows me to skip the processing steps that go before whichever part I'm modifying.
It also allows me to cache and pre process some content, such as the textures, for easy processing.
I'm not completely done with this yet, but hopefully it won't take much longer.

Other than that I've implemented my own disk cache code, and what a difference it makes!
Right now you wouldn't be able to tell if the level was textured normally or virtual textured, other than if you pass through a wall you have one or two frames of lower resolution pages.

I've pre cached all the smallest MIP-levels, so they're always in memory, this is to ensure that no matter what you always have at least a lower resolution version of your page in memory.
These cached pages cannot be unloaded as well.

All in all right now I'm allocating a whopping 500 Mb of page cache memory, but this should decrease a lot when I put the new texture compression code in which I mentioned in my previous posts, the idea is to store compressed textures in the cache and decompress them on demand.

After a while, hiccups appear in the application, and I'm not entirely sure what's causing them.
It might have to do with certain areas in the map which have lots and lots of different pages visible, which can probably only be fixed by improving my tool set.

I've also noticed that the TextPrinter I'm using from OpenTK has some pretty horrendous memory leaks, which makes my memory consumption grow rather rapidly.
I'm currently using an older version though, so this might already been fixed.