Since my last post I've been working on cleaning up my tool set and breaking it up into several different pieces instead of one huge monolithic tool which did all the processing.
I really needed to do this because my processing time jumped up from 10 seconds per level to more than 20 minutes, which is clearly unacceptable.
Splitting it up into multiple pieces allows me to skip the processing steps that go before whichever part I'm modifying.
It also allows me to cache and pre process some content, such as the textures, for easy processing.
I'm not completely done with this yet, but hopefully it won't take much longer.
Other than that I've implemented my own disk cache code, and what a difference it makes!
Right now you wouldn't be able to tell if the level was textured normally or virtual textured, other than if you pass through a wall you have one or two frames of lower resolution pages.
I've pre cached all the smallest MIP-levels, so they're always in memory, this is to ensure that no matter what you always have at least a lower resolution version of your page in memory.
These cached pages cannot be unloaded as well.
All in all right now I'm allocating a whopping 500 Mb of page cache memory, but this should decrease a lot when I put the new texture compression code in which I mentioned in my previous posts, the idea is to store compressed textures in the cache and decompress them on demand.
After a while, hiccups appear in the application, and I'm not entirely sure what's causing them.
It might have to do with certain areas in the map which have lots and lots of different pages visible, which can probably only be fixed by improving my tool set.
I've also noticed that the TextPrinter I'm using from OpenTK has some pretty horrendous memory leaks, which makes my memory consumption grow rather rapidly.
I'm currently using an older version though, so this might already been fixed.