Friday, May 28, 2010

Realtime CSG - Optimizations

The last time I blogged about realtime CSG, I mentioned that in my test scene if I performed CSG on the entire scene it would take about 80ms.
(And I should mention again that this algorithm is intended for fast updates, not necessarily doing this faster for the entire CSG tree. Even though that would be a nice bonus)

I also mentioned a couple of things that could be done to improve performance, and that there are a lot of ways to improve performance.

Today I had a simple idea to improve performance.
By checking the bounds for each left and right branch separately at a higher level, so that we know what operation we're performing, we can handle some common situations in a more simplified, and faster, way.

Here's the updated pseudo code for the CSGBranch.Categorize method.

void CSGBranch.
       Categorize( categorizationNode,
                   inputPolygons,
                   ..., 
                   inside, 
                   touchInside, 
                   touchOutside, 
                   outside)
{
  switch (Operator)
  {
    case CSGOperator.Addition:
    {
      // Check if the given polygons
      // are possibly touching the 
      // left branch
      if (boundsOfPolygons
            .IsOutside(Left.Bounds))
      {
        // Polygons are not touching 
        // the left branch.

        // Check if the given polygons
        // are possibly touching the
        // right branch
        if (boundsOfPolygons
              .IsOutside(Right.Bounds))
        {
          // Nope, all of them are
          // categorized as 'outside'
          outside.AddRange(
                    inputPolygons);
        } else
        {
          // We only need to check
          // with the right branch
          Right.
            Categorize(...);
        }
      } else
      // Polygons are touching 
      // the left branch.
      // Check if the given polygons
      // are possibly touching the
      // right branch
      if (boundsOfPolygons
            .IsOutside(Right.Bounds))
      {
        // We only need to check
        // with the left branch
        Left.
          Categorize(...);
      } else
      {
        // Polygons are touching
        // both branches 

        // same as before ...
        // (Left || Right)
        LogicalOr(categorizationNode,
                  inputPolygons,
                  ...,  
                  inside, touchInside, 
                  touchOutside, outside, 

                  false,
                  false
                  );
      }
      break;
    }
    case CSGOperator.Common:
    {
      // Check if polygons are outside
      // either left or right branch 
      if (boundsOfPolygons
            .IsOutside(Left.Bounds) ||
          boundsOfPolygons
              .IsOutside(Right.Bounds))
      {
        // Considering that in a
        // 'common' operation we need
        // to find the parts that are
        // shared with both branches, 
        // we know that if the polygons
        // are outside just one branch 
        // that they are categorized as
        // 'outside'
        outside.AddRange(
                  inputPolygons);
      } else
      {
        // Polygons are touching
        // both branches 

        // same as before ...
        // !(!Left || !Right)
        LogicalOr(categorizationNode,
                  inputPolygons,
                  ...,  
                  outside, touchOutside, 
                  touchInside, inside, 
                
                  true, // reverse Left
                  true  // reverse Right
                  );
      }
      break;
    }
    case CSGOperator.Subtraction:
    {
      // Check if the polygons are
      // touching the left branch
      if (boundsOfPolygons
            .IsOutside(Left.Bounds))
      {
        // The right branch removes
        // and the left branch keeps
        // whatever is inside it.
        // If all the polygons are
        // outside the left branch
        // then we know they're all
        // categorized as 'outside'
        // this branch.
        outside.AddRange(
                  inputPolygons);
      } else
      // Polygons are touching 
      // the left branch.
      // Check if the given polygons
      // are possibly touching the
      // right branch
      if (boundsOfPolygons
              .IsOutside(Right.Bounds))
      {
        // If we're only touching the 
        // left branch, we don't need
        // to do a more complicated
        // operation, and just
        // categorize the left branch.
        Left.
          Categorize(...); 
      } else
      {
        // Polygons are touching
        // both branches 

        // same as before ...
        // !(!Left || Right)
        LogicalOr(categorizationNode,
                  inputPolygons,
                  ...,  
                  outside, touchOutside, 
                  touchInside, inside, 

                  true, // reverse Left
                  false
                  );
      }
      break;
    }
  }
}

This alone increased the speed from about 80ms to 25-30ms for the entire test scene.

I'm going to use a much bigger scene in the future,
because it really is too small to properly test for scalability.

Wednesday, May 26, 2010

MD3View - A short trip down nostalgia lane

A long time ago, about 10 years, Quake 3 Arena was released.
Within 24 hours Matthew Baranowski and I had reverse engineered the MD3 model format and created a model viewer that could display all the models in all it's Q3A shaded glory.

Well actually it could've been 24 hours after Q3Test, but anyway..

We named it MD3View and released it with the source, and because open source obviously equals GPL we slapped the GPL license on top of it. (at least that's we thought at the time, not that we gave it much thought)

Interestingly enough Raven software found our tool, expanded on it and used it on games such as Star Trek: Voyager - Elite force (We both got a special thanks in the credits because of this) and apparently Star Wars: Jedi Knight - Jedi Academy.

Not long after Matthew, me and Volker Schoenefeld went on an awesome road trip, starting at New York (well technically Volker & I flew to NY from Germany and the Netherlands) and we (well Matt) drove all the way to Siggraph 2000 in New Orleans (which was still in one piece at the time) and finally to Quakecon 2000.

I still remember being nervous as hell talking to a pretty cool Raven guy there, I mean there I was, this young geeky guy actually talking to someone from Raven Software!!
I politely told him, well mumbled, that MD3View was released under the GPL and that the modified source code should be released as well, or something or other.
Some time later we received an NDA through email, and then got a copy of the source code... and unfortunately my memory gets rather fuzzy beyond that.
But for no particular reason, certainly not because of Raven, the modified source code never got released.

I would've never given this any more thought if someone wouldn't, out of the blue, just asked me if I still had the source code.

Which I didn't.
I just couldn't find it.
Not the original (can anyone out there help me with that?) or the version from Raven.

So I figured, what the hell, I'll just politely ask Raven software for it, it's not like it's state of the art technology any more anyway.

Within hours I got an email back from Ste Cork:
Wow, you won't believe how hard it was to find this, 
we stopped using SourceSafe years ago (and that was
the only place this was ever stored), and it took a
while to even install the client (doesn't work on
Vista) and find all the databases and various INI 
files etc, then pick out the one this was in. 
Anyway, as you can see from the attachment our
IT guy finally managed it.
Whoops, sorry to put your IT guy through that! (so sorry!)
But thanks a lot for all the effort!

And after a couple of emails back and forth, making sure that the headers contained the proper legal information and converting it to a more modern version of visual studio: here it is.
Update: Apparently the original hosted file disappeared, so I put it on Codeplex this time, you can find it here.

It's old, it's outdated, and it's yours, all yours.

Update: Matthew confirmed it, 24 hours after Q3Test, not Quake 3 Arena.
Update 2: He also said that we got a cease and desist from Id software and that we had to take it down, wait a couple of weeks until Quake 3 Arena was released before we could put it up again.
I honestly can't remember that... is that a grey hair?
Update 3: Wow, this article brought almost 5000 visitors to my blog in a day!?

Friday, May 21, 2010

Realtime CSG - Part 5

In part 1 I explained the basics you'll need to know to understand how to perform real-time CSG.
In part 2 I described how to build a brush, which is the basic building block of the algorithm I'm describing.
In part 3 I wrote about CSG operations.
In part 4 I wrote about cutting a brush with another brush.

When I started on writing about real time CSG, I had my original real time CSG algorithm in mind.
It's a simple enough algorithm, but a very simplistic subset of CSG.
I foolishly thought I could just figure our how to generalize it as I went along, it turned out to be a little bit harder than that.

Eventually I managed to get an algorithm to work, but it was complicated and un-intuitive.
The additive and common operations seemed coherent all right, I could rationalize every piece of it.
But the subtractive operation, my god, it was frankenstein as code..
It just didn't make any sense!

I managed to make it work, but only through trial and error.
And even though it seemed to work in all my tests, I didn't really trust it.
It definitely didn't feel right to unleash that piece of code into the wild, imagine what kind of damage that could do in the wrong hands!

And then, as I blogged about before, I read a post at filmic games and it hit me; not only can CSG operations be seen as logical operations (which makes it so much easier to express), but the subtractive operation is actually a combination of operations!
That's why it was so hard to get it to work coherently, I was trying to doing multiple things at the same time!

After that figuring out the lean and mean version of my previous algorithm was fairly easy.

Categorization

As I mentioned before, the final solution of the entire CSG tree consists of all the polygons of all the brushes MINUS all the pieces of these polygons that we reject.

We don't add any new polygons, at all.
Ever.
We just remove parts.

So the heart of this CSG algorithm is the categorization code, this code takes the polygons of a brush, goes through the CSG tree and categorizes each piece of the polygon as being inside, outside, or touching the shape that the entire CSG tree (or sub-tree) represents.
Well actually we have two types of "touch" categories, inside-touching (plane aligned) and outside-touching (aligned with the reverse of the plane).

We essentially do this for every left / right branch and then swizzle the categorization depending on the CSG operation and the results from the left and right branch.

First the code for the branches in the tree (pseudo code)
void CSGBranch.
       Categorize( categorizationNode,
                   inputPolygons,
                   ..., 
                   inside, 
                   touchInside, 
                   touchOutside, 
                   outside)
{
  // Early out, check if the bounding box of
  // the categorizingBrush intersects with 
  // the bounding box of this branch. 
  // If it doesn't, don't bother going any further
  // (This effectively works as a, somewhat
  // unbalanced, bounding volume hierarchy)
  if (boundsOfPolygons
        .IsOutside(this.Bounds))
  {
    // ... add all input polygons to outside
    outside.AddRange(inputPolygons);
    return;
  }

  switch (Operator)
  {
    case CSGOperator.Addition:
    {
      // (Left || Right)
      LogicalOr(categorizationNode,
                inputPolygons,
                ...,  
                inside, touchInside, 
                touchOutside, outside, 

                false,
                false
                );
      break;
    }
    case CSGOperator.Common:
    {
      // !(!Left || !Right)
      LogicalOr(categorizationNode,
                inputPolygons,
                ...,  
                outside, touchOutside, 
                touchInside, inside, 
                
                true, // reverse Left
                true  // reverse Right
                );
      break;
    }
    case CSGOperator.Subtraction:
    {
      // !(!Left || Right)
      LogicalOr(categorizationNode,
                inputPolygons,
                ...,  
                outside, touchOutside, 
                touchInside, inside, 

                true, // reverse Left
                false
                );
      break;
    }
  }
}
Notice how elegant it is?
The LogicalOr method is basically the CSG Addition operator.
The inside, touchInside, touchOutside, outside parameters are lists to which the polygon pieces are added.
Notice how the 'not operator' is basically nothing more but a reversal of the parameters!
The last two parameters are used to tell the LogicalOr method if the left and/or right branch need to have their parameters reversed.

So here's the LogicalOr method:
void CSGBranch.
       LogicalOr( categorizationNode,
                  inputPolygons,
                  ..., 
                  inside, 
                  leftTouchInside, 
                  leftTouchOutside, 
                  leftOutside)
{
  //        right
  //        inside | touch-I | touch-O | outside
  // left          |         |         |
  // inside    I   |    I    |    I    |    I
  // touch-I   I   |   tI    |    I    |   tI
  // touch-O   I   |    I    |   tO    |   tO
  // outside   I   |   tI    |   tO    |    O

  ... create some temp lists ...

  // if anything is inside the left branch, 
  // we don't need to check it again for 
  // the right branch. Everything else is put
  // in temporary lists whose contents we'll
  // check against the right branch ...
  if (!inverseLeft)
    Left.
      Categorize(categorizationNode,
                 inputPolygons,
                 ..., 
                 inside, leftTouchInside, 
                 leftTouchOutside, leftOutside);
  else    
     // .. same as above but with parameters
     //    reversed

  if (!inverseRight)
  {
    //        right
    //        inside | touch-I | touch-O | outside
    // left          |         |         |
    // touch-I   I   |   tI    |    I    |   tI
    Right.
      Categorize(categorizationNode,
                 leftTouchInside,
                 ..., 
                 inside, touchInside, 
                 inside, touchInside);

    //        right
    //        inside | touch-I | touch-O | outside
    // left          |         |         |
    // touch-O   I   |    I    |   tO    |   tO
    Right.
      Categorize(categorizationNode,
                 leftTouchOutside,
                 ...,  
                 inside, inside,
                 touchOutside, touchOutside);

    //        right
    //        inside | touch-I | touch-O | outside
    // left          |         |         |
    // outside   I   |   tI    |   tO    |    O
    Right.
      Categorize(categorizationNode,
                 leftOutside,
                 ..., 
                 inside, touchInside, 
                 touchOutside, outside);
  } else
  {
     // .. same as above but with parameters
     //    reversed
  }
}

Note that if a polygon is touching-inside on one branch, and touching-outside on the other branch, then it means the two branches are touching each other there.
Which means that any polygon in that area is essentially inside both, and therefore 'inside'.


And here's the code that handles the leafs in the tree; the brushes themselves:
void CSGBrush.
       Categorize( categorizationNode,
                   inputPolygons,
                   ..., 
                   inside, 
                   touchInside, 
                   touchOutside, 
                   outside)
{
  // Early out, check if the polygons we're 
  // processing belong to the same brush as
  // we're currently checking against
  if (categorizingBrush == this)
  {
    // We're looking for the parts of 
    // 'categorizingBrush' that are visible
    // and at this position in the tree, these
    // polygons are definitely visible ...
    foreach (var polygon in inputPolygons)
      polygon.Visible = true;
    
    // We know all polygons are interesecting
    // with the brush they lie on
    touchInside.AddRange(inputPolygons);
    return;
  }

  // Early out, check if the bounding box of
  // the categorizingBrush intersects with 
  // the bounding box of this brush. 
  // If it doesn't, don't bother going any further
  if (boundsOfPolygons
        .IsOutside(this.Bounds))
  {
    // ... add all input polygons to outside
    outside.AddRange(inputPolygons);
    return;
  }

  ... create some temp lists ...

  this.Split(inputPolygons,
             ...,      
             inside,
             tempTouchingInside,
             tempTouchingOutside,
             outside);

  // We know that the current brush
  // is not the 'categorizingBrush'.
  // We also know that any polygon liying
  // on the surface of this brush cannot 
  // be lying on the surface of 
  // 'categorizingBrush' (well okay, it
  // can, but one overrides the other 
  // depending on order, which is exactly
  // what we want)
  // So we set all the polygons lying on 
  // the surface of this brush to invisible.
  // This solves overlapping polygon problems
  foreach (var polygon in tempTouchingInside)
    polygon.Visible = false;
  foreach (var polygon in tempTouchingOutside)
    polygon.Visible = false;

  touchInside.AddRange(tempTouchingInside);
  touchOutside.AddRange(tempTouchingOutside);
}

And finally here's what we do at the top level:
void CSGTree.
       ProcessBrush(categorizationNode,
                    inputPolygons)
{
  .. create temporary lists ...
 
  // Categorize the inputPolygons
  // depending on their location in 
  // the tree ...
  RootNode.
    Categorize(categorizationNode,
               inputPolygons,
               ...,
               // Store results in
               // temporary lists ..
               invisiblePolygons,
               visiblePolygons,
               reversedPolygons,
               invisiblePolygons);

  // We set all polygons that are outside
  // or inside the tree as being invisible ...
  foreach (var polygon in invisiblePolygons)
    polygon.Visible = false;

  // We reverse the order of all the polygons
  // that the tree categorized as having a
  // reversed orientation ...
  foreach (var polygon in reversedPolygons)
    ReverseVertexOrder(polygon);
}

And that's it!
After you've build all the geometry, when a brush moves, simply reprocess it and all the brushes it touches.


The result:
Points are vertices moved towards the center of the polygon
to make it easier to see where there are any t-junctions,
or where multiple vertices lie on the same line


Note that I didn't describe the CSGBrush.Split method, I'm going to retroactively modify the previous posts to add more pseudo code (it belongs there), and update the repository as well.
(but not today)
I'll post about it when I've done that.

Limitations and Future work

When I use this algorithm on my test level which has 234 brushes and 467 nodes, I can generate a mesh with it within about 80ms.

Keep in mind that this algorithm was designed with dynamically updating a handful of brushes in mind, not so much with updating everything all the time.

The resulting mesh has not been optimized yet, there are T-junctions and polygons that should be joined together, but that's a different topic and really deserves a series on it's own.
If the polygons are optimized per brush, T-junctions should be a rarity considering that the cutting plane that split an edge, would've cut any aligned edges on another brush as well.
(Of course, there might still be T-junctions because of floating point errors, so don't completely rely on this)


As for performance, there is much room for improvement here:
  • All brushes are processed independently, which makes them a prime candidate for parallelization.
    (This is what I did in my editor)
  • This is the big elephant in the room; The algorithm should be rewritten with CPU cache usage in mind, this alone could make it an order of a magnitude faster. Half edges are the biggest problem here. The code would become more complex.
  • A higher level bounding volume hierarchy, hashed grid, or sweep & prune phase could theoretically speed things up. Although when I tried it, it only slowed things down.
    I'm guessing that this is because we're already doing some sort of (unbalanced) hierarchical culling while going through the CSG-tree, so perhaps there's not too much to be gained. Perhaps this only becomes a problem when the tree grows very large.
  • I'm convinced that it should be possible to figure out some short cuts when going through the tree. Perhaps it's even possible to work from the bottom-up, by starting at the leafs.
    Perhaps that would allow getting rid of all the temporary lists, which is not so much a problem in a language such as C# where allocation is very cheap, but a bigger problem in languages such as C++.
  • There are a lot of lower level optimizations that can help a lot with performance too.
    I'm using a lot of methods and enumerations in the example code for clarity and readability, and these hurt performance.
    Once you understand the code, you should consider copying all the planar/vector methods code directly into the methods where they're used.
    Also, you should consider not converting planar side calculations into enums, but using the floating point values instead.
    This should help a lot in the inner loops and could seriously improve performance.

That's it for now, let me know if I need to explain something in more detail or if you use or improve on my algorithm!

Tuesday, May 18, 2010

Andreas Neu's virtual texturing thesis online

A while ago I mentioned a very interesting thesis written by Andreas Neu, but back then it wasn't online yet.

Today Andreas send me an email telling me that it's online now, and you can find it here.

Be sure to check it out if you're in any way interested in virtual texturing technology!

Friday, May 14, 2010

CSG operations update

In my last post I mentioned that I suddenly realized that all popular CSG operations (Addition, Subtraction & common) can be expressed as logical operations.
Using that knowledge it was trivial to rewrite all operations as a mix of logical OR (the simplest of operations) and logical NOT operations.

To reiterate:
  •  ( A ||  B) = CSG Addition
  • !(!A ||  B) = CSG Subtraction
  • !(!A || !B) = CSG Union

Now the way my CSG code iterated over the CSG tree was basically to tell each child-node in which list to store the polygons that are inside, outside or touching (I actually have 2 categories of touching polygons, depending on the alignment of the plane that touches it).

Performing a logical NOT comes down to just swizzle the lists I'm passing on, so it's essentially free.
Leaving me with code for just 1 CSG operation that needed to work perfectly!

Although I got everything working with the more complicated code before, it had some really scary parts because of the CSG subtraction operation which was really hard to get right because I flipped the orientation of the polygons to get it working.
This caused a cascading effect which complicated all the other operations as well.

So when I rewrote it today I found a much more elegant solution which allows me to flip the orientation of the polygons at the very end, again simplifying everything even more.

It's pretty clean and easy to read now, I still need to test it.
I'm way more confident now about it working in all test cases compared to before because of it's reduced complexity.

Wednesday, May 12, 2010

CSG operations

Lately I've been posting a lot of random thoughts, and this is one of them.
Today I read a post at filmic games about CSG, and for one reason or another a thought popped up in my head:

CSG operations can be expressed more intuitively as logical operations!

Technically, bitwise operations would work too, at least from a programming pov, but would make less sense (we're not working with bits here, aren't we).

  • A CSG complement operation would behave like a logical NOT operation (! in C languages), where the shape becomes "everything outside of the shape" (which is only useful as an intermediate step, not so much as an actual operation).
  • A CSG common (also known as 'intersection') operation behaves like a logical AND operation (&& in C languages) because only the parts that are kept are the ones shared between the two shapes.
  • A CSG addition (also known as 'union') operation behaves like a logical OR operation (|| in C languages) because all the parts of both shapes in this operation are kept afterwards.
  • A CSG subtraction operation would actually be a combination of AND and NOT (A&&!B).
The pictures have shamelessly been taken from wikipedia on the topic of logical operators which interestingly enough showed CSG like pictures!
(They're actually Venn diagrams)

Which shows that these things are, in fact, the very same thing!
(Hmm ... I wonder why I never noticed that before)

Note that logical OR (CSG union) operations can also be expressed as !(!A && !B), and that I've already shown that CSG subtraction can be expressed as (A && !B), meaning that all popular CSG operations can be implemented using only NOT and AND operations.

Now consider that a NOT operation would effectively not change the mesh at all, but would only invert the classification of triangles being inside or outside, there would effectively only be one kind of mesh modifying operation left..

Interesting!

Tuesday, May 11, 2010

More random thoughts

Random thought 1: GJK / backface culling

So I was reading another chapter in my "Game Engine Architecture" book, written by Jason Gregory (which, incidentally, is a really good book), about Physics or more specifically, about the GJK algorithm.

I've seen the algorithm mentioned before but only glaced over it, not realizing how useful it is, but now I do!
It's very useful for doing intersection tests for my CSG algorithm.

Part of the GJK algorithm is finding a point furthest along a particular direction within a convex polytope (which is, basically, the same thing as a CSG brush).

Maybe they're already doing this in real life, but I couldn't help but wonder, why aren't they backface culling polygons (and all it's vertices) using this direction vector?

Update: Giving this some more thought makes me think that this won't work.
When you have relatively few points then all the back face culling operations would take more cycles compared to just going through all points. Yet if you have lots and lots of points you end up having more branching and CPU-cache misses. So ignore this idea.

I wonder if storing vertices into 8 octants, where each octant is a particular direction, and then using the signs of the x,y & z components of the direction vector to pick an octant would work.


Random thought 2: CPU Cache & Component entity systems

Couldn't help but think that component entity systems, data orientated design (optimize for CPU cache), block based memory allocations, splitting all the work into tasks (for easy multi-threading) would work really really well together, if all where designed together.

After all, component entity systems have lots of lists of similar components on which systems work.
This is exactly what you'd want to be doing with data based design because it's cache friendly, and it also allows you to break things up into small tasks within the system..

Of course entity systems are traditionally only used for the gameplay code, and usually written in a very flexible (and potentially very slow) way, but I'm wondering if the entire engine could be written around these principles..


Random thought 3: Each VT Page mip should decrease in size

In 'traditional' virtual texturing a lower resolution mip of a page contains several lower resolution versions of higher resolution pages, so that each page, no matter what mip, has the same size (for example, 128x128)

This is useful because you can more easily mix different pages of different mip levels within the GPU page cache, and this is fine.

I've been wondering though, should we really be doing the same on the disk side?

After all, we could just update the pieces of a lower resolution mip page in the GPU cache, instead of loading the entire mip, which would include an incrementally larger virtual texture area the lower resolution the mip level is.

This would all not be a problem if a virtual texture would just be a texture, but it isn't.
It's a texture management system, or a texture atlas at best.

There are a lots of seems within the virtual texture where individual mip pages contain pieces of texture that, within the level, can be very far apart.

Only loading the pieces of the mip pages that we need would reduce loading pixels that we're not going to use anyway.

It also makes it easier when compressing the pages, because we can more easily omit channels when they're not being used by a page.
Before, any used channel would travel up the mip map chain, because one of the higher resolution mips would be using it.

This is all, of course, not an issue for all the pre-cached pages.
And, in all fairness, the highest resolution pages are always the biggest problem, and this would only optimize loading the lower resolution pages.

All the above issues could also, potentially, be solved by grouping pages within the virtual texture in a more optimal way, but you'd probably end up with a lot of wasted space.

Also compressing pages in 8x8 blocks (or something) and omitting unused channels in those individual blocks would help to avoid the above problems too, and would probably be a good idea anyway.


Random thought 4: Compressing the mip map chain

I'm also wondering if there are good ways to compress an entire mip map chain together in context, considering that there isn't likely to be too much difference between two mip-map levels.

We're already pre-caching several mip map levels... so, can we leverage that information so that we can decrease the disk size of all the pages that we load on the fly?

Thursday, May 6, 2010

Worldspace shadow-map / virtual texture - another random idea

Just a new random, not even a nearly completely thought out idea of mine.
Which I blame on only having time to dream up these things, but not actually try them $@#!
Yes, I am frustrated ;)

Anyway.

Suppose we perform our lighting within our virtual texture cache (aka surface caching), which I've blogged about often, and we store our texel world-space coordinates in the cache, wouldn't we also be able to use this information to do shadow mapping?

The (HUGE) downside is that we'd need a place to store the texel coordinates even for pages that aren't visible (being able to analytically cull pages and lights would definitely help here... page bounding boxes? bounding volume hierarchies?)
Would lower resolution mips be enough for lighting? (sometimes?)
I suppose some sort of preprocessing could help here.. I can imagine that a lot of the geometry that uses a specific page would essentially be flat.

The upside is that we'd only have to render geometry exactly once per frame (or less, with caching), and perhaps we could be rendering the lighting directly into the cache..
In which case having the knowledge of all the transparent pages between your texel and the light-source would be interesting! (Probably slow as hell though, even IF you manage to get this to work somehow)

What about texel world coordinate mip maps?
Would we need to store min/max coordinates instead of 'just' coordinates?
Could that be used in a sort of hierarchical z-buffer type of setup somehow?
For (rough) culling on the CPU side perhaps?
For both the shadow map and camera?
That would require us to have the texel coordinates on the CPU side though.. (static geometry only?)
Preprocessed and stored with the pages? (geometry images?)
But bandwidth is already an issue...
Then again, it would probably be very compressible, most of the time.
At least, for the higher resolution mips..

Argh, too many angles, too many ideas!
Most of them look extremely impractical to begin with, but it's a lot of fun to think about ;)


Besides all that, this idea popped in my head:
If every texel coordinate is unique with virtual texturing, wouldn't we be able to use this, somehow, to eliminate shadow acne? (Assuming you're not using cascaded shadow maps already & this would really be complicated by page mip maps)

Tuesday, May 4, 2010

Propagation volumes

I don't have much time to actually experiment with ideas these days, but I can still find a little time here and there to just let my thoughts go and see where they lead me.

Here's a thought I was playing with yesterday.

Suppose for a minute that we'd have the ultimate computer, memory is so large that it doesn't form any barriers, and CPU and memory speed is high enough to practically do anything we want.
So 'of course' all assets would consist out of a form of voxels,
but what about lighting?
Well all lighting would be performed within very high resolution propagation volumes of course!
Obviously, the propagation volumes would even handle light reflection, refraction, scattering and absorption.

This led me to the following, more interesting, idea.. what about audio?

Now I should mention that I'm nowhere near an expert on audio, so take this with a grain of salt, but audio is in the frequency domain, right?
Well, so are the spherical harmonics used in light propagation volumes!
Hmm.. doesn't that mean it should be possible to perform audio within a propagation volume-esque solution?

Just imagine being able to simulate how audio behaves within a scene, with reflections etc.!
I have no idea though which x-th order spherical harmonics, or what resolution of the volume would be "enough", nor do I have any idea how good this would sound.

But it sure sounds cool.
Err.. If you know what I mean.