Tuesday, May 4, 2010

Propagation volumes

I don't have much time to actually experiment with ideas these days, but I can still find a little time here and there to just let my thoughts go and see where they lead me.

Here's a thought I was playing with yesterday.

Suppose for a minute that we'd have the ultimate computer, memory is so large that it doesn't form any barriers, and CPU and memory speed is high enough to practically do anything we want.
So 'of course' all assets would consist out of a form of voxels,
but what about lighting?
Well all lighting would be performed within very high resolution propagation volumes of course!
Obviously, the propagation volumes would even handle light reflection, refraction, scattering and absorption.

This led me to the following, more interesting, idea.. what about audio?

Now I should mention that I'm nowhere near an expert on audio, so take this with a grain of salt, but audio is in the frequency domain, right?
Well, so are the spherical harmonics used in light propagation volumes!
Hmm.. doesn't that mean it should be possible to perform audio within a propagation volume-esque solution?

Just imagine being able to simulate how audio behaves within a scene, with reflections etc.!
I have no idea though which x-th order spherical harmonics, or what resolution of the volume would be "enough", nor do I have any idea how good this would sound.

But it sure sounds cool.
Err.. If you know what I mean.

9 comments:

  1. Hi!
    I know a man in my lab that was working on sound propagation before he also starts working on light propagation for global illumination. Indeed, equations seems similar and as with light, there is reflection, refraction, etc... Today, he publishes papers on sound such as "Real Time Acoustic Rendering of Complex Environments Including Diffraction and Curved Surfaces".

    Actually, I wonder if something has been done like light propagation volume applied to sound propagation... Investigation required here! :)

    ReplyDelete
  2. I had a scary thought though, if audio is recorded at, say 44.1 kHz, does that mean the audio propagation volume would need to be updated 44100 times a second?!
    Hmm. Perhaps this is not something which can be done in real time "any time soon"(tm) ;)

    ReplyDelete
  3. I agree, that's scary! :)
    I was thinking about another way of doing this. Maybe sources origin could be first added in the volume. Then, their contribution would be propagated in the volume by only taking into account the "modification" such as attenuation, etc. Instead of storing the "sound" in voxel, I would stored the "modification" applied to the sources sound per voxel: indeed, this would required a linked list for each voxel with each element containing "modification" applied on each sources when listen at this position. Finally, you would just need to sample the voxel at the camera position and play the sources sound that have reached this position with final "modifications" applied. The only problem is that I do not have enough background in sound to know if it is possible to store these "modifications": it seems quite intuitive for volume attenuation but is it ok for other modification such as frequencies or pitch modification when sound is traveling through medium etc... If you see what I mean...

    ReplyDelete
  4. Also, it would be important to find an accurate way to add virtual environment occlusion and surfaces properties in the volume. This in order to compute correct modification when the sound is propagated in the volume.

    Arf, now I want to try to implement this! XD

    ReplyDelete
  5. Perhaps every sound source could be identified using specific per sound source frequencies? But then again these frequencies probably would get distorted a lot..
    Somehow I don't think the frequency domain would fit your idea ;)

    ReplyDelete
  6. Lol, well let me know what your results are if you do! ;)

    ReplyDelete
  7. @Sander: well if sound is 44.1 khz, and you assume that your simulation would need to run with 44.1khz, think about light: that's dozens of THz! So the answer is: no, you don't need that temporal resolution to get useful results...

    ReplyDelete
  8. @root42 As I mentioned before, I'm hardly an expert on audio ;)
    So the question then becomes, what would be an acceptable audio "frame rate"?

    ReplyDelete
  9. The required accuracy of the simulation will depend on the maximum frequency of the signal - this is why we only simulate indirect lighting, the usually diffuse surface BRDF causes indirect lighting to be low-frequency enough that we don't notice the sampling artefacts.

    ReplyDelete

To you spammers out there:
Spam will be deleted before it shows up, so don't bother.