I've done a couple of things however.
First of all I've moved the actual CSG code from Unity/C# to C++. The reason for this is because the more low level optimizations I did the more it felt like C# was actively resisting me trying to optimize everything.
The more you do low level optimizations, the more it starts to feel like C++ but without all the helpful syntax such as 'inline' and with pointer juggling without all the unsafe nonsense. I was actually already working with lots of big C# arrays + indices so I was effectively already doing memory management. Moving everything to C++ actually made it possible for me to clean up the code and make it MORE readable. Not something you usually hear when you compare C++ to C#!
The whole plugin was always intended as an editor only plugin anyway, so fortunately I won't condemn myself to supporting a gazillion platforms by moving everything to C++!
Other big benefits are that I have a clear separation between the Unity part and the CSG part, I can use good quality low level profilers to find bottlenecks and I have way more low level control over memory and performance now.
While I was moving everything to C++ it also gave me a chance to revisit some parts of the CSG process and fix some issues. One improvement I made, for instance, is normal smoothing, which at the moment can only be used on polygons that share the same texture coordinate surface.
Here you can see the normals in unity's 'normal' debug view.
Another, internal, improvement is that the CSG representation is now an n-tree internally instead of a binary tree. This matches the unity hierarchy better and allows me to have shallower hierarchies which should perform better.
My focus now is to get something out there, so I'm going to fix the remaining bugs and implement only what really needs to implemented for the first version.