So, I'd like to stick with the old-fashioned way of building each cpp file independently when in debug mode to keep compile times fast when debugging, but switch to using a unity build for release mode to allow the compiler to optimize across translation unit boundaries
This sounds the wrong way around. Unity builds are meant to be faster, and unity builds have just one translation unit.
Although if your builds take around twenty seconds, I suspect you're not going to see any benefit at all from a unity build.
Yes, lots of what you've said doesn't make much sense in the context of unity builds.