I just wrote a couple of tests to see what happens if a programs grows it's stack until the limit. In one I allocated a very large array, and in the second I perform a very deep recursion. In both cases I received a SIGSEGV (on Linux, would happen the same in other systems?).
Can a program with strict reliability requirements detect if it is approaching to a such disastrous situation?
// At global scope...
void* stack_start = 0;
int main() {
char aVariable;
stack_start = &aVariable;
}
Then you could do the same thing in some function to determine the current stack pointer and print out the difference between the current stack pointer and stack_start.
getrlimit() will allow you to query the maximum stack allocated to your program.
As for detecting this condition (ie, almost out of stack space) on the fly (as opposed to explicitly checking, as I suggested above) I'm not sure you can.