How can I allocate enough memory for this program to m_fork in C?

Question

I have an application running on IRIX which uses m_fork() to create a number of processes. Source code.

When I run the application with 3 forks, everything works as expected. When I ask for four forks, it fails to fork with an "ENOMEM" error:

bash-2.05$ ./hello 3
 Process spawn limit is  8
 Hello world from process 1
 Hello world from process 2
 Hello world from process 0
 2nd line: Hello world from process 1
 2nd line: Hello world from process 2
 2nd line: Hello world from process 0
bash-2.05$ ./hello 4
 Process spawn limit is  8
m_fork() did not work

I know this is a memory allocation issue because when I throw in malloc(900000000); right before m_fork(), everything works as expected.

While this does fix the problem, it's not a suitable solution because it's just an arbitrarily large number. How can I dynamically allocate just enough memory for the forks?

e: The core question I'm trying to get answered here is, "how can I allocate enough memory for the forks?"