CODEWITHSUNDEEP
c
M. Averbach
M. Averbach

Reputation: 125

C Concurrent Processes and Static variables

I have created a c program, that does everything in a single process. For example, it sequentially reads file by file, and outputs something. I had to use a HUGE array called vectors, and so I declared it static (because it was giving me a seg fault). static double vectors[100000][10000].

Now I need to create the same output of the previous program using multiple concurrent processes.

What I have so far:

pid_t pids[argc - 1];
int pid;
for (e=1; e < argc; e++)
{
     pid = fork();
     if (pid < 0)
     {
          //error
     }
     else if (pid > 0)
     {
           pids[e-1] = pid
     }
     else
     {
           printf("The child process of %d is started\n", pids[e-1]);
           printf("The child process of %d is finished\n", pids[e-1]);
     }
}

for (int i  = 0 ; i < argc - 1 ; i++) 
{
int status;
waitpid(pids[i], &status, 0);
printf("Process %d is finished\n", pids[i]);
}

Now i'm just trying to see if the outputs of the child processes interleave which means they will run concurrently.

So far i'm getting "Killed" message when I run the above, but once I comment out the static vectors array, it runs fine. Why is that?

The output when it does run is really weird, basically I have 0 for pids elements.

Any help would be very appreciated. Thank you.

Upvotes: 0

Views: 133

Answers (1)

Ctx
Ctx

Reputation: 18410

Your process gets killed by the OOM-Killer (Out of memory).

static double vectors[100000][10000] needs about 100000*10000*8 bytes of memory, which makes about 8GB. This memory is not physically allocated until something is written to it (memory overcommitment). If you fork() n times and write to these pages in each process, the memory needed is about n*8GB, which quickly exceeds your physical memory + swap, I assume. dmesg should show you a message regarding this.

A solution is, to create a shared map with mmap() before fork()ing and make all processes work on the same array (if that is what you need):

double *vectors = mmap(NULL, 10000*100000*sizeof(double), 
                       PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, -1, 0);

and instead of

vectors[a][b]

access

vectors[a*10000+b]

Upvotes: 1

Related Questions