CODEWITHSUNDEEP
c++multithreadingc++11cluster-computing
Tyler
Tyler

Reputation: 1837

Add cluster capability to C++11 threaded app

I have a multithreaded C++11 program (using the C++11 mt libraries) that would benefit from more computational resources. Memory isn't an issue at the scale I'm interested in, but being scalable wouldn't be a bad thing.

Specifically, the problem I'm solving is finding the size of the max clique in a graph. To distribute the program, I take an N order graph and break it into N subgraphs, each subgraph being the neighborhood of a unique node. The size of the max clique of each of these subgraphs is found, and the maximum of these plus the one is returned as the max clique of the entire graph.

The nodes of each graph (the total graph and the subgraphs) are represented as an array of integers. The same is true of edges. So, technically, all I need to communicate between nodes is an array of ints. The return result would be an int, the size of the max clique found.

From what I've read, MPI is focused on problems needing distributed memory. This seems to be unnecessary for my problem. The problem is minutes long at 200MB of memory (w/ 16 threads on 16 cores), so, at 4GB (the max of most of my compute nodes), it would be computationally infeasible even with all the nodes I have. So, my question is: What it a good way to accomplish this? If I'm wrong about MPI, please feel free to correct me.

Upvotes: 2

Views: 403

Answers (1)

pyCthon
pyCthon

Reputation: 12341

With out knowing much about the graph problem your trying to solve, here are some of your options that may be worth considering.

MPI

-Which you already mentioned, it is your standard academic style, High performance computing defacto API for distributed computing, you can't go wrong and it will scale to almost any number of cores. Theres also a boost C++ api. Cons are it will crash on first error or message failure.

any of the various AMQP protocols

0MQ QPID exc.

-These are also message passing based but fault tolerant and less performant than MPI. Slightly easier to work with than MPI but fault tolerance may not be relevant to your problem.

HPX

-a hybrid approach, that extends the C++11 threading API and also allow distributed computing, in a state of the art fashion. If your most familiar with threads this maybe a good option.

Upvotes: 2

Related Questions