Creating multiple TCP Socket connections

Question

In one of my parsing program i have to do the following(in C,C++,VC++):-

I have to connect 100 machines(dialysis machines) through socket programming. Send request to each machine and receive the response from the machine and the data that i will get from the machine i will have to parse that data and write the values in the a file.I have to send the request to each machine after every 5seconds.

So to accomplish the above task i am thinking to do it as:-

I will read the ip and port of each machine from the data base,make a thread to connect to each machine and in each thread a sub thread(child thread) will be made which will send and receive and parse the data from the machine after every 5 second(and write the values in a txt file). My parsing function will be common.

Is it the feasible solution. Please help me.Thnaks in advance.

Answer 1

Your solution sounds reasonable except for one point. You mention that you will create a thread to connect to each machine, then a sub-thread to manage sending, receiving, and parsing. I dont understand why you need to create a sub-thread. You should be able to handle everything in the connection thread. Also consider that 1 thread per connection may not scale well, if this application has to deal with a larger number of machines, a thread per machine should be avoided.

It may even be possible to achieve this with a simple thread pool instead of 1 thread per connection, which wont scale well. You could consider creating tasks that get put on a worker queue every 5 seconds, and the pool of threads would connect, read, disconnect, parse, and process. Assuming this is TCP/IP you probably shouldnt keep the connection open, but rather connect/disconnect for each read, similar to HTTP.

Here is a vc++ thread pool related question. And here is some more related info.

Another alternative could be to use libevent for the socket communication. As for the parsing, there are also other libraries that can be used like Apache Thrift or JSon, all of which are open source. The down side to these parsing libraries is that you may have to also modify the dialysis machines which may not be an option. If you can use something like Thrift, you can get everything from one library: socket comm and parsing.

Here is some code for the simple case of 1 thread per connection:

class ThreadInfo
{
public:
  ThreadInfo(const string &ipAddress, uint16_t port) : ipAddress_(ipAddress), port_(port) {}
  string getIpAddress() {return ipAddress_;}
  uint16_t getPort() {return port_;}
  string getRecvBuffer() {return recvBuffer_;}

private:
  string ipAddress_;
  uint16_t port_;
  string recvBuffer_;
};

void *threadEntryPoint(void *userData)
{
  ThreadInfo *threadInfo = (ThreadInfo*) userData;

  // You need to decide if you want to keep the connection open while sleeping
  // or open and close it for each transaction. Change code here accordingly.
  // Create socket with threadInfo->getIpAddress() and threadInfo->getPort()

  // while(1)
  //   Send request to each machine
  //   Get response from each machine and store in threadInfo->getRecvBuffer()
  //       The buffer could also be a local var in this function, decide accordingly
  //   parse data accordingly
  //   sleep 5 seconds
}

uint16_t getPort(int machineNum) { return 3456; }
string getIpAddress(int machineNum) { return string("192.168.1.2"); }
int main(int argc, char **argv)
{
   // 3 items that we need, and that you will have to plugin accordingly:
   //   1) Num threads, assuming 100 for now
   //   2) IP address of each external machine, implement getIpAddress() accordingly
   //   3) port of each machine, implement getPort() accordingly

   int numThreads(100);
   list<pthread_t> threadIdList;

   for(int i = 0; i < numThreads; ++i)
   {
      pthread_t threadId;
      ThreadInfo *threadInfo = new ThreadInfo(getIpAddress(i), getPort(i));
      pthread_create(&threadId, NULL, threadEntryPoint, threadInfo);
      threadIdList.push_back(threadId);
   }

   // Wait for the threads to finish
   std::list<pthread_t>::iterator iter = threadIdList.begin();
   while(iter != threadIdList.end())
   {
     pthread_t threadId = *iter++;
     pthread_join(threadId, NULL);
   }
}

Answer 2

For 100 machines, polling every 5 seconds, one thread per machine is reasonable - the threads are going to be blocked on the Sleep(5000) for most of the time and blocked on socket I/O or disk I/O for most of the rest of the time. For this sort of loading, (or even five times the loading), I don't see any need to resort to async I/O or thread pools - why complicate matters unnecessarily?

As pointed out by @Brady, I don't see why you would need more than one thread per connection, assuming that the requirement is as you post - just polling every ~5 seconds and writing the replies to a text file.

I'm guessing, (hoping:), that the 5 second interval is not a safety-critical real-time requirement and that the dialysis machines will continue to operate normally if the period is occasionally 6 seconds due to some temporary software or network delay. I'm not a haematologist/nephrologist, but I would be surprised if any dialysis machine could make any significant change to overall treatment, (which takes hours), if a poll/instruction was occasionaly delayed for an extra second.

Edit - re. 'parsing function and writing data to a file function is common for all threads' - should be fine, assuming a different text file for each machine. If the logs are all written one log file, that's more of a problem - each log entry should really be queued off to one logger thread that, alone, writes to the log file. Using an existing, proven logger framework that already supports this kind of functionality would be the easiest solution.

Answer 3

If you'd use the boost::asio framework for both network and disk IO you would likely get away with far less threads than one per machine to handle.

Plus, it has a nice high-level interface for socket programming.