CODEWITHSUNDEEP
tensorflowipc
xpilot
xpilot

Reputation: 1019

Sharing tensorflow model between processes

I have several processes running on a server, one of which is training a model in tensorflow. Periodically, I want the trainer to send the current model to the other processes. The way I do this now is with the usual Saver class that can save to and restore from disk.

However, I think this form of IPC is rather inefficient, and it is potentially causing filesystem lockups on the server. If there were a way to serialize the variables into some blob I could send that over a zmq broadcast pipe, I but I haven't found this in the docs.

Alternatively, distributed tensorflow is probably up to the task, but I don't think I need something so complicated.

Upvotes: 1

Views: 1031

Answers (1)

Jules Gagnon-Marchand
Jules Gagnon-Marchand

Reputation: 3800

You could preshare the architecture then use tf.get_collection(tf.GraphKeys.VARIABLES) at every number of steps of your liking, and run it to get the values, then you can use variable.assign at the other end to load up the values in the appropriate variables.

Upvotes: 2

Related Questions