CODEWITHSUNDEEP
tensorflowneural-networkdeep-learning
lenhhoxung
lenhhoxung

Reputation: 2746

Pre-train neural networks in TensorFlow

I'm training a deep neural network. Training directly the full network is difficult, therefore, I prefer to train layer by layer. Firstly, I train a network with a single hidden layer. After that, I save the model using tf.train.Saver. Next time, I restore variables by calling: 

saver.restore(sess, "runs/simple-model.ckpt")

Of course, this time, the architecture has changed because I insert a new layer between the hidden layer and the output layer, and a new variable is used. For this reason, Tensorflow throws an error: Key not found

My question is that: How to train neural network layer by layer in Tensorflow? I want to do it before final fine-tuning of the full network. Many thanks.

Upvotes: 3

Views: 1781

Answers (2)

lenhhoxung
lenhhoxung

Reputation: 2746

Finally, I come to a simple solution. In Python, a numpy array can be saved easily by using numpy.save. After that, we can load the array by: numpy.load. Therefore, after training, I can evaluate the variable using var.eval() to convert it to a numpy array, then save it.

Upvotes: 1

ginge
ginge

Reputation: 1972

Try to define the entire network with shared weights. Instead of training over all the layers define a function that runs just the layers that you want and finally save the entire network.

Upvotes: 1

Related Questions