CODEWITHSUNDEEP
machine-learningtensorflowconv-neural-network
Ayes
Ayes

Reputation: 61

Why use TensorFlow for Convolutional Neural Networks

I recently took a courser by Andrew Ng on Coursera. After that I shifted to Python and used Pandas, Numpy, Sklearn to implement ML algorithms. Now while surfing I came across tensorFLow and found it pretty amazing, and implemented this example which takes MNIST data as input. But I am unsure why use such as library(TensorFlow)? We are not doing any parallel calculations, since the weights updated in the previous epoch are used in the next one??? I am finding it difficult to find a reason to use such a Library?

Upvotes: 1

Views: 865

Answers (2)

chris
chris

Reputation: 1894

You might be able to find pre-baked models in sklearn or other libraries, but TensorFlow allows for really fast iteration of custom machine learning models. It also comes with a ton of useful functions that you would have to (and probably shouldn't) write yourself.

To me, it's less about performance (though they certainly care about performance), and more about whipping out neural networks really quickly.

Upvotes: 0

mrry
mrry

Reputation: 126154

There are several forms of parallelism that TensorFlow provides when training a convolutional neural network (and many other machine learning models), including:

  1. Parallelism within individual operations (such as tf.nn.conv2d() and tf.matmul()). These operations have efficient parallel implementations for multi-core CPUs and GPUs, and TensorFlow uses these implementations wherever available.

  2. Parallelism between operations. TensorFlow uses a dataflow graph representation for your model, and where there are two nodes that aren't connected by a directed path in the dataflow graph, these may execute in parallel. For example, the Inception image recognition model has many parallel branches in its dataflow graph (see figure 3 in this paper), and TensorFlow can exploit this to run many operations at the same time. The AlexNet paper also describes how to use "model parallelism" to run operations in parallel on different parts of the model, and TensorFlow supports that using the same mechanism.

  3. Parallelism between model replicas. TensorFlow is also designed for distributed execution. One common scheme for parallel training ("data parallelism") involves sharding your dataset across a set of identical workers, performing the same training computation on each of those workers for different data, and sharing the model parameters between the workers.

In addition, libraries like TensorFlow and Theano can perform various optimizations when they can work with the whole dataflow graph of your model. For example, they can eliminate common subexpressions, avoid recomputing constant values, and generate more efficient fused code.

Upvotes: 2

Related Questions