How do we recognize text on real image after training MNIST with TensorFlow?

Question

I have just ran the code to train the MNIST model. How can we apply it to recognize handwriting in the real image? I'm a newbie and have just started learning this part. I've searched and couldn't find info about this.

Answer 1

A neural network is essentially very good at recognizing "stuff" that you feed it. For the MNIST example, you are feeding it lots of 28x28x1 images of handwritten digits.

Therefore, as an extension, if you want to use the exact same model for handwriting (I assume you mean characters from A-Z now), you would then have to feed it lots of 28x28x1 images of handwritten characters.

Answer 2

You must take into account that the MNIST data set is very much a "toy data set", it is just the "Hello world!" of image recognition, and thus, it is not very useful for real life problems. If you recreate the training images conditions and try to predict that, you will get good results, but again, it is not a real life problem, but maybe it's a fun project to start with.

With that said, maybe you are not yet familiar with feature extraction, a technique that modern convolutional neural networks use. Using that maybe you could have something useful out of an MNIST trained model. Tensorflow's CNN tutorial would be very helpful in that sense. To further understand this tecniques I would highly recommend chapter 6 of Michael Nielsen's Deep Learning online book

Hope it helps

Answer 3

It depends on your specific task. The MNIST model can classify character digits and so that is the data you need to feed it.

If you insist on using the MNIST model (RNNs specifically LSTMs are a much better option which most OCRs use.) one approach would be to run a sliding window over your hand written text image and create a text file of the character predicted by your model. But that presents its own set of challenges like novelty detection and slide window size. Its an overkill