Deploying Tensorflow models as Windows exe

Question

I want to use Tensorflow 1.4 for my ML modeling needs. My use case requires:

• Training the model on GPU <--- I know how to do this with TF

• Deploying the trained model on an ordinary box - as an .exe on CPU running Windows (for inference) <---- I don't know how to do this.

Can somebody tell me if TF 1.4 supports this and if so, point me to a guide or explain how its done ?

Answer 1

This is a little late but this video on youtube covers it pretty well.

He uses pyinstaller which grabs everything needed and puts it all either into one executable without anything else, or a folder with the exe in there and other stuff.

I've tried this myself and it works pretty well, although since pyinstaller smashes everything needed into one folder which gets really huge, it includes the entire tensorflow library, the python interpreter and if you use tensorflow-gpu, it also includes the cudnn files as well which are like 600mb, effectively leaving you with over a 1gb worth of files in the end. That can be reduced by excluding modules that you don't need, I recommend creating a virtual environment and start with a clean installation of python.

Hope this helps in anyway.