Failed precondition: Python interpreter state is not initialized. The process may be terminated

Question

I am trying train my images. This data's size is 50.000 images.

My images properities are:

If i should change my images properities, how can i do that?

This is my first image classifier work so there can be a lot of mistake. Please understand.

Can you help improve my code?

This is my code

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv2D,MaxPooling2D,Activation,Dropout,Flatten,Dense
from tensorflow.keras.preprocessing.image import ImageDataGenerator,img_to_array,load_img
import matplotlib.pyplot as plt
from glob import glob
from PIL import Image
import os


import numpy as np

train_path="Düzgün2/"

images = [train_path for train_path in os.listdir() if train_path.endswith(('jpeg', 'png', 'jpg'))]
for x in images:
    img = Image.open(x)
    img.thumbnail((600,600))
    img.save("resized_"+x, optimize=True, quality=40)
test_path="Test/"
data=load_img(train_path + "Basler_acA1440-220um__40052667__20201123_114618747_7932_result.jpg")
x=np.asarray(data)

plt.imshow(data)
plt.axis("off")
plt.show()


print(x.shape)
className=glob(train_path + "/*")
print(className)
numberOfClass=len(className)
print("NumberOfClass:",numberOfClass)

#CNN model

model=Sequential()
model.add(Conv2D(32,(3,3),input_shape=(224,224,3)))#xshape burada gelen resimin pixseli 3 ise rgb yi tesmil ediyor so that senin çivi resimlerine göre ayarla
model.add(Activation("relu"))
model.add(MaxPooling2D())
model.add(Conv2D(32,(3,3)))#xshape burada gelen resimin pixseli 3 ise rgb yi tesmil ediyor so that senin çivi resimlerine göre ayarla"""
model.add(Activation("relu"))
model.add(MaxPooling2D())

model.add(Flatten())
model.add(Dense(1024))#1024 nörondan oluşuyor
model.add(Activation("relu"))
model.add(Dropout(0.5))
model.add(Dense(numberOfClass))#output
model.add(Activation("softmax"))

model.compile(loss="categorical_crossentropy",optimizer="rmsprop",metrics=["accuracy"])

batch_size=32# her iterasyonda 32 resmi kullan

#data generator

train_datagen=ImageDataGenerator(rescale=1./255,shear_range=0.3,
                   horizontal_flip=True,zoom_range=0.3)#rgb 0-255 0-1 ile bir arası değer alıyoruz,shear sağa sola yatırma
test_datagen=(ImageDataGenerator(rescale=1./255))

train_generator=train_datagen.flow_from_directory(train_path,
                                                 target_size=x.shape[:2],batch_size=batch_size,color_mode="rgb",
                                                 class_mode="categorical")
test_generator=test_datagen.flow_from_directory(test_path,
                                                 target_size=x.shape[:2],batch_size=batch_size,color_mode="rgb",
                                                 class_mode="categorical")

model.fit_generator(generator=train_generator,steps_per_epoch=1600 // batch_size,epochs=100,validation_data=test_generator,
                        validation_steps=800 // batch_size)
   

This my output:

NumberOfClass: 11376
2020-12-15 14:10:26.449262: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library nvcuda.dll
2020-12-15 14:10:27.159243: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: 
pciBusID: 0000:01:00.0 name: GeForce GTX 1660 Ti computeCapability: 7.5
coreClock: 1.59GHz coreCount: 24 deviceMemorySize: 6.00GiB deviceMemoryBandwidth: 268.26GiB/s
2020-12-15 14:10:27.159530: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_101.dll
2020-12-15 14:10:27.221768: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cublas64_10.dll
2020-12-15 14:10:27.269324: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cufft64_10.dll
2020-12-15 14:10:27.275602: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library curand64_10.dll
2020-12-15 14:10:27.301737: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusolver64_10.dll
2020-12-15 14:10:27.314461: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusparse64_10.dll
2020-12-15 14:10:27.407152: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudnn64_7.dll
2020-12-15 14:10:27.407410: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0
2020-12-15 14:10:27.407830: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2020-12-15 14:10:27.416924: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x193e2fdee00 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-12-15 14:10:27.417440: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-12-15 14:10:27.417880: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: 
pciBusID: 0000:01:00.0 name: GeForce GTX 1660 Ti computeCapability: 7.5
coreClock: 1.59GHz coreCount: 24 deviceMemorySize: 6.00GiB deviceMemoryBandwidth: 268.26GiB/s
2020-12-15 14:10:27.418242: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_101.dll
2020-12-15 14:10:27.418413: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cublas64_10.dll
2020-12-15 14:10:27.418586: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cufft64_10.dll
2020-12-15 14:10:27.418754: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library curand64_10.dll
2020-12-15 14:10:27.418922: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusolver64_10.dll
2020-12-15 14:10:27.419095: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusparse64_10.dll
2020-12-15 14:10:27.419268: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudnn64_7.dll
2020-12-15 14:10:27.419477: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0
2020-12-15 14:10:28.461954: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-12-15 14:10:28.462106: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108]      0 
2020-12-15 14:10:28.462194: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0:   N 
2020-12-15 14:10:28.462442: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4750 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1660 Ti, pci bus id: 0000:01:00.0, compute capability: 7.5)
2020-12-15 14:10:28.465422: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1942c7ee0b0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2020-12-15 14:10:28.465660: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): GeForce GTX 1660 Ti, Compute Capability 7.5
Found 0 images belonging to 0 classes.
Found 0 images belonging to 0 classes.
WARNING:tensorflow:From C:/Users/tatu/PycharmProjects/pythonProject4/main.py:70: Model.fit_generator (from tensorflow.python.keras.engine.training) is deprecated and will be removed in a future version.
Instructions for updating:
Please use Model.fit, which supports generators.
Epoch 1/100
2020-12-15 14:10:30.182839: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cublas64_10.dll
Traceback (most recent call last):
  File "C:/Users/tatu/PycharmProjects/pythonProject4/main.py", line 70, in <module>
    validation_steps=800 // batch_size)
  File "C:\Users\tatu\miniconda3\envs\tensorfloww\lib\site-packages\tensorflow\python\util\deprecation.py", line 324, in new_func
    return func(*args, **kwargs)
  File "C:\Users\tatu\miniconda3\envs\tensorfloww\lib\site-packages\tensorflow\python\keras\engine\training.py", line 1479, in fit_generator
    initial_epoch=initial_epoch)
  File "C:\Users\tatu\miniconda3\envs\tensorfloww\lib\site-packages\tensorflow\python\keras\engine\training.py", line 66, in _method_wrapper
    return method(self, *args, **kwargs)
  File "C:\Users\tatu\miniconda3\envs\tensorfloww\lib\site-packages\tensorflow\python\keras\engine\training.py", line 848, in fit
    tmp_logs = train_function(iterator)
  File "C:\Users\tatu\miniconda3\envs\tensorfloww\lib\site-packages\tensorflow\python\eager\def_function.py", line 580, in __call__
    result = self._call(*args, **kwds)
  File "C:\Users\tatu\miniconda3\envs\tensorfloww\lib\site-packages\tensorflow\python\eager\def_function.py", line 644, in _call
    return self._stateless_fn(*args, **kwds)
  File "C:\Users\tatu\miniconda3\envs\tensorfloww\lib\site-packages\tensorflow\python\eager\function.py", line 2420, in __call__
    return graph_function._filtered_call(args, kwargs)  # pylint: disable=protected-access
  File "C:\Users\tatu\miniconda3\envs\tensorfloww\lib\site-packages\tensorflow\python\eager\function.py", line 1665, in _filtered_call
    self.captured_inputs)
  File "C:\Users\tatu\miniconda3\envs\tensorfloww\lib\site-packages\tensorflow\python\eager\function.py", line 1746, in _call_flat
    ctx, args, cancellation_manager=cancellation_manager))
  File "C:\Users\tatu\miniconda3\envs\tensorfloww\lib\site-packages\tensorflow\python\eager\function.py", line 598, in call
    ctx=ctx)
  File "C:\Users\tatu\miniconda3\envs\tensorfloww\lib\site-packages\tensorflow\python\eager\execute.py", line 60, in quick_execute
    inputs, attrs, num_outputs)
tensorflow.python.framework.errors_impl.InvalidArgumentError:  Reduction axis -1 is empty in shape [0,0]
     [[node ArgMax (defined at /Users/tatu/PycharmProjects/pythonProject4/main.py:70) ]] [Op:__inference_train_function_921]

Function call stack:
train_function

2020-12-15 14:10:30.637723: W tensorflow/core/kernels/data/generator_dataset_op.cc:103] Error occurred when finalizing GeneratorDataset iterator: Failed precondition: Python interpreter state is not initialized. The process may be terminated.
     [[{{node PyFunc}}]]

Process finished with exit code 1

This is my error

Function call stack:
train_function

2020-12-15 14:10:30.637723: W tensorflow/core/kernels/data/generator_dataset_op.cc:103] Error occurred when finalizing GeneratorDataset iterator: Failed precondition: Python interpreter state is not initialized. The process may be terminated.
     [[{{node PyFunc}}]]

Answer 1

Just add this before your program i.e after importing dependencies

gpus = tf.config.experimental.list_physical_devices('GPU')
tf.config.experimental.set_memory_growth(gpus[0], True)

Answer 2

[EDIT] As said before, your input model needs to match the image shape.

If you're using this:

 model.add(Conv2D(32,(3,3),input_shape=(224,224,3)))

Your input image dimension should be (224, 224, 3), and as you showed the input is 1440x1080, but you're converting it to 600x600, right? Choose one resolution size and stick with it. But if you're using a pretrained model, you need to use the model's resolution.

---

You can't change the input layer shape of a trained or already built model. That's what he's complaining about. The expected input have 4 dimensions and you're passing 3.

model.build(input_shape=(150, 150, 3)) # what you did
model.build(input_shape=(w, x, y, z)) # what you should do

First of all, check the dimension of your data, are they RGB, RGBA, I assume that since you're using a CNN, you're dealing with images (which may not be true). If your data shape isn't compatible with the model, you might need to transform your data if you want to use the same model. Another approach is to find or create another model that use your data's shape as input.

As indicated by David Waterworth, I suppose here that in your case, the first dimension w is related to the bash size used, and (x, y, z) are related to the data itself.