It's OK if run in anaconda prompt but ModuleNotFoundError: No module named 'keras' in Spyder

Question

I run a project downloaded from GitHub. Strangely, if I run it in anaconda prompt, it is OK, but if I run it in Spyder, it emerges(I installed anaconda and Sypder in win10, and keras installed as well):

runfile('E:/MySourceCode/neural_image_captioning-master-oarriaga/src/train.py', wdir='E:/MySourceCode/neural_image_captioning-master-oarriaga/src') Reloaded modules: evaluator, generator Traceback (most recent call last):

File "", line 1, in runfile('E:/MySourceCode/neural_image_captioning-master-oarriaga/src/train.py', wdir='E:/MySourceCode/neural_image_captioning-master-oarriaga/src')

File "D:\ProgramData\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 710, in runfile execfile(filename, namespace)

File "D:\ProgramData\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 101, in execfile exec(compile(f.read(), filename, 'exec'), namespace)

File "E:/MySourceCode/neural_image_captioning-master-oarriaga/src/train.py", line 3, in from keras.callbacks import CSVLogger

ModuleNotFoundError: No module named 'keras'

train.py is as follows:

from evaluator import Evaluator
from generator import Generator
from keras.callbacks import CSVLogger
from keras.callbacks import ModelCheckpoint
from keras.callbacks import ReduceLROnPlateau
from models import NIC
from data_manager import DataManager

num_epochs = 50  
batch_size = 64 
root_path = '../datasets/IAPR_2012/'
captions_filename = root_path + 'IAPR_2012_captions.txt'
data_manager = DataManager(data_filename=captions_filename,
                            max_caption_length=30,
                            word_frequency_threshold=2,
                            extract_image_features=False,
                            cnn_extractor='inception',
                            image_directory=root_path + 'iaprtc12/',
                            split_data=True,
                            dump_path=root_path + 'preprocessed_data/')

data_manager.preprocess()
print(data_manager.captions[0])
print(data_manager.word_frequencies[0:20])

preprocessed_data_path = root_path + 'preprocessed_data/'
generator = Generator(data_path=preprocessed_data_path,
                      batch_size=batch_size)

num_training_samples =  generator.training_dataset.shape[0]
num_validation_samples = generator.validation_dataset.shape[0]
print('Number of training samples:', num_training_samples)
print('Number of validation samples:', num_validation_samples)

model = NIC(max_token_length=generator.MAX_TOKEN_LENGTH,
            vocabulary_size=generator.VOCABULARY_SIZE,
            rnn='gru',
            num_image_features=generator.IMG_FEATS,
            hidden_size=128,
            embedding_size=128)

model.compile(loss='categorical_crossentropy',
              optimizer = 'adam',
              metrics=['accuracy'])

print(model.summary())
print('Number of parameters:', model.count_params())

training_history_filename = preprocessed_data_path + 'training_history.log'
csv_logger = CSVLogger(training_history_filename, append=False) 
model_names = ('../trained_models/IAPR_2012/' +
               'iapr_weights.{epoch:02d}-{val_loss:.2f}.hdf5')
model_checkpoint = ModelCheckpoint(model_names,                 #Callback
                                   monitor='val_loss',
                                   verbose=1,
                                   save_best_only=False,
                                   save_weights_only=False)

reduce_learning_rate = ReduceLROnPlateau(monitor='val_loss', factor=0.1,  #Callback
                                         patience=5, verbose=1)

callbacks = [csv_logger, model_checkpoint, reduce_learning_rate]

model.fit_generator(generator=generator.flow(mode='train'), 
                    steps_per_epoch=int(num_training_samples / batch_size),
                    epochs=num_epochs,
                    verbose=1,
                    callbacks=callbacks,
                    validation_data=generator.flow(mode='validation'),
                    validation_steps=int(num_validation_samples / batch_size))

evaluator = Evaluator(model, data_path=preprocessed_data_path,
                      images_path=root_path + 'iaprtc12/')

evaluator.display_caption()

Is there anything wrong with setting in Spyder? And in anoconda prompt, is there any difference while pip keras before "activate tensorflow" and after " activate tensorflow"?Many thanks.

Answer 1

The error may be that you are using the wrong environment. In anaconda navigator make sure you are in the right environment before you launch spyder

Answer 2

If you're using conda then it's best to install everything with conda if you can, rather than mixing it up with pip.

The purpose of conda is that you can create multiple environments which each have different packages installed, or different versions of the same package, without causing conflicts, because only one environment is activated at one time. Those packages include Python itself, and Spyder. So to use a Python module with Spyder you need to create a conda environment containing both the module and Spyder:

conda create -n mykerasenv keras spyder

Then to use that environment you need to activate it before starting Spyder:

activate mykerasenv # (or 'source activate mykerasenv' if that doesn't work)
spyder

Depending on your conda/Anaconda installation you may be able to do the same thing through the Anaconda Navigator and/or Anaconda Start menu shortcuts, but the command line version should always work.

If you do need to use pip to install a package that you can't get via conda, you need to do that after creating and activating the conda environment, so that pip installs it in the correct place.

If you've tried installing things into your root conda environment (i.e. without first creating a new environment with conda create and then activating it) then you may be best off uninstalling Anaconda and starting from scratch. If any of this is confusing I recommend having another read of the conda docs.

Answer 3

It might be an issue with your keras installation. If you already tried installing it with:

$ conda install keras

and it didn't work, I suggest uninstalling it and then reinstalling it with pip3:

$ conda uninstall keras
$ sudo pip3 install keras

You can check if it works by importing keras in Spyder:

> import keras