CODEWITHSUNDEEP
pythontensorflowkerasdeep-learning
gtbono
gtbono

Reputation: 483

How to get Keras model predicted text back into list of words?

I'm trying to built an Autoencoder neural network for finding outliers in a single column list of text. The text input is like the following:

about_header.png
amaze_header_2.png
amaze_header.png
circle_shape.xml
disableable_ic_edit_24dp.xml
fab_label_background.xml
fab_shadow_black.9.png
fab_shadow_dark.9.png
fab_shadow_light.9.png
fastscroller_handle_normal.xml
fastscroller_handle_pressed.xml
folder_fab.png

The problem is that I don't really know what I'm doing, I'm using Keras, and I've converted these lines of text into a matrix using the Keras Tokenizer, so they can be fed into Keras Model so I can fit and predict them.

The problem is that the predict function returns what I believe is a matrix, and I can't really know for sure what happened because I can't convert the matrix back into the list of text like I originally had.

My entire code is as follows:

import sys
from keras import Input, Model
import matplotlib.pyplot as plt
from keras.layers import Dense
from keras.preprocessing.text import Tokenizer

with open('drawables.txt', 'r') as arquivo:
    dados = arquivo.read().splitlines()

tokenizer = Tokenizer(filters='', nb_words=None)
tokenizer.fit_on_texts(dados)

x_dados = tokenizer.texts_to_matrix(dados, mode="count")

tamanho = len(tokenizer.word_index) + 1
tamanho_comprimido = int(tamanho/1.25)

x = Input(shape=(tamanho,))

# Encoder
hidden_1 = Dense(tamanho_comprimido, activation='relu')(x)
h = Dense(tamanho_comprimido, activation='relu')(hidden_1)

# Decoder
hidden_2 = Dense(tamanho, activation='relu')(h)
r = Dense(tamanho, activation='sigmoid')(hidden_2)

autoencoder = Model(input=x, output=r)

autoencoder.compile(optimizer='adam', loss='mse')

history = autoencoder.fit(x_dados, x_dados, epochs=25, shuffle=False)

plt.plot(history.history["loss"])
plt.ylabel("Loss")
plt.xlabel("Epoch")
plt.show()

encoded = autoencoder.predict(x_dados)

result = ???????

Upvotes: 1

Views: 1441

Answers (1)

xashru
xashru

Reputation: 3580

You can decode the text using original encoding tokenizer.sequences_to_texts. This accepts a list of integer sequences. To get the sequences you can use np.argmax. 

encoded_argmax  = np.argmax(encoded, axis=1)
text = tokenizer.sequences_to_texts([encoded_argmax])  # since your output is just a number needs to convert into list

Upvotes: 2

Related Questions