CODEWITHSUNDEEP
pythonmachine-learningscikit-learnnlptext-classification
Flanimal
Flanimal

Reputation: 56

Deploying a text classification model on new (unseen) text

I am working on a text classification problem. I have attached a simple dummy snippet of a text classification model I have trained.

How do I deploy the model on new_text? When the model is used on check_predictions, it classifies text correctly, however, when new data is used, the classification is incorrect.

Is this because the new_text would need to be vectorised? Am I missing something fundamental?

from collections import Counter
from sklearn.naive_bayes import MultinomialNB
from sklearn.metrics import accuracy_score
import pandas as pd
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from sklearn.metrics import accuracy_score, precision_score, recall_score

df = pd.read_csv("/Users/veg.csv")
print (df)

first 15 rows of df

X_train, X_test, y_train, y_test = train_test_split(df['Text'], df['Label'],random_state=1, test_size=0.2)
cv = CountVectorizer()

X_train_vectorized = cv.fit_transform(X_train)
X_test_vectorized = cv.transform(X_test)

naive_bayes = MultinomialNB()
naive_bayes.fit(X_train_vectorized, y_train)
predictions = naive_bayes.predict(X_test_vectorized)

print("Accuracy score: ", accuracy_score(y_test, predictions))
print('accuracy %s' % accuracy_score(predictions, y_test))
print(classification_report(y_test, predictions))

Output

check_predictions = []
for i in range(len(X_test)):   
    if predictions[i] == 0:
        check_predictions.append('vegetable')
    if predictions[i] == 1:
        check_predictions.append('fruit')
    if predictions[i] == 2:
        check_predictions.append('tree')
        
dummy_df = pd.DataFrame({'actual_label': list(y_test), 'prediction': check_predictions, 'Text':list(X_test)})
dummy_df.replace(to_replace=0, value='vegetable', inplace=True)
dummy_df.replace(to_replace=1, value='fruit', inplace=True)
dummy_df.replace(to_replace=2, value='tree', inplace=True)
print("DUMMY DF")
print(dummy_df.head(10))

test df

new_data=['carrot', 'grapes',
          'banana', 'potato',
          'birch','carrot', 'grapes',
          'banana', 'potato', 'birch','carrot','grapes',
          'banana', 'potato',
          'birch','carrot', 'grapes',
          'banana', 'potato', 'birch','grapes',
          'banana', 'potato', 'birch']

new_predictions = []
for i in range(len(new_data)):    
    if predictions[i] == 0:
        new_predictions.append('vegetable')
    if predictions[i] == 1:
        new_predictions.append('fruit')
    if predictions[i] == 2:
        new_predictions.append('tree')
        
new_df = pd.DataFrame({'actual_label': list(y_test), 'prediction': new_predictions, 'Text':list(new_data)})        
new_df.replace(to_replace=0, value='vegetable', inplace=True)
new_df.replace(to_replace=1, value='fruit', inplace=True)
new_df.replace(to_replace=2, value='tree', inplace=True)
print("NEW DF")
print(new_df.head(10))

New data df

Upvotes: 0

Views: 323

Answers (1)

desertnaut
desertnaut

Reputation: 60318

Whatever (new) text you are feeding into your model must go through the exact same preprocessing steps as your training data - here the CountVectorizer as already fitted with your X_train:

new_data_vectorized = cv.transform(new_data) # NOT fit_transform
new_predictions = naive_bayes.predict(new_data_vectorized)

Upvotes: 1

Related Questions