CODEWITHSUNDEEP
tensorflowtensorflow2.0
yanachen
yanachen

Reputation: 3753

How to get the labels from tensorflow dataset

ds_test = tf.data.experimental.make_csv_dataset(
    file_pattern = "./dfj_test/part-*.csv.gz",
    batch_size=batch_size, num_epochs=1, 
    #column_names=use_cols, 
    label_name='label_id',
    #select_columns= select_cols,
    num_parallel_reads=30, compression_type='GZIP',
    shuffle_buffer_size=12800)

This is my tesetset during training. After completing the model, I want to zip the columns of predictions and labels for the df_test .

preds = model.predict(df_test)

Getting the predictions is quite simple, and it is of numpy array format. However, I don't know how to get the corresponding labels from the df_test. I want to zip(preds, labels) for further analysis. Any hint? Thanks.

(tf version 2.3.1)

Upvotes: 0

Views: 2862

Answers (1)

Proko
Proko

Reputation: 2021

You can map each example to return the field you want

# load some exemplary data
TRAIN_DATA_URL = "https://storage.googleapis.com/tf-datasets/titanic/train.csv"
train_file_path = tf.keras.utils.get_file("train.csv", TRAIN_DATA_URL)
dataset = tf.data.experimental.make_csv_dataset(train_file_path, batch_size=100, num_epochs=1) 

# get field by unbatching 
labels_iterator= dataset.unbatch().map(lambda x: x['survived']).as_numpy_iterator()
labels = np.array(list(labels_iterator))

# get field by concatenating batches
labels_iterator= dataset.map(lambda x: x['survived']).as_numpy_iterator()
labels = np.concatenate(list(labels_iterator))

Upvotes: 1

Related Questions