CODEWITHSUNDEEP
pythontensorflowkaggletpu
Talha Anwar
Talha Anwar

Reputation: 2949

UnimplementedError: File system scheme '[local]' not implemented

I am getting an error while implementing TensorFlow in TPU

UnimplementedError: File system scheme '[local]' not implemented (file: '1.png')

I know this question has been answered before but my issue is different, I am getting this error when I do

for i, j in train_dataset.take(3):
    print(i,j)

It works with train_dataset.take(3)

Here are my functions

def decode(img,image_size=(IMG_SIZE, IMG_SIZE)):
    bits = tf.io.read_file(img)
    image = tf.image.decode_jpeg(bits, channels=3)
    image = tf.cast(image, tf.float32) / 255.0
    image = tf.image.resize(image, image_size)
    
    image = tf.image.random_flip_left_right(image, seed=2020)
    image = tf.image.random_flip_up_down(image, seed=2020)
    image = tf.image.random_crop(image,size=[IMG_SIZE,IMG_SIZE,3],seed=2020 )
    image = tf.image.random_brightness(image,max_delta=0.5 )
    image = tf.image.rot90(image)
    return image

def decode_image(img,labels=None ):
    if labels is None:
        return decode(img)
    else:
        return decode(img),labels 

train_image=tf.data.Dataset.from_tensor_slices((train.iloc[:,0],train.iloc[:,1::] ))
train_dataset=train_image.map(decode_image, num_parallel_calls=AUTO).repeat().shuffle(512).batch(BATCH_SIZE).prefetch(AUTO)
test_image=tf.data.Dataset.from_tensor_slices((test.iloc[:,0]))
test_dataset=test_image.map(decode_image, num_parallel_calls=AUTO).batch(BATCH_SIZE)

How should I resolve it?

It might an issue with the path. SO I am adding how is set path This is how the directory looks like

weights
images
-train
--train
---train
----img1
----img2
---csv
-val
--val
---img1

When I run

GCS_DS_PATH = KaggleDatasets().get_gcs_path('images')
!gsutil ls $GCS_DS_PATH

I got following

gs://kds-aab923e1c9bc934f088881f1e537365b8f18fe192b3b3dc14e272a37/train/
gs://kds-aab923e1c9bc934f088881f1e537365b8f18fe192b3b3dc14e272a37/val/

This is how my paths are set

def train_format_path(st):
    return GCS_DS_PATH + '/train/train/train/' + st 

def test_format_path(st):
    return GCS_DS_PATH + '/val/val/' + st 

train_paths = train.ID.apply(train_format_path).values
test_paths = test.ID.apply(test_format_path).values

With train_paths[0] I got

'gs://kds-aab923e1c9bc934f088881f1e537365b8f18fe192b3b3dc14e272a37/train/train/train/1.png'

Upvotes: 2

Views: 2451

Answers (1)

Talha Anwar
Talha Anwar

Reputation: 2949

As suggested by @Allen Wang, the solution is to use train_paths instead of train to pass images.

This is what I have changes to make it work

    train_image=tf.data.Dataset.from_tensor_slices((train_paths,train.iloc[:,1::] ))
    train_dataset=train_image.map(decode_image, num_parallel_calls=AUTO).repeat().shuffle(512).batch(BATCH_SIZE).prefetch(AUTO)
    test_image=tf.data.Dataset.from_tensor_slices((test_paths))
    test_dataset=test_image.map(decode_image, num_parallel_calls=AUTO).batch(BATCH_SIZE)

Upvotes: 1

Related Questions