CODEWITHSUNDEEP
python-3.xgoogle-colaboratory
sandeep mahato
sandeep mahato

Reputation: 11

How to read a large 30GB tar.xz with colab

I downloaded a 30GB tar.xz file to my G-drive using Google Colab. I need help in extracting and reading this folder in Colab. Inside the tar folder, there are ten folders. Is it possible to read these folders individually? I have tried the following but it failed.

  1. Untar the 30GB folder in G-drive but it failed because of the limitations with reading and writing files in G-Drive.
  2. I can directly download the file to the local Colab directory, but because of the space limitations in Colab I cannot extract or read it in the local directory.

Any suggestion about how to proceed with this problem. Thank you

Upvotes: 1

Views: 1179

Answers (1)

korakot
korakot

Reputation: 40838

You can extract only a directory inside the tar file, using --wildcards option.

!tar xf file.tar.xz --wildcards 'path_to/dir/*'

Here's an example notebook.

Upvotes: 1

Related Questions