CODEWITHSUNDEEP
google-cloud-platformgoogle-cloud-storagebigdatabinary-treebucket
SARIM
SARIM

Reputation: 1132

How to check whether a folder exists or not in gcp cloud storage out of 1 million folders

For example I have structure like this.

bucketname/checked/folder1/some files
bucketname/checked/folder2/some files
bucketname/checked/folder3/some files
bucketname/checked/folder4/some files
bucketname/checked/folder5/some files
bucketname/checked/folder6/some files
bucketname/checked/folder7/some files
bucketname/checked/folder8/some files
bucketname/checked/folder9/some files
bucketname/checked/folder10/some files
bucketname/checked/folder11/some files
......
......

bucketname/checked/folder-1million/some files

Now,

1. If I have to check whether folder99999 exists or not. So,what would be the best way to check it (we have information of folder name - folder99999) ?

2. If we simply check path that exists or not, and if not then it means, folder don't exists. would it work fine If we have millions of folders?

3. Which data structure gcp uses to retrieve the folder data ?

Upvotes: 0

Views: 4324

Answers (1)

guillaume blaquiere
guillaume blaquiere

Reputation: 76018

The true answer is this one provided by John: folder doesn't exist. All the files are stored at the root directory (bucket level) and the file name is the full path. By human convention, the / is the folder separator and the console display fake folders.

If you haven't files in a "folder", the "folder" doesn't exist, it's not interpreted/deduced from the name fully qualified path. The folder is not a Cloud Storage resource

It's also for that reason that you search only by path prefix

However, it depends what you want to check. If you exactly know which folder you want to check and validate, and if there is at least one file in it, you can directly list the files with the folder path as prefix.

Upvotes: 4

Related Questions