CODEWITHSUNDEEP
javagoogle-app-enginegoogle-cloud-storage
c f
c f

Reputation: 196

Unable to copy folders using Google Cloud Storage JSON API

I have a GAE application created using Java. I have a bucket with folders and files that I'm trying to copy over to another bucket within the same project. The folder structure of my bucket is as follows:

Bucket_test
    |-------Folder1/
                |-----testtxt1.txt
    |-------Folder2/
                |-----testtxt2.txt

I am trying to copy the objects over using the following code

String old_bucket = "Bucket_test";
String new_bucket = "Bucket_test_new";

Bucket isbkt = new GoogleStorageHelper()
.tryCreateBucket(new_bucket, storage);
Storage.Objects.List listObjects = storage.objects().list(old_bucket);

Objects objects;
objects = listObjects.execute(); 

for (StorageObject object
: objects.getItems()) {
Storage.Objects.Copy copyObject 
= 
storage.objects()
.copy(old_bucket, object.getName(), new_bucket, object.getName(), object);

try {
System.out.println("Trying to copy over " + object.getName() + " from " + old_bucket + " >>>> " + new_bucket);
//copy the file over to the new bucket
Object copyRes = copyObject.execute();
System.out.println(copyRes.toString());
}
catch (Exception e) {
System.out.println("Exception trying to copy over " + object.getName() + " " + e.getLocalizedMessage());
    }

What is happening? I get the following error:

{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Invalid bucket name: 'Bucket_test/Folder1'.",
"reason" : "invalidParameter",
"extendedHelp" : "https://developers.google.com/storage/docs/bucketnaming"
 } ],
 "message" : "Invalid bucket name: 'Bucket_test/Folder1'."
  }
}

What should happen instead?

I expect it to copy files and folders over because both are considered Objects. I was successfully able to copy files over if they were directly in the bucket and not inside any folders.

  1. Is there another way to copy files and folders over?
  2. Do I need to list them using prefix and delimiters SRC: https://cloud.google.com/storage/docs/json_api/v1/objects/list?

Upvotes: 1

Views: 1241

Answers (1)

David
David

Reputation: 5519

In Cloud Storage, folders are not considered objects. As stated in the documentation :

Because the Google Cloud Storage system has no notion of folders, folders created in the Google Developers Console are a convenience to help you organize objects in a bucket.

So what happens is if your object name follows the pattern "part1/part2/part3" then the browser will show you two folders "part1" and "part2" and then a file called "part". But what's really stored in the system is just a file called "part1/part2/part3".

As a result, copying a folder does not mean anything to Cloud Storage since folders don't exist. What you probably want to do is copying all the files within the folder "folder1". In that case you will need to :

  • List all the objects whose name start with "folder1/"
  • For each of those objects, issue a copy command

You can also use gsutil who automates that for you.

Upvotes: 2

Related Questions