CODEWITHSUNDEEP
javaspringgoogle-cloud-storage
codem1966
codem1966

Reputation: 33

Setting GOOGLE_APPLICATION_CREDENTIALS in Spring

I'm attempting to use Spring to access files from Google Storage Buckets with the end goal of using MultiResourceItemReader to read in multiple XML files from the bucket. I currently have Spring working with this process when the XML files are locally on my machine (not GCP)

Now, I want to do the same thing, but instead of XML files on my machine, the files are in a GCP Storage bucket. I can access the bucket contents outside of Spring, one file at at time. For example this little bit of test code allows me to get access to the bucket and then see the files in the bucket. In this snippet, I setup the credentials via the JSON key file. (not an environment variable) 

public static void storageDriver() throws IOException {
// Load credentials from JSON key file. If you can't set the GOOGLE_APPLICATION_CREDENTIALS
// environment variable, you can explicitly load the credentials file to construct the
// credentials.

    String name = "";
    String bucketName = "";
    String bucketFileName = "";
    String bucketFullPath = "";
    Resource myBucker;
    GoogleCredentials credentials;
    File credentialsPath = new File("mycreds.json");  
    try (FileInputStream serviceAccountStream = new FileInputStream(credentialsPath)) {
        credentials = ServiceAccountCredentials.fromStream(serviceAccountStream);
    }


    Storage storage = StorageOptions.newBuilder()
            .setCredentials(credentials)
            .setProjectId("myProject")
            .build()
            .getService();

    for (Bucket bucket:storage.list().iterateAll()){
        if(bucket.getName().equalsIgnoreCase("myGoogleBucket")){
            bucketName = bucket.getName();
            System.out.println(bucket);
            for (Blob blob : bucket.list().iterateAll()){
                bucketFileName = blob.getName();
                bucketFullPath = "gs://"+bucketName+"/"+bucketFileName;
                System.out.println(bucketFullPath);

            }
        }
    };

However, when I try to do the following with Spring, Spring complains that I don't have the GOOGLE_APPLICATION_CREDENTIALS defined. (which of course I don't since I'm doing it programmatically.

For example, I'll add

@Value("gs://myGoogleBucket")
private Resource[] resources;

The Application Default Credentials are not available. They are available if running in Google Compute Engine. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials.

Upvotes: 2

Views: 7797

Answers (2)

Mohammed Mukhtar
Mohammed Mukhtar

Reputation: 1741

I tried many ways, but at last this excerpt from spring docs is the one which worked for me:

Due to the way logging is set up, the GCP project ID and credentials defined in application.properties are ignored. Instead, you should set the GOOGLE_CLOUD_PROJECT and GOOGLE_APPLICATION_CREDENTIALS environment variables to the project ID and credentials private key location, respectively. You can do this easily if you’re using the Google Cloud SDK, using the gcloud config set project [YOUR_PROJECT_ID] and gcloud auth application-default login commands, respectively.

Upvotes: 0

João André Martins
João André Martins

Reputation: 371

Spring Cloud GCP simplifies your GCS configuration.

You can add Storage support to your app. Then, either specify the location of your service account credentials through the spring.cloud.gcp.storage.credentials.location property, or by logging in with application default credentials using the Google Cloud SDK.

This will automatically provide you with a fully configured Storage object and things like @Value(gs://YOUR-BUCKET/YOUR-FILE) should just work.

Upvotes: 1

Related Questions