CODEWITHSUNDEEP
gogoogle-cloud-platformgoogle-cloud-storage
Cmag
Cmag

Reputation: 15770

google storage file download

Folks, Why would it matter how a publicly accessible object which is stored in a Google Storage Bucket is downloaded?

https://storage.cloud.google.com/convertedexcelfiles/test.png

Using tools such as wget or curl... seem to be garbling the file. 

$ wget https://storage.cloud.google.com/convertedexcelfiles/test.png

...
$ ls -all -h
56K Feb  2 01:32 test.png

Same goes for the trusty:

package main

import (
    "io"
    "net/http"
    "os"
)

func main() {

    fileUrl := "https://storage.cloud.google.com/convertedexcelfiles/test.png"

    if err := DownloadFile("test.png", fileUrl); err != nil {
        panic(err)
    }
}

// DownloadFile will download a url to a local file. It's efficient because it will
// write as it downloads and not load the whole file into memory.
func DownloadFile(filepath string, url string) error {

    // Get the data
    resp, err := http.Get(url)
    if err != nil {
        return err
    }
    defer resp.Body.Close()

    // Create the file
    out, err := os.Create(filepath)
    if err != nil {
        return err
    }
    defer out.Close()

    // Write the body to file
    _, err = io.Copy(out, resp.Body)
    return err
}

The only way that seems to be reliable is using gsutil or browsing to the gcp console. Thoughts?

Is it possibly because its returning a 302?

curl -I https://storage.cloud.google.com/convertedexcelfiles/test.png                                                                                                        ~/Downloads/transforms/tmp
HTTP/2 302
content-type: application/binary
location: https://accounts.google.com/ServiceLogin?service=cds&passive=1209600&continue=https://storage.cloud.google.com/convertedexcelfiles/test.png&followup=https://storage.cloud.google.com/convertedexcelfiles/test.png
content-length: 0
date: Sun, 02 Feb 2020 06:33:13 GMT
server: ESF
x-xss-protection: 0
x-frame-options: SAMEORIGIN
x-content-type-options: nosniff
alt-svc: quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000

How does one grab a reliable link if thats the case?

Upvotes: 2

Views: 1496

Answers (1)

Cmag
Cmag

Reputation: 15770

As folks have kindly mentioned, the contents of the file were html.... After a more careful look, the following is apparent:

Do not use the URL in the box here: dont use

But rather right-click on Public : use

As you can see, the 2 URLs are slightly different.

https://storage.cloud.google.com/convertedexcelfiles/test.png https://storage.googleapis.com/convertedexcelfiles/test.png

Second one works... I have a strange feeling that others, less technically savvy would have an awful time with this. At 3am this was infuriating. I should have simply catted the file... but who would have expected it! Great job on the UI google... the hell?

Upvotes: 6

Related Questions