CODEWITHSUNDEEP
google-cloud-platformyamlmd5crc32
Kaushik J
Kaushik J

Reputation: 1072

python, google cloud platform: unable to overwite a file from google bucket: CRC32 does not match

I am using python3 client to connect to google buckets and trying to the following

Here is the code that i used

from google.cloud import storage
import yaml

client = storage.Client()
bucket = client.get_bucket('bucket_name')
blob = bucket.blob('my_rules_file.yaml')
yaml_file = blob.download_as_string()

doc = yaml.load(yaml_file, Loader=yaml.FullLoader)
doc['email'].clear()
doc['email'].extend(["[email protected]"])
yaml_file = yaml.dump(doc)

blob.upload_from_string(yaml_file, content_type="application/octet-stream")

This is the error I get from the last line for upload

BadRequest: 400 POST https://storage.googleapis.com/upload/storage/v1/b/fc-sandbox-datastore/o?uploadType=multipart: {
  "error": {
    "code": 400,
    "message": "Provided CRC32C \"YXQoSg==\" doesn't match calculated CRC32C \"EyDHsA==\".",
    "errors": [
      {
        "message": "Provided CRC32C \"YXQoSg==\" doesn't match calculated CRC32C \"EyDHsA==\".",
        "domain": "global",
        "reason": "invalid"
      },
      {
        "message": "Provided MD5 hash \"G/rQwQii9moEvc3ZDqW2qQ==\" doesn't match calculated MD5 hash \"GqyZzuvv6yE57q1bLg8HAg==\".",
        "domain": "global",
        "reason": "invalid"
      }
    ]
  }
}
: ('Request failed with status code', 400, 'Expected one of', <HTTPStatus.OK: 200>)

why is this happening. This seems to happen only for ".yaml files".

Upvotes: 1

Views: 711

Answers (1)

Paddy Popeye
Paddy Popeye

Reputation: 1824

The reason for your error is because you are trying to use the same blob object for both downloading and uploading this will not work you need two separate instances... You can find some good examples here Python google.cloud.storage.Blob() Examples

You should use a seperate blob instance to handle the upload you are trying with only one...

.....
blob = bucket.blob('my_rules_file.yaml')
yaml_file = blob.download_as_string()
.....

the second instance is needed here

....
blob.upload_from_string(yaml_file, content_type="application/octet-stream")
...

Upvotes: 3

Related Questions