CODEWITHSUNDEEP
pythongoogle-cloud-platformgnupg
RobTheRobot16
RobTheRobot16

Reputation: 341

GCP & GPG - Stream File From Cloud Storage & Upload To Cloud Storage

This is a little similar to this question previously asked: gnupg - decrypt into Python bytesio stream

I need to stream files from GCP Cloud Storage, encrypt them within a stream using GPG, and simultaneously upload the encrypted data to GCP Cloud Storage. Is this possible?

Reading the API documentation (https://googleapis.dev/python/storage/latest/blobs.html), I can stream a file using:

from google.cloud import storage
client = storage.Client()
bucket = client.bucket("bucket-name")
blob = bucket.get_blob("blob-name.txt")
with blob.open("rt") as f:
    print(f.read())

I can encrypt a local file using:

with open('Test.txt', 'rb') as f:
                    status = gpg.encrypt_file(
                        f,sign=public_key_fingerprint,
                        recipients=private_key_recipient,
                        passphrase=private_key_pass,
                        always_trust=True,
                        output = output_file
                    )

And I can stream the GCP file to a local .gpg file during encryption using:

with blob.open("rb") as f:
             status = gpg.encrypt_file(
                    f,sign=public_key_fingerprint,
                    recipients=private_key_recipient,
                    passphrase=private_key_pass,
                    always_trust=True,
                    output = output_file
                )

But I need the output to be the file within Cloud Storage, and not the local system.

I tried:

bucket = storage_client.bucket(input_bucket) 
blob = bucket.get_blob(input_file)
uploadbucket = storage_client.bucket(output_bucket)
uploadblob = bucket.blob(output_file)
with blob.open("rt") as f:
    with uploadblob.open("wt") as en:
        status = gpg.encrypt_file(
            f,sign=public_key_fingerprint,
            recipients=private_key_recipient,
            passphrase=private_key_pass,
            always_trust=True,
            output = en
        )

But that gave me the error:

path should be string, bytes, os.PathLike or integer, not TextIOWrapper

Any help with this would be greatly appreciated!

Upvotes: 1

Views: 1764

Answers (1)

Iñigo González
Iñigo González

Reputation: 3955

The problem lies in the gpg.encrypt_file(..., output=en)call. The output parameter must be string with the destination of the content.

You can use gpg.encrypt() instead, then write the content to the blob, like this:

with blob.open("rt") as f:
    with uploadblob.open("wt") as en:
        encrypted = gpg.encrypt(
            f,sign=public_key_fingerprint,
            recipients=private_key_recipient,
            passphrase=private_key_pass,
            always_trust=True,
            armor=True)
        en.write(encrypted)

(I've used armor=True explicitly to remember that the output is ascii-armored output, not a binary).

Upvotes: 1

Related Questions