CODEWITHSUNDEEP
amazon-web-servicesamazon-s3stream
Madhav Patekar
Madhav Patekar

Reputation: 31

Writing to file in S3 Bucket

I want to create a file in my S3 bucket.

I have a list which has some 10,000 strings in single iteration, I want to write those to a S3, clear the list. Then, in the second iteration, list is populated again with 10,000 entries, is it possible to write these new 10,000 entries to the same file in my S3 bucket ? How can I store these entries in S3 without storing it on local machine ?

Upvotes: 2

Views: 4949

Answers (1)

John Rotenstein
John Rotenstein

Reputation: 270114

There are several ways to store objects in Amazon S3.

The simplest is to copy a local file to S3, which can be done programmatically or with the AWS Command-Line Interface (CLI). For example:

aws s3 cp foo.txt s3://my-bucket/foo.txt

The aws s3 cp command also has the ability to take input from stdin and send output to stdout. So, if you have a program outputting text to stdout, you could store it in S3 with:

./myapp | aws s3 cp - s3://my-bucket/foo.txt

See: Uploading a local file stream to S3 in AWS S3 CP documentation.

Alternatively, you could write objects to Amazon S3 directly from your application by using an AWS SDK for your preferred language. This could, for example, stream the data into an Amazon S3 object without having to write it to a local disk first.

Upvotes: 3

Related Questions