CODEWITHSUNDEEP
amazon-web-servicesamazon-s3
farjam
farjam

Reputation: 2289

Combine all csv files into one single csv inside AWS CLI

I have a S3 bucket with a folder called batches. Inside the batches folder, I have 20 CSV files. Using AWS ClI (or a bash file to be exact), how can I combine all these csv files into a single CSV file and move it up one folder.

Normally in terminal this is how I do it:

cd batches && cat ./*csv > combined.csv

What would be a comparable way to do this for an S3 bucket inside AWS CLI?

Upvotes: 0

Views: 1348

Answers (1)

jellycsc
jellycsc

Reputation: 12259

If you only have 20 CSV files in total, then Marcin's suggestion is probably the right way to go. If you have more bigger-sized CSV, then I would suggest taking advantage of multipart upload in S3 and processing them in aws lambda.

Upvotes: 1

Related Questions