CODEWITHSUNDEEP
amazon-s3pentahokettle
galusben
galusben

Reputation: 6402

Pentaho - upload list of files to Amazon s3

I am looking for a way to upload a list of files to Amazon S3. I have tried this: http://open-bi.blogspot.co.il/2010/03/kettel-job-plugin-send-files-to-amazon.html But it did not work for me. I am using ketle 5. I would prefer a transformation step, but a job step would also be great.

Thanks

Upvotes: 3

Views: 2871

Answers (2)

Naim Shaik
Naim Shaik

Reputation: 11

Install AWS CLI in your system and then using shell script step (job level step) just write the shell script to copy the files to S3.

Copy Single File to AWS S3 Bucket : aws s3 cp file.txt s3://

AWS S3 Copy Multiple Files : aws s3 cp s3:// –recursive

To know more use the link : https://docs.aws.amazon.com/cli/latest/userguide/cli-services-s3-commands.html

Upvotes: 1

bytor99999
bytor99999

Reputation: 1001

I am looking for the same thing. I think the best solution might be to use ftp? I think you can send files to S3 with ftp.

In my scenario, I also have to move and rename the files before uploading. So we have the path to the file and filename, we can use a FileExists step to make sure it exists first. Then run the move and rename file. Then I was going to try with an sftp step to upload the entire directory of tiles up to Amazon.

Upvotes: 1

Related Questions