L.H
L.H

Reputation: 75

Use NiFi to write into AWS S3 Bucket with MFA enabled

Is it possible to use the NiFi processor PutS3Object to write data into a S3 Bucket, while MFA is active?

And if not, what would be an alternative considering writing data to S3 from hive/HDFS?

Upvotes: 1

Views: 840

Answers (1)

Andy
Andy

Reputation: 14194

As of version 1.8.0, I do not believe this is supported by the Apache NiFi *S3Object processors (or the AWSCredentialsProviderControllerService). I recommend opening a feature request ticket on the Jira site. You may be able to accomplish this immediately by using one of the following options:

  • Use ExecuteStreamCommand or ExecuteProcess to call a terminal command or shell script using the AWS S3 CLI
  • Use ExecuteScript to run a custom Python (/Ruby/Groovy/etc.) script which uses the AWS S3 SDK directly (example code)
  • Write a custom processor which uses the AWS S3 SDK directly

Upvotes: 2

Related Questions