CODEWITHSUNDEEP
google-app-enginegoogle-cloud-dataflow
user2846616
user2846616

Reputation: 31

Dataflow on AppEngine giving access denied

I am trying to run my dataflow program on appEngine but when i run it locally using mvn appengine:deveserver , it gives below access denied exception

[INFO] WARNING: Error for /AppEngineDataFlow
[INFO] org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] access denied ("java.io.FilePermission" "C:\Users\KAPIL~1.KUM\AppData\Local\Temp" "read")
[INFO]  at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:159)
[INFO]  at org.xerial.snappy.Snappy.<clinit>(Snappy.java:47)

I am trying to send pipeline arguments as below:

 String[] args = new String[]{"--project=adtech-core","--stagingLocation=gs://adtech-nyt-net/Ad-Delivery/test", "--runner=BlockingDataflowPipelineRunner", "--numWorkers=10", "--filesToStage=gs://adtech-nyt-net/Ad-Delivery/stage","--maxNumWorkers=15", "--inputDate=20160617_2"}

ProcessDFPLogFilesOptions options = PipelineOptionsFactory.fromArgs(args).withValidation().as(ProcessDFPLogFilesOptions.class);     
Pipeline pipeline = Pipeline.create(options);
PCollection<String> dfp_network_data_log = pipeline.apply(TextIO.Read.named("ReadLines").from(_bucket_location).withCompressionType(TextIO.CompressionType.AUTO));

Upvotes: 1

Views: 219

Answers (1)

Sam McVeety
Sam McVeety

Reputation: 3214

The standard AppEngine environment lacks the dependencies necessary to run the Dataflow Java SDK. As recommended in Scheduling Dataflow pipelines using App Engine Cron Service or Cloud Functions, you will want to use the flexible environment for running on AppEngine.

Upvotes: 1

Related Questions