CODEWITHSUNDEEP
google-cloud-dataflowapache-beam
Tadayasu Yotsu
Tadayasu Yotsu

Reputation: 159

How to coexist dataflow sdk and aws sdk

I'm trying to write S3 custom sink by using Dataflow Java SDK. Although AWS SDK uses httpclient4.5.2, Dataflow SDK still uses 4.0.1. So it's conflict when I try to run the job. I guess to upgrade the version of Dataflow's httpclient is the way to solve it. How can I update it?

Thank you.

Upvotes: 0

Views: 286

Answers (1)

Frances
Frances

Reputation: 4021

The HttpClient dependency comes via Dataflow SDK 1.8.0 -> google-api-client 1.22.0 -> google-oauth-client 1.22.0 -> google-http-client 1.22.0 -> httpclient 4.0.1

According the the documentation for Google Http Client, it needs HttpClient 4.0.1 or compatible.

Which version of HttpClient is Maven resolving to? (See mvn dependency:tree -Dverbose.) If it's 4.0.1, you might be able to force 4.5.2, assuming they are compatible. (Semantic versioning would imply so.)

Upvotes: 1

Related Questions