CODEWITHSUNDEEP
javaamazon-s3apache-flinkparquetflink-table-api
Lior Livshits
Lior Livshits

Reputation: 41

How to read Parquet files from S3 Ceph in Java using Flink 1.16.0 Table API?

I am trying to read Parquet files from S3 Ceph in Java using flink 1.16.0 and Table API FileSystem connector but encountering the following error:

java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration

As written in Flink’s documentation, I used the following dependencies in my pom.xml:

<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-parquet</artifactId>
  <version>1.16.0</version>
</dependency>

<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-connector-files</artifactId>
  <version>1.16.0</version>
</dependency>

Following the mentioned error I tried to add some Hadoop dependencies in order to resolve this exception but it still not working and looks like the solution should be simpler.

Has anyone encountered a similar problem?

Thank you in advance for your help!

Upvotes: 4

Views: 218

Answers (0)

Related Questions