CODEWITHSUNDEEP
javajsonmavenhadoopparquet
TalO
TalO

Reputation: 11

from java objects to parquet file

I have a json file and want to convert it to parquet file. I know how to convert from json file to java objects but still need the conversion from java objects to parquet file. I don't want to convert using AVRO/ Protocol Buffers/Thrift/etc.. and also, don't want to use the Apache drill- just a java code.

Im using Java project with maven, my pom.xml look like this:

<dependencies>
        <dependency>
            <groupId>org.apache.parquet</groupId>
            <artifactId>parquet-common</artifactId>
            <version>1.8.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.parquet</groupId>
            <artifactId>parquet-encoding</artifactId>
            <version>1.8.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.parquet</groupId>
            <artifactId>parquet-column</artifactId>
            <version>1.8.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.parquet</groupId>
            <artifactId>parquet-hadoop</artifactId>
            <version>1.8.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-common</artifactId>
            <version>2.2.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-core</artifactId>
            <version>0.20.2</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-client</artifactId>
            <version>2.2.0</version>
        </dependency>
    </dependencies>

Means I don't have a problem using hadoop.

So I found this link (the last comment):

https://groups.google.com/forum/#!topic/parquet-dev/lfWanFOc040

The above link has what I want, but I have a problem with 2 imports there and can't find the correct dependencies for them.

Those are the 2 imports I can't find:

import org.apache.hadoop.hive.ql.io.parquet.writable.BinaryWritable;
import org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriteSupport;

will appreciate your help for finding the dependencies.

THANX:)

Upvotes: 0

Views: 1287

Answers (1)

TalO
TalO

Reputation: 11

OK I found the answer:

<dependency>
            <groupId>org.apache.hive</groupId>
            <artifactId>hive-exec</artifactId>
            <version>0.13.0</version>
        </dependency>

enjoy!

Upvotes: 1

Related Questions