Kite SDK: Getting error while Json-import

Question

I am facing an issue while importing a json using kite sdk.

Used command to generate avro schema:

./kite-dataset json-schema /vagrant/satyam/kite/restaurant-sample.json  -o sample.avsc --record-name HGW

Then passed command to create dataset in hdfs file system:

./kite-dataset create dataset:hdfs:/user/falcon/datasets/hgw --schema sample.avsc

To import the JSON file I am running following command:

./kite-dataset -v json-import /vagrant/satyam/kite/restaurant-sample.json dataset:hdfs:/user/falcon/datasets/hgw

Getting error:

1 job failure(s) occurred:
org.kitesdk.tools.CopyTask: Kite(dataset:file:/tmp/datasets/.temp/3759e9f8-7406-4ced-... ID=1 (1/1)(1): java.io.FileNotFoundException: File does not exist: hdfs://integration.mycorp.kom:8020/tmp/crunch-878994294/p1/REDUCE
at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1122)
at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1114)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1114)
at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:750)
at org.apache.hadoop.mapreduce.v2.util.MRApps.parseDistributedCacheArtifacts(MRApps.java:568)
at org.apache.hadoop.mapreduce.v2.util.MRApps.setupDistributedCache(MRApps.java:460)
at org.apache.hadoop.mapred.LocalDistributedCacheManager.setup(LocalDistributedCacheManager.java:93)
at org.apache.hadoop.mapred.LocalJobRunner$Job.<init>(LocalJobRunner.java:163)
at org.apache.hadoop.mapred.LocalJobRunner.submitJob(LocalJobRunner.java:731)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:536)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1293)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293)
at org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob.submit(CrunchControlledJob.java:329)
at org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchJobControl.startReadyJobs(CrunchJobControl.java:204)
at org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchJobControl.pollJobStatusAndStartNewOnes(CrunchJobControl.java:238)
at org.apache.crunch.impl.mr.exec.MRExecutor.monitorLoop(MRExecutor.java:112)
at org.apache.crunch.impl.mr.exec.MRExecutor.access$000(MRExecutor.java:55)
at org.apache.crunch.impl.mr.exec.MRExecutor$1.run(MRExecutor.java:83)
at java.lang.Thread.run(Thread.java:745)

Can anyone help me to understande what is getting wrong??? Thanks in advance

Answer 1

I think you are using Kite SDK 1.1.0 version. I also got same error when I was doing csv-import. When I switched to Kite SDK 1.0.0 version there was no such error.

I would suggest you to switch to Kite SDK 1.0.0 version.

Moreover there has been no new release of Kite SDK after 1.1.0 version and even this release happened in June 2015.