CODEWITHSUNDEEP
javawindows-7hadoop
Benjamin Crouzier
Benjamin Crouzier

Reputation: 41875

Task fails in hadoop because of "Type mismatch in key from map"

I have been folowing this tutorial for setting up hadoop on a single windows machine. Both the NameNode and JobTracker services are working respectively on http://localhost:50070 and http://localhost:50030. When I start my Map/Reduce driver on the local hadoop server (right click on TestDriver.java > run as > Run on Hadoop), I have the folowing output:

12/01/11 20:04:11 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
12/01/11 20:04:11 INFO mapred.FileInputFormat: Total input paths to process : 5
12/01/11 20:04:11 INFO mapred.JobClient: Running job: job_201201112003_0001
12/01/11 20:04:12 INFO mapred.JobClient:  map 0% reduce 0%

12/01/11 20:04:21 INFO mapred.JobClient: Task Id : attempt_201201112003_0001_m_000000_0, Status : FAILED
java.io.IOException: Type mismatch in key from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.LongWritable
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:558)
    at org.apache.hadoop.mapred.lib.IdentityMapper.map(IdentityMapper.java:37)
    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
    at org.apache.hadoop.mapred.Child.main(Child.java:158)

[Previous block is repeated 2 times, the task id changes]

java.io.IOException: Job failed!
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1232)
    at driverPkg.TestDriver.main(TestDriver.java:42)

Do you have an idea of what's going on ?

Upvotes: 0

Views: 889

Answers (2)

Snake Eye
Snake Eye

Reputation: 535

You can use KeyValueTextInput format instead of TextInputFormat to temporarily resolve the issue.

Have a look at following link for detailed example.

http://sanketraut.blogspot.in/2012/06/hadoop-example-setting-up-hadoop-on.html

Peace. Sanket Raut

Upvotes: 0

Chris Shain
Chris Shain

Reputation: 51329

This is the relevant line:

Type mismatch in key from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.LongWritable

It looks like your Mapper is defined with a TextWritable output (look in the <> in the Mapper class and the Map method), but you are outputting a LongWritable. Either change the definition, or what you are writing.

Upvotes: 1

Related Questions