CODEWITHSUNDEEP
javahadoopmapreduce
kliangh
kliangh

Reputation: 37

Add data into HashMap with loop

I'm trying to put the data which is read from sequence file, and put it into Hash Map. After the loop finished, I try to print the content which is wrong.

I've tried to just print the key and value in the first loop, and the result is correct. When I try to print the key in the second while loop, the result is several duplicate records. I can't figure it out what is going wrong.

    while(reader.next(key, value)) {
        byte[] value_bytes = value.getBytes();
        data_HashMap.put(key, value_bytes);
    }

    IOUtils.closeStream(reader);

    Iterator<Text> keySetIterator = data_HashMap.keySet().iterator();
    while(keySetIterator.hasNext()){
      Text index = keySetIterator.next();
      System.out.println("key: " + index);
    }

here is the result

Key: 123
Key: 123
Key: 123
Key: 123
Key: 123
Key: 123

If I modified the first while loop like this

while(reader.next(key, value)) {
    byte[] value_bytes = value.getBytes();
    System.out.println("Key: " + key);
}

Here is the result and it's correct.

Key: 123
Key: 456
Key: 789
Key: 741
Key: 852
Key: 963

Upvotes: 1

Views: 2429

Answers (1)

Thomas Jungblut
Thomas Jungblut

Reputation: 20969

You are reusing the same key all over again:

while(reader.next(key, value)) {
    byte[] value_bytes = value.getBytes();
    data_HashMap.put(**key**, value_bytes);
}

I don't know what the type is, but if it's Text, simply copy it or use String instead.

while(reader.next(key, value)) {
    byte[] value_bytes = value.getBytes();
    data_HashMap.put(key.toString(), value_bytes);
}

You will hit the same issue with the value bytes as well, so I recommend you to do a defensive copy of this array too.

Upvotes: 2

Related Questions