CODEWITHSUNDEEP
javahadoopthread-safetymapreduce
ngrislain
ngrislain

Reputation: 1072

Has a Mapper class to be thread safe

To avoid allocating memory at every instance of a Hadoop Mapper class and then rely on the GC to clean this memory I use static objects that I access read and write in the map method.

Do I have to care about thread-safety ?

Upvotes: 0

Views: 1473

Answers (2)

Manish Malhotra
Manish Malhotra

Reputation: 80

Adding to what Enno and Quetzalcoatl has said. As When ever a map task runs in a JVM, it creates a single object of Mapper class, and calls map fn for each input ( i.e. for each row of a file when using TextInputFormat). But it calls the map fn in sequence and not in parallel. So, there should be no problem of Thread Safety of the objects you are using.

Regards, Manish

Upvotes: 1

Enno Shioji
Enno Shioji

Reputation: 26882

By default, each map task will run on its own JVM. So you won't get any benefit from using a static object. I would recommend not to do something weird unless you actually face performance issues.

Upvotes: 3

Related Questions