CODEWITHSUNDEEP
hadoop
Vladimir Kroz
Vladimir Kroz

Reputation: 5367

How to clear Hadoop jobs history

I'm using Hadoop Map/Reduce Administration UI coming with standard Apache distro. With the time the list of executed jobs is growing too big. How I can remove old jobs from the history list?

Upvotes: 1

Views: 3124

Answers (2)

Chris White
Chris White

Reputation: 30089

Another suggestion (more extreme) is just to restart your jobtracker.

Upvotes: 0

user1261215
user1261215

Reputation:

Hadoop job execution has life cycle, which is explained here

Job status flow
Running -> Completed -> Retired

There are important configurations using which we can control the job retention.

mapred.jobtracker.completeuserjobs.maximum: Number of jobs to keep in memory after Running state. Try configure to less value...

mapred.jobtracker.retirejob.interval: Default interval value is  24 * 60 * 60 * 1000 (1 Day). That means completed jobs will be retired after one day by default.

Try configuring property "mapred.jobtracker.retirejob.interval" to 1 hour or so.

Upvotes: 1

Related Questions