CODEWITHSUNDEEP
jenkins
Paul Praet
Paul Praet

Reputation: 1387

Difference between 'blocked', 'stuck', 'pending', 'buildable' jobs in Jenkins

I cannot find a definition for any of these common Jenkins terms anywhere.

I also find it strange that in the metrics plugin https://wiki.jenkins.io/display/JENKINS/Metrics+Plugin all "jenkins.queue..." metrics seems to pertain to blocked state.

Upvotes: 2

Views: 3331

Answers (2)

Martin McNulty
Martin McNulty

Reputation: 2631

To add to Mike's answer, I wanted to know how "too long" was defined for the jenkins.queue.stuck metric. Here's the relevant part of the source code:

        @Override
        public boolean isStuck() {
            Label label = getAssignedLabel();
            if (label != null && label.isOffline())
                // no executor online to process this job. definitely stuck.
                return true;

            long d = task.getEstimatedDuration();
            long elapsed = System.currentTimeMillis() - buildableStartMilliseconds;
            if (d >= 0) {
                // if we were running elsewhere, we would have done this build ten times.
                return elapsed > Math.max(d, 60000L) * 10;
            } else {
                // more than a day in the queue
                return TimeUnit.MILLISECONDS.toHours(elapsed) > 24;
            }
        }

So, by my reading of the code, an item is stuck in the queue if:

  • it needs a particular label and no executors with that label are online; or
  • it has been in the queue for more than the greater of ten times its estimated duration and one minute; or
  • it has been in the queue for more than 24 hours

Hope that helps someone else!

Upvotes: 0

Mike
Mike

Reputation: 615

Items in the queue go through several stages, as depicted below

(enter) --> waitingList --+--> blockedProjects
                           |        ^
                           |        |
                           |        v
                           +--> buildables ---> pending ---> left
                                    ^              |
                                    |              |
                                    +---(rarely)---+

jenkins.queue.blocked means that the number of jobs that are blocked by Jenkins. The cause of block can be anything from below

  1. target label is busy
  2. target node is offline
  3. target Node is busy
  4. because a node (or its retention strategy) is not accepting tasks
  5. target label is offline

jenkins.queue.stuck means that the job is stuck in the queue even though all Node resources are available or there is an issue with the executor (if the job is starving for an executor for too long.).

jenkins.queue.pending means that the job is waiting for the next executor to run the job and all the node resources are health and reserved for the job.

jenkins.queue.buildable returns the total jobs which are in the buildable stage

Reference: javadoc.jenkins.io/hudson/model/Queue.html

Upvotes: 4

Related Questions