CODEWITHSUNDEEP
rubyjenkinscucumber
Steve
Steve

Reputation: 4635

Displaying number of tests which failed with a certain tag ( Ruby, Cucumber, in Jenkins)

I'm considering adding @high_priority and @low_priority to certain tests in our test suite in order to find out how many high priority (risk) tests have failed. Ideally I'd like a column in Jenkins next to the test job which displays

1/100 high priority and 8/60 low priority tests failed.

Though I'm happy with a similar output in the console output if necessary.

Currently Jenkins jobs are running a command line execution like:

cucumber --tags @AU_smoke ENVIRONMENT=beta --format html --out 'C:\git\testingworkspace\Reports\smoke_BETA_test_report.html'  --format pretty

Edit: Adding extra jobs isn't really a solution, we have a large amount of jobs which run subsets of all of the tests, so adding extra jobs for high and low priority would require tripling the number of jobs we have.

Upvotes: 2

Views: 822

Answers (1)

Steve
Steve

Reputation: 4635

I've settled on using the description setter plugin with the extra columns plugin. This allows me to add the build description as a column on my views and in my code I have 

After do |scenario|
  if scenario.status.to_s=="passed"
    $passed=$passed+1
  elsif scenario.status.to_s=="failed"
    $failed=$failed+1
    puts "FAILED!"
  elsif scenario.status.to_s=="undefined"
    $undefined=$undefined+1
  end
  $scenario_count=$scenario_count+1
  if scenario.failed?
    Dir::mkdir('screenshots') if not File.directory?('screenshots')
    screenshot = "./screenshots/FAILED_#{scenario.name.gsub(' ','_').gsub(/[^0-9A-    Za-z_]/, '')}.png"
    @browser.driver.save_screenshot(screenshot)
    puts "Screenshot created: #{screenshot}"
    embed screenshot, 'image/png'
    #@browser.close
  end
  #@browser.close
end

at_exit do
  end_time=Time.now

  elapsed_time=end_time.to_i - $start_time.to_i
  puts "\#description#scenarios total: #{$scenario_count}, passed: #{$passed}, failed:     #{$failed}, known bug fails: #{$known_bug_failures}, undefined: #{$undefined}.#description#"
...

Then in the description setter plugin I use the regex 

/#description#(.+)#description#/

and use the first match group as the build description name. This also allows me to look at a job's build history and see at a glance how many tests there were and how many passed over the previous few weeks.

Upvotes: 2

Related Questions