CODEWITHSUNDEEP
rtestthat
Lukasz
Lukasz

Reputation: 148

How to add a reporter to testthat?

I'm trying to add a JUnit reporter to testthat. While I can do it inside the package (https://github.com/hadley/testthat/pull/481) I cannot make it pass the same set of tests when in a separate package (https://github.com/lbartnik/testthatJUnit).

My main problem is that tests designed for reporters and present in testthat do not pass if I pull JUnit reporter code to a separate package. I'm guessing that there is something in those tests that "sets the scene" for testing reporters but I'm unable to identify this "something".

Any help much appreciated.

Details follow.

> library(devtools)
> test("testthatJUnit")
Loading testthatJUnit
Testing testthatJUnit
JUnitReporter: 

DONE ==========================================================================
> test("testthat")     
Loading testthat
unloadNamespace("testthat") not successful, probably because another loaded package depends on it.Forcing unload. If you encounter problems, please restart R.
Testing testthat
Bare expectations: 
Colours: ..
compare.character: .................
compare.numeric: .....................
compare.time: .......
# ... lines cut ...

DONE ===========================================================================
> test("testthatJUnit")
Loading testthatJUnit
Testing testthatJUnit
JUnitReporter: .............

If I modify the test to see why the first test("testthatJUnit") is silent I see this:

> test("testthatJUnit")
Loading testthatJUnit
Testing testthatJUnit
JUnitReporter: Bare: .1
Failed -------------------------------------------------------------------------
1. Error: Junit reporter regression (@test-reporter-junit.R#13) ----------------
attempt to apply non-function
1: test_dir("test_dir") at /mnt/storage-a/lukaszb/src/third-party/testthatJUnit/tests/testthat/test-reporter-junit.R:13
2: test_files(paths, reporter = reporter, env = env, ...)
3: with_reporter(reporter = current_reporter, results <- lapply(paths, test_file, env = env, 
       reporter = current_reporter, start_end_reporter = FALSE, load_helpers = FALSE))
4: force(code)
5: lapply(paths, test_file, env = env, reporter = current_reporter, start_end_reporter = FALSE, 
       load_helpers = FALSE)
6: FUN(X[[i]], ...)
7: with_reporter(reporter = reporter, start_end_reporter = start_end_reporter, {
       lister$start_file(basename(path))
       source_file(path, new.env(parent = env), chdir = TRUE)
       end_context()
   })  
8: force(code)
9: source_file(path, new.env(parent = env), chdir = TRUE)
10: eval(exprs, env)
11: eval(expr, envir, enclos)
12: expect_that(1, equals(1)) at test_dir/test-bare-expectations.R:3
13: condition(object)
14: expect_equal(x, expected, ..., expected.label = label)
15: expect(comp$equal, sprintf("%s not equal to %s.\n%s", lab_act, lab_exp, comp$message), 
       info = info)
16: withRestarts(if (expectation_broken(exp)) {
       stop(exp)
   } else {
       signalCondition(exp)
   }, continue_test = function(e) NULL)
17: withOneRestart(expr, restarts[[1L]])
18: doWithOneRestart(return(expr), restart)
19: signalCondition(exp)
20: (function (e) 
   {   
       register_expectation(e, frame + 11, sys.nframe() - 6)
       invokeRestart("continue_test")
   })(structure(list(message = "1 not equal to 1.\nEqual\n", srcref = NULL), .Names = c("message", 
   "srcref"), class = c("expectation_success", "expectation", "condition")))
21: register_expectation(e, frame + 11, sys.nframe() - 6)
22: get_reporter()$add_result(context = get_reporter()$.context, test = test, result = e)
23: o_apply(self$reporters, "add_result", context = context, test = test, result = result)
24: lapply(objects, function(x) x[[method]](...))
25: FUN(X[[i]], ...)
26: x[[method]](...)

But if I run test("testthat") and again test("testthatJUnit") everything comes back to normal:

> test("testthatJUnit")
Loading testthatJUnit
Testing testthatJUnit
JUnitReporter: Bare: ..
Basic: .......
empty: 1
error: 2.3456.
failures: 7.8..
helper: .
skip: S

My guess is that there is something in testthat's tests that sets up the scene for testing reporters but so far I haven't been able to identify it. Any help?

Upvotes: 0

Views: 1075

Answers (1)

Lukasz
Lukasz

Reputation: 148

Actually, it turned out there is a bug in testthat 1.0.2 that is already fixed in the github sources (https://github.com/hadley/testthat/commit/99c25fc4efefa012a36d8fc912210ba3452a978d). With this version of testthat all tests in my package pass with no errors.

Upvotes: 1

Related Questions