CODEWITHSUNDEEP
rtestthat
mb147
mb147

Reputation: 83

Is there a way to document testthat tests and get a report output?

I need to document the tests that have been completed against a package I've written.

I want to be able to produce a report that gives details of the tests that were run, which an end user can look at to determine if there are gaps in the testing. End user is a committee, so needs something non-code.

What I was picturing was adding something akin to a roxygen block for each test to explain what it's doing. Within that I want to assign tests to particular groups based on what they're testing, e.g. bad inputs, expected warnings etc. And obviously would include details of which function being tested etc. Some of my tests are also more like integration tests, which test the high-level flow of functions is correct.

Is there a way to achieve that with RStudio and testthat?

Doesn't have to be HTML, but I figured that would be easiest to repackage for downstream use.

Only thought I've had so far is to use 'context' calls, but in latest version of testthat it's deprecated, so looking for a more supported solution.

All help appreciated.

Upvotes: 2

Views: 1504

Answers (1)

mb147
mb147

Reputation: 83

I'm still interested in a more elegant solution, but here's a minimal example of what I came up with, which serves my purpose in the short-term.

Summary

  • Used multiple context calls within my test files to partition my tests into meaningful groups.
  • Used the ListReporter to get results in a form that I can parse them.
  • Used a RMarkdown report to parse the results into HTML, split by file and context.

Example test file (testUnitTests.R)

context("Check unit-testing functionality")
test_that("Succeed", {succeed()})
test_that("Fail", {fail()})

context("Check unit-testing condition reporting")
test_that("Error", {stop("Force an error")})
test_that("Warn+succeed", {warning("Force a warning, but succeed"); succeed()})
test_that("Warn+fail", {warning("Force a warning, and fail"); fail()})

Example Markdown file (TestReport.Rmd)

---
title: "Title"
output: html_document
---
```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = FALSE, message = FALSE, results = "asis")
library(dplyr)

SummariseResults <- function(res) {
  res %>% summarise_if(is.numeric, sum) %>% knitr::kable()
}
WriteResults <- function(res) {
  purrr::pwalk(list(res$file, res$Categories), WriteFileResults)
}
WriteFileResults <- function(file, categories, ...) {
  cat("###", file, "\n")
  purrr::pwalk(list(categories$context, categories$Results), WriteCategoryResults)
}
WriteCategoryResults <- function(category, tests, ...) {
  cat("####", category, "\n")
  print(knitr::kable(tests))
}
```

```{r GenerateResults}
testResultsRaw <- testthat::test_file("testUnitTests.R", reporter = testthat::ListReporter)

testResultsIndividual <- testResultsRaw %>%
  as_tibble() %>%
  rename(Test = test) %>%
  group_by(file, context, Test) %>%
  summarise(NumTests = first(nb),
            Passed   = sum(passed),
            Failed   = sum(failed),
            Warnings = sum(warning),
            Errors   = sum(as.numeric(error)),
            Skipped  = sum(as.numeric(skipped)),
            .groups = "drop")

testResultsNested <- testResultsIndividual %>%
  nest_by(file, context, .key = "Results") %>%
  nest_by(file, .key = "Categories")
```

## Summary
```{r WriteSummary}
SummariseResults(testResultsIndividual)
```

## Results
```{r WriteResults}
WriteResults(testResultsNested)
```

Upvotes: 3

Related Questions