adding annotation_custom with rasterGrob after function call

Question

Apologies for the title, I know it sucks.

I am trying to create a waterfall chart function. So, I am trying to create a basic plot, which people can configure however they wish. I ran into a problem, though, adding a gradient to the plot. For example:

I have this df:

> wfDF
                 category   value sign id    end  start labels
1         Basic Materials  0.0024  pos  1 0.0024 0.0000 0.0024
2          Communications  0.0492  pos  2 0.0516 0.0024 0.0516
3      Consumer, Cyclical  0.0268  pos  3 0.0784 0.0516 0.0784
4  Consumer, Non-cyclical  0.0245  pos  4 0.1029 0.0784 0.1029
5             Diversified -0.0037  neg  5 0.0992 0.1029 0.1029
6                  Energy -0.0040  neg  6 0.0952 0.0992 0.0992
7               Financial  0.0445  pos  7 0.1397 0.0952 0.1397
8              Industrial  0.0006  pos  8 0.1403 0.1397 0.1403
9              Technology -0.0059  neg  9 0.1344 0.1403 0.1403
10                  Total  0.1345  pos 10 0.0000 0.1344 0.1344

With this code:

ggplot(wfDF, aes(category, fill = sign, color = sign)) + guides(fill = FALSE, color=FALSE) +
  ggtitle("Risk by Industry") + 
  annotation_custom(g, xmin=-Inf, xmax=Inf, ymin=-Inf, ymax=Inf) +
  theme(plot.title = element_text(vjust=1.5, face="bold", size = 20),
        axis.title.x = element_blank(), axis.title.y = element_blank()) +
  geom_rect(aes(x = category, xmin = id - 0.475, xmax = id + 0.475, ymin = end, ymax = start)) +
  scale_fill_manual(values=c("red", "forestgreen")) + 
  scale_color_manual(values=c("black", "black")) + 
  scale_y_continuous(labels = percent) +     
  scale_x_discrete("", breaks = levels(wfDF$category), labels = gsub(" ", "\n", levels(wfDF$category))) + 
  geom_text(data = wfDF, aes(id, labels, label = paste0(value*100, "%")), vjust = -.5, size = 5, fontface = 4)

Which produces this graph:

Which looks great. I am trying to write a function which will do all this with any set of categories and values, and allows for any colors or customization to be added or used. I have this function:

waterfall <- function(categories, values, has.total = FALSE, offset = .475, labelType = c("decimal", "percent")) {
  library(scales)
  library(grid)
  library(ggplot2)
  library(dplyr)

  theData <- data.frame("category" = as.character(categories), "value" = as.numeric(values))
  if (labelType == "percent") theData$value = theData$value/100
  if (!has.total) theData <- theData %>% rbind(.,list("Total", sum(.$val)))
  theData$sign <- ifelse(theData$val >= 0, "pos","neg")
  theData <- data.frame(category = factor(theData$category, levels = unique(theData$category)),
                           value = round(theData$value,4),
                           sign = factor(theData$sign, levels = unique(theData$sign)))
  theData$id <- seq_along(theData$value)
  theData$end <- cumsum(theData$value)
  theData$end <- c(head(theData$end, -1), 0)
  theData$start <- c(0, head(theData$end, -1))
  theData$labels <- paste0(theData$value*100, "%")
  theData$labellocs <- pmax(theData$end,theData$start)

  theGG <- ggplot(theData, aes(category, fill = sign, color = sign)) + 
    geom_rect(aes(x = category, xmin = id - offset, xmax = id + offset, ymin = end, ymax = start)) +
    scale_x_discrete("", breaks = levels(theData$category), labels = gsub(" ", "\n", levels(theData$category))) + 
    geom_text(data = theData, aes(id, labellocs, label = labels), vjust = -.5, size = 5, fontface = 4)
    return(theGG)
}

 waterfall(categories = riskDecomp$ID, values = riskDecomp$val, labelType = "percent")

Which produces a pretty ugly basic thing:

However, if I try to run something like the following:

test <- waterfall(categories = riskDecomp$ID, values = riskDecomp$val, labelType = "percent")
g <- rasterGrob(blues9, width=unit(1,"npc"), height = unit(1,"npc"), interpolate = TRUE)
  test + guides(fill = FALSE, color=FALSE) +
  ggtitle("Risk Decomposition") + 
  annotation_custom(g, xmin=-Inf, xmax=Inf, ymin=-Inf, ymax=Inf) + 
  theme(plot.title = element_text(vjust=1.5, face="bold", size = 20),
        axis.title.x = element_blank(), axis.title.y = element_blank()) +
  scale_fill_manual(values=c("red", "forestgreen")) + 
  scale_color_manual(values=c("black", "black")) + 
  scale_y_continuous(labels = percent)

I get this nonsense:

The rasterGrob thing seems to overlay the entire rest of the plot. The only workaround I can find is to add the gradient to the inside of the function. Which kind of removes the... customization of the function. Is there a way to fix this? To fix the order of the grobs? If that makes sense?

Answer 1

you can change the order of the layers manually,

library(grid)
library(ggplot2)

g <- rasterGrob(matrix(blues9, ,1), interpolate=TRUE,
                width=unit(1,"npc"), height=unit(1,"npc"))

p <- qplot(rnorm(10), rnorm(10)) +
  annotation_custom(g)

nl <- length(p$layers)
p$layers <- c(p$layers[[nl]], p$layers[-nl])
p