geom_smooth not working/showing anymore after adding more data

Question

I pull data from a database every week and plot some charts using ggplot2. This week geom_smooth is not showing up anymore. When I remove the last record it works, why?

Sample Data

data <- structure(list(Status = structure(c(3L, 3L, 3L, 3L, 3L, 3L, 3L, 
3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 4L, 4L), .Label = c("Cancelled", 
"Closed", "In SAP", "Open"), class = "factor"), Year_Month = c("2017-06", 
"2017-07", "2017-08", "2017-09", "2017-10", "2017-11", "2017-12", 
"2018-01", "2018-02", "2018-03", "2018-04", "2018-05", "2018-06", 
"2018-07", "2018-08", "2018-09", "2018-10", "2018-11", "2018-10", 
"2018-11"), CNT = c(63L, 52L, 66L, 45L, 47L, 49L, 42L, 44L, 48L, 
67L, 46L, 46L, 58L, 41L, 50L, 45L, 57L, 29L, 19L, 46L), per = c(67.74, 
71.23, 70.97, 71.43, 78.33, 71.01, 63.64, 67.69, 53.93, 73.63, 
60.53, 54.76, 81.69, 69.49, 63.29, 70.31, 69.51, 33.33, 23.17, 
52.87), date = structure(c(17318, 17348, 17379, 17410, 17440, 
17471, 17501, 17532, 17563, 17591, 17622, 17652, 17683, 17713, 
17744, 17775, 17805, 17836, 17805, 17836), class = "Date")), .Names = c("Status", 
"Year_Month", "CNT", "per", "date"), row.names = c(NA, -20L), class = c("grouped_df", 
"tbl_df", "tbl", "data.frame"), vars = "Year_Month", drop = TRUE, indices = list(
0L, 1L, 2L, 3L, 4L, 5L, 6L, 7L, 8L, 9L, 10L, 11L, 12L, 13L, 
14L, 15L, c(16L, 18L), c(17L, 19L)), group_sizes = c(1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 
2L), biggest_group_size = 2L, labels = structure(list(Year_Month = c("2017-06", 
"2017-07", "2017-08", "2017-09", "2017-10", "2017-11", "2017-12", 
"2018-01", "2018-02", "2018-03", "2018-04", "2018-05", "2018-06", 
"2018-07", "2018-08", "2018-09", "2018-10", "2018-11")), row.names = c(NA, 
-18L), class = "data.frame", vars = "Year_Month", drop = TRUE, .Names = "Year_Month"))

Plot with 20 records

ggplot(data,aes(x=date, y=per)) +
  geom_point(aes(colour=Status),size=3) +
  geom_smooth(method = 'loess',aes(group=data$Status, color=Status))

Plot with 19 records

data <- head(data,19)

Answer 1

As a general solution, one can do:

library(ggplot2)
library(dplyr)

min_number <- 5    # set this to something reasonable of your choice

ggplot(data, aes(x = date, y = per, color = Status)) +
  geom_point(size=3) +
  geom_smooth(
    data = . %>% group_by(Status) %>% filter(n() >= min_number),
    method = 'loess'
  )

To only plot smooths for groups with at least min_number of observations.

Answer 2

Depending on what you're trying to do/show, you can either:

1) Only use geom_smooth with groups where you have enough data, by specifying the data argument in the call

ggplot(data,aes(x=date, y=per)) +
  geom_point(aes(colour=Status),size=3) +
  geom_smooth(data = data %>% filter(Status != "Open"), method = 'loess', aes(color = Status))

or

2) Use geom_smooth on all the data together

ggplot(data,aes(x=date, y=per)) +
  geom_point(aes(colour=Status),size=3) +
  geom_smooth(data = data, method = 'loess')

The warnings you saw (per @hrbrmstr's comments) are from the loess function. I'd check out ?loess. It's always helpful to know what's behind your plots.

Answer 3

As mentioned in comments geom_smooth cannot handle 2 elements in the status group of Open. Interestingly, it is ok with 1 or 3 elements. To solve the problem I decided to exclude Open from the data and it worked without any problem.

data2 <- data %>% dplyr::filter(Status!="Open")

ggplot(data,aes(x=date, y=per)) +
  geom_point(aes(colour=Status),size=3) +
  geom_smooth(data=data2,method = 'loess',aes(group=Status, color=Status))