How to draw line between dodged geometry in ggplot?

Question

I'm trying to figure out how to draw a line between dodged the geometry within ggplot.

I would like a line connecting the tops of the bars (or other geometry) that is being dodged using position_dodge()

Here is an example using mtcars

I would like that line to connect the tops of the bars within a single gear group.

However, if I only specify a position_dodge for the geom_line() the resulting line connects the top of the carb levels.

ggplot(
  data = mtcars,
   aes(
       x=gear,
       y=cyl,
       group=carb
       )
    )+
    geom_col(
    position = position_dodge(width = 0.9)
    )+
    geom_line(
    position = position_dodge(width = 0.9)
    )

Specifying gear as the grouping within geom_line(), resulting in just a single vertical line.

ggplot(
  data = mtcars,
  aes(
     x=gear,
     y=cyl,
     group=carb
    )
  )+
   geom_col(
     position = position_dodge(width = 0.9)
   )+
   geom_line(
    aes(
      group=gear
    ),
    position = position_dodge(width = 0.9)
  ) 

The desired output would look like this:

Answer 1

You can try the following. Explanations inline:

library(magrittr)
library(dplyr)

mtcars %>%
  # rename the variables for easier generalisation to other data frames
  rename(x = gear, y = cyl, group = carb) %>%

  # keep only the max y value for each dodged bar; we don't want additional
  # points
  group_by(x, group) %>%
  summarise(y = max(y)) %>%
  ungroup() %>%

  ggplot(aes(x = x, y = y, group = group)) +
  geom_col(position = position_dodge(width = 0.9)) +

  # create invisible point layer that captures the dodged x values, while keeping
  # the values at each x point a different colour, so that we can identify points 
  # that belonged to the same x value for joining later
  geom_point(aes(color = x), position = position_dodge(width = 0.9), 
             alpha = 0, show.legend = FALSE) +

  theme_bw() -> p

p + geom_line(data = layer_data(p, 2L),
              aes(x = x, y = y, group = colour),
              color = "red", size = 2)