Responsive masonry layout with just three elements

Question

I want to achieve an HTML/CSS template with three dynamic content divs such that

• when #2 is non-blank (and in this case biggest of all) put #3 to the bottom of #1 and to the left of #2
• when #2 is blank put #3 to the right of #1
• on small screens, put everything in one column

AFAIU this is called a masonry layout and generally requires a JS library to implement.

But I wonder if a special case of just three elements is possible with some smart CSS trick and without Javascript?

.container {
  display: flex;
  flex-wrap: wrap;
  align-items: flex-start;
  margin: 1em;
}

.box {
  width: 50%;
}

.red {
  background-color: red;
}

.green {
  background-color: green;
  width: unset;
}

.blue {
  background-color: blue;
}

.media-sim {
  width: 30%;
}

.media-sim .box {
  width: 100%;
}

<div class='container'>
  <div class='red box'>
    <div style='margin: 5em'>
    </div>
  </div>
  <div class='green box'>
    <div style='margin: 9em'>
    </div>
  </div>
  <div class='blue box'>
    <div style='margin: 3em'>
    </div>
  </div>
</div>

<div class='container'>
  <div class='red box'>
    <div style='margin: 5em'>
    </div>
  </div>
  <div class='green box'>
    <div></div>
    <div></div>
  </div>
  <div class='blue box'>
    <div style='margin: 3em'>
    </div>
  </div>
</div>

<div class='container media-sim'>
  <div class='red box'>
    <div style='margin: 5em'>
    </div>
  </div>
  <div class='green box'>
    <div style='margin: 9em'>
    </div>
  </div>
  <div class='blue box'>
    <div style='margin: 3em'>
    </div>
  </div>
</div>

Result: blue in #1 goes to the bottom of top row, not to the left of green Note: media-sim class here simulates the CSS @media query

---

EDIT someone suggested this to be a duplicate of CSS-only masonry layout which I think is fast and wrong because that question does not have a CSS answer and asks to solve any number of elements, not just three.

EDIT Codepen link: https://codepen.io/anton-duzenko/pen/XWQYYGv

EDIT