Attach client side only state to history in Sveltekit

Question

What is the best way to work with client side state and the browser's history api?

As a learning experiment I'm building a dungeon crawler style webapp.
Based on URL segments it renders a piece of the story and comes with a few available choices and discovered locations.
Depending on the user's previous actions they can choose some of those choices or one of the unvisited locations they discovered throughout their play. Those lead to a new URL with other pieces of the story and so on.

The story pieces are the same for each user so can be rendered server side.
The choices and locations available at that story piece can be different for each user.
User sessions are client side only.

How can I allow the user to use the back button (history api) so they can trace back their steps to the previous story segment complete with the choices and locations available at that point in time?

Currently a custom store saves every permutation of the user's state and can undo/redo.
I don't know how to link that functionality to history actions though.
I tried appending a unique querystring, but I can't find a way to have my custom store subscribe to querystring changes.

The contextApi and shallow routing seems promising, but I just can't figure out how to attach and then pop the context on navigation events.

I also keep running into server errors that try to work with context server side so it feels I'm on the wrong path.