Jetpack Compose Wear OS - PositionIndicator state lost inside a ViewPager2

Question

I struggle to find a solution to a weird behavior with PositionIndicator in Jetpack Compose. Firstly, I am in a specific case when I want to setup a Jetpack Compose fragment to be displayed inside a ViewPager2 component.

So I have a ViewPager2 with 4 fragments. One of them is a Jetpack Compose fragment. Within the fragment, I override the onCreateView method :

override fun onCreateView(inflater: LayoutInflater, container: ViewGroup?, savedInstanceState: Bundle?): View? {
        return ComposeView(requireContext()).apply {
            setContent {
                val scalingLazyState = rememberScalingLazyListState()
                Scaffold(
                    positionIndicator = {
                        PositionIndicator(scalingLazyListState = scalingLazyState)
                    }
                )
                {
                    ScalingLazyColumn(state = scalingLazyState) {
                        items(100) { index ->
                            Text("Item #$index", modifier = Modifier.padding(8.dp))
                        }
                    }
                }
            }
        }
    }

When I run this in the simulator and I pan to this fragment, I can see the scroll bar and when I scroll, I see the thumb moving. Then I go to another fragment, then back to the Jetpack's one. Scrollbar is still there, but when I try to scroll, the thumb is not moving anymore, besides the view is still fully scrollable.

A gif is better than 1000 words, so here it is:

It seems to be in relation with fragment lifecycle and how the Jetpack layout is restoring it's state. When I use:

viewPager.setOffscreenPageLimit(4)

The issue disappear and the thumb is updated correctly. I can use this, but this is not very efficient for smart watches, as every fragment in my ViewPager2 are awake with this method.