Getting a pointer to a value type on the heap that updates when gc moves the value

Question

Let's imagine we have an instance of a value type already in the managed heap(as part of a class or an Array). We can then get a Memory instance that references some element of the array.

Memory<int> memory = new(array, 42, 1);

That works and does what I want but the memory struct is 16 bytes on x86_64. I am wondering if it would be possible to achieve the same goal as above using only 8 bytes(or size of IntPtr on a given platform). Unmanaged pointers are not really viable since the GC can move the array whenever it runs.

I looked for a way to maybe create a GCHandle to specific array element but without success. I was also considering somehow getting a box to the value type but so that the boxed value is still part of the array. This then could be unboxed with Unsafe.Unbox<>() but I didn't found a way to create such a box.

Another way would be to create a pinned GCHandle to the array and store it somewhere as long as it is needed and then just use pointers. This would work but I would essentially stand in the way of the GC for a significant period of time(possibly whole lifetime of the application). Is it possible to allocate something on the managed heap with the guarantee that it never will be moved and at the same don't stand in the way of the GC?

I could allocate the type on the unmanaged heap but this would mean I have to do it for some contents of the type as well. This would increase the overall complexity greatly and the amount of unsafe code would grow exponentially. I want to write nice safe C# code as much as possible.

One feasible workaround would be to store boxes to my value type instances in the array. Then a simple GCHandle would suffice. I would loose the array linearity for my types but in my case it is not as crucial as actually having a way to reference the values.

That being said, ref struct's wouldn't work for me because I would need to store it on the heap eventually. I am almost certain that the only way would be the array of boxes or unmanaged heap but maybe there is a better way that I am missing.

As to why I am doing this: I have a complex type that internally stores collections or values of other complex types. The problem is that these types internally also may store other types as values or collections and in fact produce cyclic references. This is not as easy to do with structs(or even impossible) which is why my current implementation uses classes. I would however like to use structs where possible since they actually may be stored as part of a class and not as reference.

Please restrain from comments like "you shouldn't use C# like that" or "just use c++" because they don't lead to an actual solution. I am well aware that this could be achieved easily with classes which is what I described earlier. If there is no way to achieve what I described this is also a valid answer since I can go to the aforementioned workaround.

Just to be clear the problematic part of this type follows:

class Top
{
    public Top(Foo foo, Bar bar)
    {
        Foo = foo;
        Bar = bar;
        foo.Bar = Bar;
        Bar.Foo = Foo;
    }
    public Foo Foo;
    public Bar Bar;
}
class Foo(Bar bar)
{
    public Bar Bar = bar;
}
class Bar(Foo foo)
{
    public Foo Foo = foo;
}