CODEWITHSUNDEEP
rustcapnproto
Jeff Burdges
Jeff Burdges

Reputation: 4261

Mutable state with Cap'n proto

How should one use Cap'n Proto for an application's mutable state similar to how Protobuf gets used? Is there a garbage collector?

Kenton Varda confirmed in his comparison of Cap'n Proto, FlatBuffers, and SBE that Cap'n Proto uses arena allocators internally to messages. A single message would grow without bound if one edits it over an extended period, say due to being written to disk and reloaded.

Are there any garbage collectors for Cap'n Proto to rearange the message and reclaim any wasted space? Would a garbage collector be the optimal approach? If not, or if not exists, then what is the recommended approach?

I'm actually writing a Rust program that must only save encrypted data anyways. I'm therefore okay with recopying the whole message structure, but I'm curious about the options more widely.

Upvotes: 2

Views: 928

Answers (1)

Kenton Varda
Kenton Varda

Reputation: 45246

The only way to reclaim wasted space is to copy the message into a new MessageBuilder. Only the "used" parts will be copied. This effectively is "GC" -- many of the best GC algorithms actually move data, which is what you'd be doing here.

There is no practical way to implement non-moving GC of arena-allocated Cap'n Proto messages.

I am considering extending the Cap'n Proto code generator in C++ to also generate a set of classes appropriate for representing the same data structures on the heap, such that you can modify the structure over time. Converting between the heap representation and the arena representation will require a copy, of course. But, this isn't yet implemented and I don't have any timeline. (The Rust implementation would likely get a similar update.)

Upvotes: 5

Related Questions