What would be a good way to represent a vector of hierarchical objects?

Question

I have an object represented as nested dictionaries \ arrays all objects in the set have the same schema, though the number of entries in arrays may vary from object to object I'm looking for a good way to store this data, that will allow simple batch operations. specifically, fast access to a column \ set of columns as numpy arrays is important (which is the main reason not to use a simple list of objects)

pandas with MultiIndex was my 1st idea, but I don't believe it can support variable length arrays

Answer 1

pytables is a very solid option for this use case. It supports variable length arrays.