CODEWITHSUNDEEP
rustapache-arrow
Mark
Mark

Reputation: 2428

How can I create Arrow Builders from a Schema in Rust?

Given an arrow schema, what would be the idiomatic way to create builders for each field so that I can populate these fields with values that match the schema so that they may later be written to a parquet file that uses this schema?

For example, given the schema:

let foo_field = Field::new("FOO", DataType::Utf8, false);
let baa_field = Field::new("BAA", DataType::UInt16, false);

let schema = Schema::new(vec![
    foo_field,
    baa_field,
]);

Then I can create builders which will match the schema using

let mut foo_builder = GenericStringBuilder::<i32>::new();
let mut baa_builder = UInt16Array::builder(batch_size);

but I would ideally like to create them from the schema so that I can be certain they will match but I can't see a way to do this.

For context, my dependencies are currently:

[dependencies]
arrow = "37.0.0"
parquet = "37.0.0"
itertools = "0.10.5"

but I am open to changing libraries and versions if needed.

Upvotes: 1

Views: 459

Answers (1)

richselian
richselian

Reputation: 731

arrow-rs do have a make_builder API to dynamically create builders from data types. however some types (for example lists and maps) are not supported at the moment.

Upvotes: 1

Related Questions