CODEWITHSUNDEEP
pyarrowfeatherintake
bowlby
bowlby

Reputation: 649

Apache Arrow or feather plugin?

I'd like to use local feather-files as a sources in Intake. Is the plugin for feather/arrow not yet existing or am I missing something?

Upvotes: 1

Views: 201

Answers (1)

mdurant
mdurant

Reputation: 28684

You are right, there is not currently a feather or arrow (i.e., framed buffers) driver for Intake, as far as I know. Neither is supported by Dask either.

Given that pandas does explicitly support feather, it would be easy to build a driver for Intake, supporting multiple remote files, and these could even be loaded in parallel with Dask, without adding code to dask.

However, first I'd like to ask: why not parquet? That seems to be the standard format, at least partly because its reach goes well beyond python/arrow. That format is supported by Intake and any other tabular data engine.

Upvotes: 2

Related Questions