CODEWITHSUNDEEP
listf#deedle
shankshuo
shankshuo

Reputation: 11

How do I choose data structures in F#?

I have a csv file which has almost 10000000 rows ,the structures like this:

date , code , ret
2001-01-01,000001,0.1
2001-01-01,000002,0.01
2001-01-02,000001,0.05
2001-01-02,000002,0.02

The fields "date" and "code" are only one key. I want to subset the file quickly, like this 

subset(code='000001')

date , code , ret
2001-01-01,000001,0.1
2001-01-02,000001,0.05

or

subset(date='2001-01-01')

date , code , ret
2001-01-01,000001,0.1
2001-01-01,000002,0.01

How should choose the right data structures so that it works efficiently?

Upvotes: 1

Views: 139

Answers (1)

Joel Martinez
Joel Martinez

Reputation: 47749

Take a look at the CSVTypeProvider from the F# Data project:
https://fsharp.github.io/FSharp.Data/library/CsvProvider.html

You could use this as the base data structure to easily parse the data into a more optimized data structure for quick access as @MarcinJuraszek describes.

Upvotes: 1

Related Questions