CODEWITHSUNDEEP
f#
kasperhj
kasperhj

Reputation: 10482

Merging data from different sources with different types

I have four data sources that are related by a common ID (incrementing integer).

One data source P is always available and the three others L's may or may not be available for a given ID (i.e. ID's are not guaranteed to exist) and are received over the internet and are distinguishable by their IP address.

I gather the data sources over some interval and then I would like to join them using their ID's.

My mental model is something like this

Before I run amok using hash sets and whatnot, it would be nice to get some ideas on how to do this. Maybe there are some cool F# features that makes this really easy.

Upvotes: 0

Views: 109

Answers (1)

TheInnerLight
TheInnerLight

Reputation: 12184

I think you can translate your domain model into types in F# a bit more directly.

One data source P is always available and the three others L's may or may not be available for a given ID (i.e. ID's are not guaranteed to exist) and are received over the internet and are distinguishable by their IP address.

This describes a record type in F# pretty neatly.

type MyDataSources = 
    {Id: int;
     P : MyDataSource;
     L1: MyOtherDataSource option;
     L2: MyOtherDataSource option;
     L3: MyOtherDataSource option;}

I gather the data sources over some interval and then I would like to join them using their ID's.

So, we need another record type for the data. 

type JoinedData<'T, 'U> =
    {PData : 'T list;
     L1Data : 'U list option;
     L2Data : 'U list option;
     L3Data : 'U list option;}

We then just need a function that takes the data sources and populates the data.

let populateDataFromSources dataSources =
    {PData = getDataFrom dataSources.P // replace with whatever logic you want here
     ....
    }

If needed, your getDataFrom function could take the Id from your data source as an argument and filter for data returned which relates to that Id (for example).

Upvotes: 2

Related Questions