CODEWITHSUNDEEP
arraysmatlabmemorystructure
Lipika Deka
Lipika Deka

Reputation: 3904

Method to store very large data structures in Matlab

I have 6000 origin-destination location pairs of England. For each pair I need to find the shortest path from a origin to a destination. The shortest path algorithm takes as input the details (node latitude and longitude, road link length and uniques id) of each link of the underlying road network and one OD pair.

The number of links on the underlying road netwrk is 4091065. I read the road network and store it in a array of structures with each element representing one link and save it in a .mat file . As you can see the size of data become too much and Matlab runs out of memory.

I would be gratefull for methods to get myself around this problem. Increasing the memory does not seem to help much.

Thanks

Upvotes: 0

Views: 75

Answers (2)

nkjt
nkjt

Reputation: 7817

The shortest path algorithm takes as input the details (node latitude and longitude, road link length and uniques id) of each link of the underlying road network and one OD pair.

Surely you can limit the amount of the road network you need to load. All 4091065 links aren't going to be needed - if you're going from Truro to London the shortest path is unlikely to take a detour through York.

Therefore, rather than having a single file, you could write a preprocessor that decides what subset of the road network you require for a given OD pair and loads that into the format you require (optionally storing then storing that subset in a '*.mat' file or some other format if you think you'll want to try a few variations of the algorithm on the same data).

Upvotes: 1

bdecaf
bdecaf

Reputation: 4732

I believe the best method is not to do it in Matlab at all - use a database.

If it has to be Matlab maybe you can partition your data in some way - maybe in Counties or so. Pretty sure you can find a way to exclude lots of nodes for every specific request as well.

Upvotes: 1

Related Questions