Reputation: 115
I'm quite new to functional programming, and I'm trying to figure out a function that recursively generates elements of the cartesian product of a set of lists.
The functionality I'm looking for is exactly like sequence
(as described here: Calculate n-ary Cartesian Product), except that I don't want to express the whole thing as a list.
I'm currently using sequence
and running into a variation of the problem described here: Summing a large list of numbers is too slow.
As an example, sequence [[1,2,3],[1,2,3]]
produces [[1,1],[1,2],[1,3],[2,1],[2,2],[2,3],[3,1],[3,2],[3,3]]
. It's perfectly acceptable to deal with each combination (i.e. [1,2]
, etc, as a list, I would just like to avoid building up the long outer list, instead processing the data recursively as it is calculated. How would I go about doing this?
I'm currently doing something similar to this quick ghci exampl:
> let stuff = sequence $ replicate 10 [0..9]
> let morestuff = map (sum . take 2 . reverse . sort) stuff
> sum morestuff
and the last command is horribly, horribly slow.
Upvotes: 3
Views: 231
Reputation: 38901
Moving my comment to an answer:
The only reason that the last line appears particularly slow is that it is forcing the work of all the other lines, which is otherwise accumulated lazily. So each sublist is being processed on an as-needed basis (though the sort on each forces the entire sublist).
To observe that all the phases take time and not just the last, we can use rnf
from Control.DeepSeq
on the intermediate results to force them as we go.
But generating and summing roughly ten billion things will take some time. As others have observed, this is a case where you should think more about a more clever way to get the result you want (i.e. what intermediate steps can be removed or transformed without changing the result, and what symmetries you can exploit) rather than just doing the brute force thing more efficiently.
Upvotes: 2