CODEWITHSUNDEEP
pythonlistfilter
Kingle
Kingle

Reputation: 504

How to filter a list based on elements in another list in python

I have a list A of about 62,000 numbers, and another list B of about 370,000. I would like to filter B so that it contains only elements from A. I tried something like this:

A=[0,3,5,73,88,43,2,1]
B=[0,5,10,42,43,56,83,88,892,1089,3165]
C=[item for item in A if item in set(B)]

Which works, but is obviously very slow for such large lists because (I think?) the search continues through the entire B, even when the element has already been found in B. So the script is going through a list of 370,000 elements 62,000 times.

The elements in A and B are unique (B contains a list of unique values between 0 and 700,000 and A contains a unique subset of those) so once A[i] is found in B, the search can stop. The values are also in ascending order, if that means anything.

Is there some way to do this more quickly?

Upvotes: 3

Views: 283

Answers (2)

Fukiyel
Fukiyel

Reputation: 1166

To be really sure what I've done is the fastest possible, I would have done that :

A=[0,3,5,73,88,43,2,1]
B=[0,5,10,42,43,56,83,88,892,1089,3165]

B_filter = B.copy()
C = []
for item in A:
    if filter in B_filter:
        C.append(item)
        B_filter.pop(0) # B_filter is a list, and it's in ascending order so always the first

If you don't care about losing your B list, you can just use B instead of B_filter and not declare B_filter, so you don't have to copy a 370k large list.

Upvotes: 0

Patrick Haugh
Patrick Haugh

Reputation: 60944

This is creating a new set(B) for every item in A. Instead, use the built-in set.intersection:

C = set(A).intersection(B)

Upvotes: 7

Related Questions