CODEWITHSUNDEEP
pythonlistlist-comprehensionnan
cjstevens
cjstevens

Reputation: 131

List comprehension that skips NaN values

I know there is a similar question to this already (List comprehension that ignores NaN) but the solutions do not fit my case.

I have a list of tuples. I'm using a list comprehension to generate a list of subtuples from this list of tuples. Depending on the data file I'm reading in, sometimes I have NaN (np.nan) values. In these cases, my list comprehension is returning tuples of nans (nan, nan), but I'd prefer it didn't.

Below is an illustration of what I've tried. It doesn't give me the output I want, but doesn't give an error either:

   list_values = 
        [('A', 1., 2., 1),
         ('B', 3., 4., 1),
         ('C', 5., 6., 1),
       ('D', nan, nan, 1),
       ('E', nan, nan, 1),
       ('F', nan, nan, 1)])

I thought this would work, but it doesn't:

pairs = [tuple((x[1] + x[3], x[2] + x[3])) for x in list_values if x[1] is not np.nan]

It returns this:

[(2.0, 3.0), (4.0, 5.0), (6.0, 7.0), (nan, nan), (nan, nan), (nan, nan)]

Whereas I would like:

[(2.0, 3.0), (4.0, 5.0), (6.0, 7.0)]

I don't know why my if conditions are doing nothing at all. Any insights appreciated, thank you!

Upvotes: 0

Views: 579

Answers (1)

user2357112
user2357112

Reputation: 281519

is not np.nan isn't how you do a NaN check. It's an identity check for the specific object np.nan, and most NaNs are not going to be that particular object.

math.isnan is a NaN check, and you can apply that check to the elements of x with a genexp. (Also, you don't need to call tuple on your tuples.)

pairs = [(x[1] + x[3], x[2] + x[3]) for x in list_values
                                    if all(not math.isnan(elem) for elem in x[1:])]

Upvotes: 4

Related Questions