Dynamic list comprehension

Question

I would like to know if python is able to create a list by comprehension using multiple and optional criteria.

Let's make an example. Considering the following object (partial description):

class Person():
    def __init__(self):
        self.id = <next id of some kind>
        self.name = 'default name'
        self.gender = 'm'
        self.age = 20

<...>

Suppose I created a list of all Persons into world. Then I want to create a GUI which will allow me to browse the collection based on search criteria (the GUI conception is out of scope of the question), e.g name (regex based), id, gender and age (with equal, not equal and greater or lesser than). None of the search criteria are mandatory (we can suppose it's None I guess) and the type do not really matter for this question.

How can I filter the list of Person in a clever python-way?

If I have known criteria I could do a comprehension :

l = [person for person in world if re.search(person.name, '.*Smith') and person.gender = 'm' and person.age < 20]

But as the user is able to choose what he wants, I won't know what criteria to use. I can of course build this as fully fledged function:

l = world
if nameSearch:
    l = [person for person in l if re.search(person.name, nameSearch)]
if genderSearch:
    l = [person for person in l if gender == genderSearch]
<...>
return l

But I feel python would have a way to do it more properly.

Answer 1

Elaborating my comment above:

As functions are first class citizens in Python, you could write a bunch of matcher functions, put them (dynamically) in a list and match against them in a single list comprehension.

Let predicates be a list of one-argument functions of type Person -> bool.

Then simply do:

[ pers for pers in world if all([f(pers) for f in predicates]) ]

Further exploring the functional route of thinking, you can create "dynamic matching functions" by creating functions returning matching functions:

def age_matcher(age):
    return lambda p: p.age > age

An age_matcher(someAge) can be added to your predicates array.

Side note

For these "database-search"-like tasks, you will probably want to really should look at libraries like Pandas, where you can make queries similar to SQL. You may be re-inventing a fairly complex type of wheel.

Answer 2

Based DCS' comment, here is a example how to use functions as filters. A filter is just a function which returns a boolean (given an instance of Person). For faster processing I suggest you take a look at pandas, which is a very good choice for data filtering/sorting/munging, but this might get you started with a simple solution. The only task that is left to you, is to create the filters based on the user's input.

from random import random

class Person():
    def __init__(self, id):
        self.id = id
        self.name = 'Name{}'.format(id)
        self.gender = 'm' if random() > 0.5 else 'f'
        self.age = int(random() * 10) + 10

    def __repr__(self):
        return 'Person-{} ({}, {}. {})'.format(self.id,
                                               self.name,
                                               self.gender,
                                               self.age)

Setting up some test data:

people = [Person(id) for id in range(10)]

[Person-0 (Name0, f. 15),
 Person-1 (Name1, f. 14),
 Person-2 (Name2, f. 12),
 Person-3 (Name3, f. 18),
 Person-4 (Name4, m. 12),
 Person-5 (Name5, f. 18),
 Person-6 (Name6, f. 15),
 Person-7 (Name7, f. 15),
 Person-8 (Name8, f. 10),
 Person-9 (Name9, m. 16)]

Output:

def by_age(age):
    return lambda person: person.age == age

def by_name(name):
    return lambda person: re.search(person.name, name)

def by_gender(gender):
    return lambda person: person.gender == gender

filters = (by_age(15),
           by_gender('f'))

filtered_people = (p for p in people if all([f(p) for f in filters]))
list(filtered_people)

Which gives us the following filtered list of people:

[Person-0 (Name0, f. 15), Person-6 (Name6, f. 15), Person-7 (Name7, f. 15)]

You could even change the predicate all to any in order select all people which match any of the specified filters.

Answer 3

How about this?

def search(self, condition):
    return filter(condition, self.l)


def search_re(self, **kwargs):
    filters = []
    for key, value in kwargs.items():
        if isinstance(value, str):
             value = re.compile(value)
             filters.append(lambda x: re.search(getattr(x, key), value))
        elif callable(value):
             filters.append(lambda x: value(getattr(x, key)))
        else:
             filters.append(lambda x: getattr(x, key) == value)
    def condition(person):
        return all(
                 f(person) for f in filters
               )

    return self.search(condition)

Usage:

persons.search(lambda x: x.name == "bla")

persons.search_re(name=".*Smith", gender="male")