CODEWITHSUNDEEP
pythonpython-2.7list-comprehension
dave
dave

Reputation: 2269

Filtering list of dictionaries by regex match

I'm writing a python script which collects metrics and I have:

I want to be able to inhibit the forwarding of those messages in which collected[i]['service'] matches at least one regular expression in denied_metrics.

I was trying to achieve my goal using list comprehension and filter, but I didn't manage to do that way.

actual solution

Given that messages have the following structure:

msg = { 
  'service': 'foo', 
  'metric':    1.0,
  'denied':  False 
}

actually I'm filtering all collected messages like follows

def filter_denied( denied_metrics, collected ):
  for pattern in denied_metrics:
    for msg in collected
      if pattern.match( msg['service'] ):
        msg['denied'] = True

return [ msg for msg in collected if msg['denied'] is not True ]

question

Is there a (better?) way to obtain a list of allowed messages using just a combination of list comprehension and filter or reduce?

EDIT

I was not aware about the possibility to approach the problem like @eyquem suggested by in his answer.

Upvotes: 0

Views: 1780

Answers (2)

DSM
DSM

Reputation: 353059

IIUC, I might do do something like

allowed = [msg for msg in collected 
           if not any( dm.search(msg['service']) 
                       for dm in denied_metrics) ]

For example:

>>> pprint.pprint(collected)
[{'denied': False, 'metric': 1.0, 'service': 'ab'},
 {'denied': False, 'metric': 1.0, 'service': 'bc'},
 {'denied': False, 'metric': 1.0, 'service': 'ca'},
 {'denied': False, 'metric': 1.0, 'service': 'cb'},
 {'denied': False, 'metric': 1.0, 'service': 'bc'}]
>>> denied_metrics = [re.compile("a"), re.compile("c$")]
>>> allowed = [msg for msg in collected 
               if not any(dm.search(msg['service'])
               for dm in denied_metrics)]
>>> allowed
[{'metric': 1.0, 'service': 'cb', 'denied': False}]

Whether you want search or match depends upon your regexes, of course. [BTW, wouldn't 'denied_services' be a better name?]

Upvotes: 2

eyquem
eyquem

Reputation: 27575

You have an XY problem.

Here are two ways to delete elments of a list while iterating in it:

li = ['a',12,45,'h',56,'ju',0]
print li
for i in xrange(len(li)-1,-1,-1):
    if isinstance(li[i],int):
        del li[i]
print li
# prints ['a', 'h', 'ju']

.

li = ['a',12,45,'h',56,'ju',0]
L = len(li)
for i,x in enumerate(reversed(li),1):
    if isinstance(x,str):
        del li[L-i]
print li
# prints [12, 45, 56, 0]

In the last code reversed() returns an iterator, no new list has to be created.

Upvotes: 1

Related Questions