CODEWITHSUNDEEP
pythonpython-2.7
Sidd
Sidd

Reputation: 291

Filter on Lambda Function Output not working in Python 3: Invalid argument, not a string or column

I am encountering an issue with one of the existing line of code working in python 2.7 and throwing exception while using python 3. Below is simple example, please note i've reduced the "Rules" to only show relevant values.

>>> Rules=[{u'source': u'SOURCE_TABLE', u'target': u'TARGET_TABLE', u'mappings': [{u'action': u'lookup', u'tColumn': u'TARGET_COLUMN', u'sColumn': u'SOURCE_COLUMN', u'entity_id': u'1'}, {u'action': u'lookup', u'tColumn': u'TARGET_COLUMN1', u'sColumn': u'SOURCE_COLUMN1', u'entity_id': u'2'}]}]
>>> for rule in Rules:
...    m=filter(lambda mapping: mapping.get('action') in ['lookup'],rule['mappings'])
...
>>> m
[{u'action': u'lookup', u'entity_id': u'1', u'tColumn': u'TARGET_COLUMN', u'sColumn': u'SOURCE_COLUMN'}, {u'action': u'lookup', u'entity_id': u'2', u'tColumn': u'TARGET_COLUMN1', u'sColumn': u'SOURCE_COLUMN1'}]

The same code while running with Python 3.7 throws an exception-

TypeError: Invalid argument, not a string or column: <function <lambda> at 0x7efdfbe708b0> of type <class 'function'>. For column literals, use 'lit', 'array', 'struct' or 'create_map' function.

I tried wrapping the filter function in the list(filter(....)) accrding to python 3 documentation, it didn't help though.

Upvotes: 1

Views: 202

Answers (1)

Daniel
Daniel

Reputation: 9534

I was having this issue in Databricks and @Barmar's comment was a hint in the right direction.

Instead of using filter() use a generator function:

m = [mapping for mapping in rule['mappings'] if mapping.get('action') in ['lookup']]

Upvotes: 1

Related Questions