CODEWITHSUNDEEP
pysparkazure-databricks
Julaayi
Julaayi

Reputation: 509

Widgets with non-duplicate values in PySpark

I am trying to build a widget from a list that has non-duplicate values based on the below scenario. Could someone please help me with getting the right way?

fileInfoList = list(filter(lambda f: f.name.endswith("") , dbutils.fs.ls(srcPath)))
for fileNames in fileInfoList:
  print(fileNames.name)

This prints: Employee EmployeeHistory Contractor ContractorHistory

What I wanted is only the values without History. Tried this but returns error:

dbutils.widgets.dropdown("FileName", "Employee", [str(fileNames.name) for fileNames in fileInfoList])

Upvotes: 0

Views: 164

Answers (1)

BICube
BICube

Reputation: 4681

Why don't you simply filter your list before feeding into the dropdown function? 

>> fileList = ['Employee', 'Contractor', 'EmployeeHistory', 'ContractorHistory']
>> print(fileList)
   ['Employee', 'Contractor', 'EmployeeHistory', 'ContractorHistory']

>> filteredFileList = [item for item in fileList if 'History' not in item]
>> print(filteredFileList)
   ['Employee', 'Contractor']

Upvotes: 1

Related Questions