CODEWITHSUNDEEP
pythonhadoophdfs
Victor Longui
Victor Longui

Reputation: 11

How to list all files in the path using the hdfs lib in python?

I'm trying to list all the files in my hdfs path but something seems strange. the function is only listing some files, but not all, even in paths that only have 7 files, it only gets the ones with ending 1. But I need to get all the .parquet files that are in the path.

Can you help me?

def list_files_hdfs(client, path):
    try:
        files = client.list(path)
        return [f"{path}/{file}" for file in files]
    except Exception as e:
        print(f"Error listing files: {e}")

Upvotes: 0

Views: 26

Answers (0)

Related Questions