CODEWITHSUNDEEP
pythonpandasdataframeparquet
Amir
Amir

Reputation: 441

AttributeError: __enter__ while using with statement with read_parquet

I want to open all parquet files existing in a specific folder and draw a scatter plot with it. I used following code: 

for file in glob.glob("*.parquet"):
    with pd.read_parquet(file, columns=["cordx", "cordy"]) as df:
        make some scaterplot

Files can be found as I let python print the file sucessfully. But i recive AttributeError: __enter__. And read_parquet is the corredct way to open those files. When I dont use columns=[ ] I get pyarrow.lib.ArrowIOError: Arrow error: Out of memory: malloc of size 9771487328 failed error because the files are very big. So its necessary to load only those columns.

Upvotes: 1

Views: 1631

Answers (1)

Colonder
Colonder

Reputation: 1576

This error tells you that pandas.read_parquet does not have a context manager implemented which means that you cannot call it using with because __enter__ and __exit__ methods are missing. Look in here How to read a Parquet file into Pandas DataFrame?

Upvotes: 3

Related Questions