How can I work with larger than memory Snowflake datasets in polars?

Question

I'm trying to work with data stored in a Snowflake database using polars in python. I see I can access the data with pl.read_database_uri with the adbc engine. I was wondering how I can do this efficiently for larger-than-memory datasets.

• Is it possible to stream the results using polar's lazy API, or any other method?
• Is it possible to batch the results as pl.read_database can? Or is it possible to partition the results, as the docs say is possible with connectorx?
• Are there any other ways I might use polars to help work with larger-than-memory datasets in this instance? Or do I need to do my processing in SQL so that the data comes into python in a manageable size?

Thanks!