CODEWITHSUNDEEP
csvgpudaskcudfblazingsql
Soumya Bhattacharjee
Soumya Bhattacharjee

Reputation: 11

How to process data larger than GPU Memory using BlazingSQL

I am trying to run a sql query with a 50 GB CSV file but my GPU Memory is of only 40GB. How can I do the processing?
Also I am only able to run blazingsql with the jupyter notebook available with their docker image, can anyone please help me how to install it locally?

As it is not being possible with the conda command available on their github.

Upvotes: 0

Views: 313

Answers (1)

Nick Becker
Nick Becker

Reputation: 4214

One way to do this today is to use Dask-SQL. Because it's built on Dask, Dask-SQL inherits Dask's ability to handle larger-than-memory workloads.

The easiest way to install Dask-SQL and use GPUs is to create a conda environment or pull a Docker container using the RAPIDS release selector.

Upvotes: 1

Related Questions