CODEWITHSUNDEEP
mysqlcsvbulk-load
siegel
siegel

Reputation: 829

losing connection to MySQL server when loading 30GB CSV file

I'm trying to load a 30GB CSV file into MySQL using:

LOAD DATA LOCAL INFILE 'C:Documents/CRSP.csv' INTO TABLE rawcrsp
FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n'  IGNORE 1 LINES

After a few hours of waiting, I get an error: Error Code: 2013. Lost connection to MySQL server during query.

The same code works with a smaller CSV file formatted the exact same way. I'm working in Windows 7. I have 24 GB RAM, and 240 GB free space on my hard drive. Any tips for how to get this to load?

Upvotes: 2

Views: 4250

Answers (1)

Thanos Pappas
Thanos Pappas

Reputation: 167

I can think of 2 reasons:

  1. Check if wait_timeout variable in your configuration file is large enough, if not increase it.
  2. If mysqld gets a packet that is too large , it thinks that something is wrong with the client and closes the connection. Try to maximize packet size limit from the configuration file.

Upvotes: 1

Related Questions