CODEWITHSUNDEEP
phpmysqlcsv
dardar
dardar

Reputation: 43

Importing Large CSV file in MySQL using php

I am trying to import a 60,000 or more rows in a CSV file. The code can import a 5000 lines only. Someone can help me?

require_once('connection.php');

if ($_FILES[csv][size] > 0) {

    //get the csv file
    $file = $_FILES[csv][tmp_name];
    $handle = fopen($file,"r");

    //loop through the csv file and insert into database
    do {
        if ($data[0]) {
            mysql_query("INSERT INTO transactions (EntryId_Pk, AccessCode, MobileNumber, TelcoCode, TIN, ORNo, ORAmt, NumOfEntries, ECN, AddedOn) VALUES
                (
                    '".addslashes($data[0])."',
                    '".addslashes($data[1])."',
                    '".addslashes($data[2])."',
                    '".addslashes($data[3])."',
                    '".addslashes($data[4])."',
                    '".addslashes($data[5])."',
                    '".addslashes($data[6])."',
                    '".addslashes($data[7])."',
                    '".addslashes($data[8])."',
                    '".addslashes($data[9])."'
                )
            ");
        }
    } while ($data = fgetcsv($handle,1000,",","'"));

Upvotes: 4

Views: 4530

Answers (2)

rocknrollcanneverdie
rocknrollcanneverdie

Reputation: 442

It could be a timeout error. Try to set

set_time_limit(0);

Also could be a good practice to import the data in chunks and insert multiple rows within a query instead of doing it row by row.

Upvotes: 0

complex857
complex857

Reputation: 20753

You might want to use the LOAD DATA MySQL statement. This can be really fast since you don't need to read everything into PHP-land and let MySQL be smart about the allocations. You can use it something like this from PHP:

// dont use mysql_* anymore in new code
mysqli_query($dblink, '
    LOAD DATA LOCAL INFILE "'.$file.'"
        INTO TABLE transactions
        FIELDS TERMINATED by ","
        OPTIONALLY ENCLOSED BY "\'"
        LINES TERMINATED BY "\n"
');

Upvotes: 4

Related Questions