CODEWITHSUNDEEP
laravel-10
Harshil Patanvadiya
Harshil Patanvadiya

Reputation: 309

Import up to 400 MB excel sheet use maatwebsite/excel package is not working

I want to import the Excel sheet using the artisan command. The excel-sheet file size is 400MB has 500 000 rows and 300 columns. This Excel sheet is stored in the public folder.

Artisan command code:

public function handle()
{
    $this->output->title('Excel Import Process Start');

    // Check type is valid
    $file = $this->option('file');

    $filePath = public_path('excel-import/' . $file);
    if (!File::exists($filePath)) {
        return $this->output->error("File is not found!");
    }

    Log::info("step 1");
    try {
        (new ImportApplications)->withOutput($this->output)->import($filePath);
    } catch (Exception $e) {
        $this->output->error("Import Exception");
        $this->output->info($e->getMessage());
        return false;
    }

    $this->output->success('Import successful');
    return true;
}

Import application code

class ImportApplications implements ToModel, WithHeadingRow, WithBatchInserts, WithChunkReading, WithUpserts, WithProgressBar , ShouldQueue
{
    use Importable;
    /**
     * @param array $row
     *
     * @return \Illuminate\Database\Eloquent\Model|null
     */

    public function model(array $row)
    {
        Log::info($row);
        return new Application([
            'FILED_NAME' => $row['column_name'],
            ...
        ]);
    }

    public function batchSize(): int
    {
        return 1000;
    }

    public function chunkSize(): int
    {
        return 1000;
    }
}

I have set memory_limit: 1500M

The given error is

XMLReader::read(): Memory allocation failed : growing input buffer

If I set memory_limit: 2500M Then the command was automatically killed

I have also added the import process in the queue but same issue

Upvotes: 1

Views: 622

Answers (1)

Urvashi
Urvashi

Reputation: 46

You can use Laravel FastExcel package.It could be considered as a faster (and memory friendly) alternative to Laravel Excel, with less features.

Here is the benchmark set by Laravel FastExcel package.

Laravel FastExcel Package Benchmark

Upvotes: 3

Related Questions