How to Import Large CSV File into Database in Laravel 11?

Hi, Dev

In this tutorial, I'll guide you through the process of importing large CSV and Excel files into a database using a Laravel 11 application.

Sometimes, we need to import large CSV files—ranging from 1GB to 4GB or more—into a database. Handling such large files can cause timeouts or other issues in Laravel. However, I have an ideal solution for importing these large CSV files into your database efficiently. We'll utilize Laravel's LazyCollection, DB, fopen(), and fgetcsv() functions to read the CSV file and store the data in the database.

Let's take a look at the example code:

Step for Laravel 11 Import Large CSV File into Database Example

Step 1: Install Laravel 11

Step 2: Create Products Table

Step 3: Create Seeder

Run Laravel App

Step 1: Install Laravel 11

This step is not required; however, if you have not created the Laravel app, then you may go ahead and execute the below command:

composer create-project laravel/laravel example-app

Step 2: Create Products Table

Here, we need to create a database migration for the "products" table and also we will create a model for the "products" table.

php artisan make:migration create_products_table

Migration:

<?php
  
use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;
  
return new class extends Migration
{
    /**
     * Run the migrations.
     */
    public function up(): void
    {
        Schema::create('products', function (Blueprint $table) {
            $table->id();
            $table->string('name');
            $table->string('amount');
            $table->text('description');
            $table->timestamps();
        });
    }
  
    /**
     * Reverse the migrations.
     */
    public function down(): void
    {
        Schema::dropIfExists('products');
    }
};

you need to run the following command to migration.

php artisan migrate

Step 3: Create Seeder

Here, we will create ProductSeeder class and write code of import large csv file.

Make sure you created products.csv file with "name", "amount" and "description" columns. Put that file in public folder.

Now, let's create seeder class using following command:

php artisan make:seeder ProductSeeder

Let's update following seeder code:

database/seeders/ProductSeeder.php

<?php
  
namespace Database\Seeders;
  
use Illuminate\Database\Console\Seeds\WithoutModelEvents;
use Illuminate\Database\Seeder;
use Illuminate\Support\Facades\DB;
use Illuminate\Support\LazyCollection;
  
class ProductSeeder extends Seeder
{
    /**
     * Run the database seeds.
     */
    public function run(): void
    {
        DB::disableQueryLog();
        DB::table('products')->truncate();
  
        LazyCollection::make(function () {
            $handle = fopen(public_path("products.csv"), 'r');
            
            while (($line = fgetcsv($handle, 4096)) !== false) {
                $dataString = implode(", ", $line);
                $row = explode(',', $dataString);
                yield $row;
            }
  
            fclose($handle);
  
        })
        ->skip(1)
        ->chunk(1000)
        ->each(function (LazyCollection $chunk) {
  
            $records = $chunk->map(function ($row) {
                          return [
                            "name" => $row[0],
                            "amount" => $row[1],
                            "description" => $row[2]
                          ];
                      })->toArray();
            
            DB::table('products')->insert($records);
  
        });
    }
}

Now you need to run following command for run all listed seeder:

php artisan db:seed --class=ProductSeeder

Now, you will able to see new row will be added in your products table.

Now I think you will understand how seeding works and we have to use it in our laravel app

I hope it can help you...