How to Import Large CSV File into Database in Laravel 10?

Hi dev,

Let's explore how Laravel 10 imports a sizable CSV file into a database in this little demonstration. We'll put a laravel 10 big excel import csv example into practise in this article. How to import a huge csv file into a database in Laravel 10 is described step by step. I'd want to demonstrate how to import a huge CSV file using Laravel 10.

We occasionally need to import huge CSV files into databases, such as those that are 1GB, 2GB, 4GB, etc. I have the ideal solution for importing large CSV files into your database despite the fact that large files can create timeouts or other problems with Laravel. We'll read the CSV file using Laravel's LazyCollection, DB, fopen(), and fgetcsv() functions, then put the information in the database.

So, let's simply see the example code:

Step 1: Install Laravel

first of all, we need to get a fresh Laravel version application using the bellow command, So open your terminal OR command prompt and run the bellow command:

composer create-project laravel/laravel example-app	

Step 2: Create Products Table

Here, we need to create a database migration for the "products" table and also we will create a model for the "products" table.

php artisan make:migration create_products_table	

Migration:

<?php
  
use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;
  
return new class extends Migration
{
    /**
     * Run the migrations.
     */
    public function up(): void
    {
        Schema::create('products', function (Blueprint $table) {
            $table->id();
            $table->string('name');
            $table->string('amount');
            $table->text('description');
            $table->timestamps();
        });
    }
  
    /**
     * Reverse the migrations.
     */
    public function down(): void
    {
        Schema::dropIfExists('products');
    }
};	

php artisan migrate	

Step 3: Create Seeder

Here, we will create ProductSeeder class and write code of import large csv file.

Make sure you created products.csv file with "name", "amount" and "description" columns. Put that file in public folder.

Now, let's create seeder class using following command:

php artisan make:seeder ProductSeeder	

Let's update following seeder code:

database/seeders/ProductSeeder.php

<?php
  
namespace Database\Seeders;
  
use Illuminate\Database\Console\Seeds\WithoutModelEvents;
use Illuminate\Database\Seeder;
use Illuminate\Support\Facades\DB;
use Illuminate\Support\LazyCollection;
  
class ProductSeeder extends Seeder
{
    /**
     * Run the database seeds.
     */
    public function run(): void
    {
        DB::disableQueryLog();
        DB::table('products')->truncate();
  
        LazyCollection::make(function () {
            $handle = fopen(public_path("products.csv"), 'r');
            
            while (($line = fgetcsv($handle, 4096)) !== false) {
                $dataString = implode(", ", $line);
                $row = explode(',', $dataString);
                yield $row;
            }
  
            fclose($handle);
  
        })
        ->skip(1)
        ->chunk(1000)
        ->each(function (LazyCollection $chunk) {
  
            $records = $chunk->map(function ($row) {
                          return [
                            "name" => $row[0],
                            "amount" => $row[1],
                            "description" => $row[2]
                          ];
                      })->toArray();
            
            DB::table('products')->insert($records);
  
        });
    }
}	

Now you need to run following command for run all listed seeder:

php artisan db:seed --class=ProductSeeder	

Now, you will able to see new row will be added in your products table.

Now I think you will understand how seeding works and we have to use it in our laravel app.

I hope it can help you...