A single linux shell script executes multiple commands, and the commands are written into multiple shell scripts and executed in sequence. Total time

Question

I am a linux beginner , I am trying to import and export data using shell script. The data about 60,000,000 rows. When I execute the import and export commands separately from the command line, I find that perform very well. But I merged commands into a single shell script and the performance slowed down, I tried many times and I don't know why.

The following is the information of my test environment

• System version information

  Operating System: CentOS Linux 7 (Core)
       CPE OS Name: cpe:/o:centos:centos:7
            Kernel: Linux 3.10.0-1160.25.1.el7.centos.plus.x86_64
      Architecture: x86-64
  

• shell script content

Export_from_POSTGRES_lineitem.sh

psql postgres://postgres:rdpadmin@192.168.120.73/sf10 -c  "\copy lineitem to './Export_from_POSTGRES_lineitem.csv' with DELIMITER '|';" 2>&1

Load_to_MYSQL_lineitem.sh

mysql -h 127.0.0.1 -P 3306 -uroot -prdpuser --database test_db -e "load data local infile './Export_from_POSTGRES_lineitem.csv' into table lineitem fields terminated by '|';" 2>&1

---

What I'm doing is exporting a table on a postgres database on another machine to a local csv file and then importing the csv file into my local mysql database. Before executing the export script, I will manually delete the existing local csv file. The csv file exported each time is about 8.6GB. Before importing to mysql, I will use ** TRUNCATE ** to clear the original data in the table. I make sure it's a clean environment before each test.

When I execute the Export script alone, it takes about 4 minutes.

When I execute the Load script alone, it takes about 5 minutes 30 seconds.

I have tested it many times, and the time is only within 20 seconds.

enter image description here

But when I merged the two scripts and executed them, I found that the time consuming increased.

• enter image description here

• enter image description here

I don't know what caused this, I hope I have described the problem I encountered in detail, and I hope to get your suggestions. If other information is needed, I will give it in time.