CODEWITHSUNDEEP
algorithmperltext-files
Srive Kobz
Srive Kobz

Reputation: 35

Dividing a big text file to short files

I want to divide (or split) a big txt file into short txt files, for example I have file.txt and it's 700 MB, and I want to divide it into seven (100 MB) txt files.

What am I going to do? Is there any way to divide it without opening the text file and change the content of text file? If there are known algorithms for this problem, please share them. Thanks

Upvotes: 0

Views: 763

Answers (5)

Bill Ruppert
Bill Ruppert

Reputation: 9026

The Perl distribution includes a split utility, if you happen to not be on Unix. It uses -l for line split.

Upvotes: 0

Nazar Merza
Nazar Merza

Reputation: 3454

In case, you want to have both options of splitting by size and by number of line (which in some cases you need) and it is in Windows environment, you may use a program called GSplit. I once had this problem and it worked for me. You can google it to download.

Upvotes: 0

Nrj
Nrj

Reputation: 6841

Simple approach -

  1. Get the file length, decide the chunk size

  2. See how many files you want to create - (n = file length/ chunk size). If any remainder, number of files would be n + 1.

  3. Open the file in read mode.

  4. Create a file in write mode. (probably append a suffix 1..n to indicate a fragment)

  5. write chunk number of bytes (or remainder bytes if remainder < chunk size) to the file, close this file.

  6. repeat 4-5 for n fragments.

Upvotes: 2

Eduardo Ivanec
Eduardo Ivanec

Reputation: 11862

Just in case it applies to your environment and needs, the standard Unix solution is to use split:

split -C 100M input.txt

-C forces split to split by line instead of byte/char. Use -b instead if you prefer to split at a specific byte.

Upvotes: 7

Martin Beckett
Martin Beckett

Reputation: 96177

If you are on unix

split -b100m filename part will split filename into parta, partb, partc etc

Upvotes: 2

Related Questions