CODEWITHSUNDEEP
splittext-filestext-editor
Yannick
Yannick

Reputation: 683

How to split text files that are too large to open with text editors?

We have a very large logfile, which we can not open with classic text editors for analysis (Notepad ++, UltraEdit, vscode, ...). Can I split the log file easy line by line so that I make several small logfiles that we can then display with the text editor?

Upvotes: 1

Views: 11934

Answers (3)

hornetbzz
hornetbzz

Reputation: 9357

head and tail are perfect for a one shot operation.

Linux power offers dozen of different solutions to meet the same result.

I believe split would be the command that you actually need for achieving what you want to do.

Alternatively, this task can also be automatized using split (with recent gnu as older ones do not have those options), as follows, provided that your files have an (single dot) extension :

SplitStartIndex=0 # you can name the splitted files starting with index 0, or any other value
MAXLINES=100 # this is the split result, ie you will get 100 lines for each file and the rest for the last file
for FILE in $BASEFILES # means $BASEFILES has to be your directory, even limited to the file(s) you need to split. for ex: BASEFILES=/home/coding/mybigcodefile.c
do

    # Count the big file lines
    countlines=$(cat $FILE | wc -l)

    # split file if size > $MAXLINES=100
    if [ $countlines -gt $MAXLINES ];then

        # output_prefix = path & filename_wo_extension & PREFIX && extension
        fullfilename_wo_extension=$(echo $FILE | cut -f 1 -d '.')

        # Split the file and append the orginal extension to the output file with profix     
        split --numeric-suffixes="$SplitStartIndex" --additional-suffix=".$EXT_BASE" -l $MAXLINES $FILE $fullfilename_wo_extension

        # Backup original file, just in case ;-)
        mv $FILE "$fullfilename_wo_extension.$EXT_BCKP"

    fi

done

Upvotes: 0

thorstenhirsch
thorstenhirsch

Reputation: 198

May I introduce to you the Unix/Linux command line tools, such as head and tail?

head -1000 $file > part1.txt
head -2000 $file | tail -1000 > part2.txt
head -3000 $file | tail -1000 > part3.txt
head -4000 $file | tail -1000 > part4.txt
[...]

Upvotes: 1

Yannick
Yannick

Reputation: 683

Ok, after a short search I found an easy-to-use tool from G.D.G Software called GSplit. The tool is free and I will tell you if it rocks.

Upvotes: 0

Related Questions