CODEWITHSUNDEEP
macossearchvimterminal
Fedor
Fedor

Reputation: 39

MacOS complete search in huge text file and output results (find solution to TextPipe)

I have a text file ar. 50 GB size. I used to process it through TextPipe but atm only mac is available and no TextPipe access. Is it possible to initiate regex search in this file with good results saving to some other file per matching line? I was thinking about vim editor but have no sufficient knowledge on where to search for. Would appreciate any suggestions.

As an example let's assume that I have the code below in my initial.txt file and I want to save lines with "Lorem" in line processed.txt.

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

Upvotes: 0

Views: 631

Answers (2)

Matt
Matt

Reputation: 15081

Vim is "genetically" related to other text-processing tools, such as sed or grep. And it also has an embedded sophisticated scripting language, so it is perfectly capable of batch text processing.

But Vim is an interactive text editor, so it feels a little wrong to use it merely as a replacement of awk or grep. However, if you're going to learn and use it for both editing and scripting, it's elegant and powerful.

To get some taste of Vim, you can solve your problem as follows (typing ':' in normal mode will automatically switch into command mode):

:e initial.txt
:g/Lorem/.w! >>processed.txt

I was thinking about vim editor but have no sufficient knowledge on where to search for. Would appreciate any suggestions.

The main problem with Vim is that you have to start from the very beginning, i.e. to learn how to open, edit and save files, and even how to properly exit the application. So you should download and install it and run vimtutor. Next, you should get used to Vim's embedded help system (:h user-manual) which by far is the best Vim's feature.

If you look for more books and tutorials, you can start from here. IMHO, Steve Oualline's "Vi IMproved" is still the best for beginners; and Drew Neil's "Practical Vim" is highly recommended for advanced vimmers.

Upvotes: 1

phd
phd

Reputation: 94417

For fixed strings use fgrep:

fgrep Lorem initial.txt > processed.txt

For regular expressions use grep and egrep (they have slightly different regexp syntax).

Upvotes: 1

Related Questions