CODEWITHSUNDEEP
bashunixsed
Chris Edwards
Chris Edwards

Reputation: 1558

Sed command doesnt fully parse input file

I have a 5.1GB file which is one line of text with no line breaks.

I have been provided with the following command to append a new line character after every {...}:

's/{[^}]*}/&\n/g' input > output.txt

This works as intended, however it doesn't fully complete, it doesn't throw an error messages to the console either.

The reason it doesn't complete is because output.txt is always 1.1GB so I am missing ~3.9GB of data.

I have also tried to do an inline sed -i 's/{[^}]*}/&\n/g' input which also produces a 1.1GB file.

Is sed limited to a specific output size? I couldn't find a similar use case.

In addition I tried using tr which did produce the full output size but it seems you cannot replace 1 character with multiple with the tr command. This didn't work:

tr '}' '}\n' < input > output.txt

Is there a way to make sed complete the file fully?

Upvotes: 0

Views: 201

Answers (2)

NeronLeVelu
NeronLeVelu

Reputation: 10039

try maybe option -u for unbuffered

sed -u 's/{[^}]*}/&\n/g' input > output.txt

an also

sed -u 's/{[^}]*}/&\n/;P;D' input > output.txt

Upvotes: 0

choroba
choroba

Reputation: 242038

What about Perl?

perl -e '$/ = "}"; print "$_\n" while <>' input > output.txt
  • $/ is the input record separator.
  • <> is the diamond operator - it reads blocks delimited by $/ from the input file.
  • $_ is the topic variable, populated by while <>.

Upvotes: 1

Related Questions