CODEWITHSUNDEEP
rubyregexnewline
brainburn
brainburn

Reputation: 55

Why is \r\n being converted to \n when a string is saved to a file?

The string is originating as a return value from: 

> msg = imap.uid_fetch(uid, ["RFC822"])[0].attr["RFC822"]

In the console if I type msg, a long string is displayed with double quotes and \r\n separating each line:

> msg
"Delivered-To: [email protected]\r\nReceived: by xx.xx.xx.xx with SMTP id;\r\n"

If I match part of it with a regex, the return value has \r\n:

> msg[/Delivered-To:.*?\s+Received:/i]
=> "Delivered-To: [email protected]\r\nReceived:"

If I save the string to a file, read it back in and match it with the same regex, I get \n instead of \r\n:

> File.write('test.txt', msg)
> str = File.read('test.txt')
> str[/Delivered-To:.*?\s+Received:/i]
=> "Delivered-To: [email protected]\nReceived:"

Is \r\n being converted to \n when the string is saved to a file? Is there a way to save the string to a file, read it back in without the line endings being modified?

Upvotes: 3

Views: 835

Answers (1)

the Tin Man
the Tin Man

Reputation: 160551

This is covered in the IO.new documentation:

The following modes must be used separately, and along with one or more of the modes seen above.

"b"  Binary file mode
     Suppresses EOL <-> CRLF conversion on Windows. And
     sets external encoding to ASCII-8BIT unless explicitly
     specified.

"t"  Text file mode

In other words, Ruby, like many other languages, senses the OS it's on and will automatically translate line-ends between "\r\n" <-> "\n" when reading/writing a file in text mode. Use binary mode to avoid translation. 

str = File.read('test.txt')

A better practice would be to read the file using foreach, which negates the need to even care about line-endings; You'll get each line separately. An alternate is to use readlines, however it uses slurping which can be very costly on large files.

Also, if you're processing mail files, I'd strongly recommend using something written to do so rather than write your own. The Mail gem is one such package that's pre-built and well tested.

Upvotes: 10

Related Questions