Why doesn't Rebol new-line? treat the newline keyword and the newline character similarly?

Question

I would think that the following Rebol 3 code:

x: [newline 1 2]
y: [
1 2]

print x
print new-line? x
print y
print new-line? y

should output:

<empty line>
1 2
true
<empty line>
1 2
true

but what is output is:

<empty line>
1 2
false
1 2
true

Both blocks, when reduced, should yield a newline character followed by '1' and '2' and so, IMO, should print identically. Less clear is whether new-line? on the two blocks should also give the same result since the newline keyword should be equivalent to the literal newline for this kind of test.

Answer 1

Let's start with NEWLINE: newline is a word bound to a char! value:

>> ? newline
NEWLINE is a char of value: #"^/"

That's Rebol's escape sequence for Unicode codepoint U+000A, commonly used as line feed ("LF") control code.

So your first example code [newline 1 2] has nothing to do with the NEW-LINE function. It simply describes a block containing three values: newline (a word!), 2 (an integer!), and 3 (another integer!). If you REDUCE the block from your first example, you'll get another block of three values: char!, integer!, and integer!:

>> reduce [newline 1 2]
== [#"^/" 1 2]

Now PRINT does not only REDUCE, it does REFORM (first REDUCE, then FORM) a block argument. FORM of a block converts the elements to a string representation and then joins those with spaces in between:

>> form [1 2 3]
== "1 2 3"

Putting those pieces together we finally know how to arrive at the output you are seeing for your first example:

>> basis: [newline 1 2 3]
== [newline 1 2 3]
>> step1: reduce basis
== [#"^/" 1 2 3]
>> step2: form step1
== "^/1 2 3"
>> print step2

1 2 3

So the question remains, why does the second example not print identically?

That's because FORM (used by PRINT, as described above) does not respect the NEW-LINE flag when converting from a block to a string.

This flag is "meta-data", not unlike e.g. the index position of an element within a block. So just as you don't have elements at index positions 8 and 6 just because you write a block like [8 6], you don't set the new-line flag for a position just because you happen to put an element there which is a character that on some systems represents a line break: [1 newline 2].

And this finally brings us to the last part of the puzzle: NEW-LINE? does not check if a given string represents a line break. It checks if a block (at its current position) has the new-line flag set.

Answer 2

The flag which is checked by new-line? and set by new-line is used only by LOAD and MOLD. For all other semantic purposes in the program, it might as well not be there.

Therefore your x and y are completely different. Note that:

x: [newline 1 2]
y: [
1 2]

3 = length? x
2 = length? y

It's a quirk of Rebol that it singles out this one bit of whitespace information to stow in a hidden place. But arguably the choice to break a line represents something that is often significant in source, that if you reflect it back out into text you'd like to preserve more than the rest of the whitespace.