CODEWITHSUNDEEP
javascriptunicodegrammarquotespegjs
Speedy
Speedy

Reputation: 477

Allowing for quotes and unicode in PEG.js grammar definitions

How would one allow for single and double quotes, as well as unicode characters inside of a PEG.js grammar definition? To be more specific, I'd like to be able to capture strings that can contain both single and double quotes (will most likely have to be \ escaped) and all unicode characters.

At the moment I have something like the following:

_ name:$(PROP_ASCII+) CHAR_SQ val:$(PROP_ASCII_INNER*) CHAR_SQ

which would capture something like

key'value'

PROP_ASCII* is defined as

PROP_ASCII = [!-&(-<>-~] PROP_ASCII_INNER = [ -&(-~]

So this works fine and dandy if value contains standard ASCII characters and does not contain single quotes... But I'd like to support what I've described above, so something like this would become possible:

key'somé\'value\'☂'

Thoughts?

Upvotes: 5

Views: 3052

Answers (2)

dan
dan

Reputation: 2388

This example should get you going. It supports both single and double quotes which can also be escaped within the string.

Try it in the online editor.

Value
  = '"' chars:DoubleStringCharacter* '"' { return chars.join(''); }
  / "'" chars:SingleStringCharacter* "'" { return chars.join(''); }

DoubleStringCharacter
  = !('"' / "\\") char:. { return char; }
  / "\\" sequence:EscapeSequence { return sequence; }

SingleStringCharacter
  = !("'" / "\\") char:. { return char; }
  / "\\" sequence:EscapeSequence { return sequence; }

EscapeSequence
  = "'"
  / '"'
  / "\\"
  / "b"  { return "\b";   }
  / "f"  { return "\f";   }
  / "n"  { return "\n";   }
  / "r"  { return "\r";   }
  / "t"  { return "\t";   }
  / "v"  { return "\x0B"; }

Upvotes: 17

Speedy
Speedy

Reputation: 477

Found a solution inside this example file PEG.js JSON grammar. Unicode strings with escape characters can be defined as such:

string "string"
  = quotation_mark chars:char* quotation_mark { return chars.join(""); }

char
  = unescaped
  / escape
    sequence:(
        '"'
      / "\\"
      / "/"
      / "b" { return "\b"; }
      / "f" { return "\f"; }
      / "n" { return "\n"; }
      / "r" { return "\r"; }
      / "t" { return "\t"; }
      / "u" digits:$(HEXDIG HEXDIG HEXDIG HEXDIG) {
          return String.fromCharCode(parseInt(digits, 16));
        }
    )
    { return sequence; }

escape         = "\\"
quotation_mark = '"'
unescaped      = [\x20-\x21\x23-\x5B\x5D-\u10FFFF]

Upvotes: 1

Related Questions