CODEWITHSUNDEEP
variablesracketlexer
Petlover620
Petlover620

Reputation: 115

How to create a variable with letters and numbers for a lexer in racket?

I am creating a lexer for a simple language in racket. This language allows variables that contain letters and numbers.

I understand how to create a numeric value:

(define-tokens names-and-values (NUMERICVALUE))

(define langlexer
  (lexer-src-pos
    [(repetition 1 +inf.0 numeric) (token-NUMERICVALUE (string->number lexeme))]))

And I also understand how to create a variable with only letters:

(define-tokens names-and-values (IDENTIFIER))

(define langlexer
  (lexer-src-pos
    [(repetition 1 +inf.0 alphabetic) (token-IDENTIFIER lexeme)]))

But I am confused on how to combine the two without the lexer splitting the letters and numbers apart. Is there a way to concatenate the two?

Upvotes: 1

Views: 259

Answers (1)

soegaard
soegaard

Reputation: 31147

Assuming you are using racket/lexer, use (union numeric alphabetic) to match digits or letters.

(define langlexer
  (lexer-src-pos
    [(repetition 1 +inf.0 (union numeric alphabetic)) 
     (if (string->number lexeme)
         (token-NUMERICVALUE (string->number lexeme))
         (token-IDENTIFIER lexeme))]))

Upvotes: 1

Related Questions