lexical analyzer in scheme

Question

hello guys im very new in programming scheme so im trying to build a lexical analyzer in scheme that basically reads a list for example < SUM + 34 > and the out put would be something like this

{ is the left bracket 
SUM is an identifier
+ is the plus operator
34 is a Digit
} is the right bracket

I'm using dr.scheme and the program does not exactly tell me what I am missing or hints of my mistakes. Everything is very new for me

This is what I have tried so far:

(define alist'( < SUM + 34 >))

(define (token? object)
  (and (list? object)
       (not (null? object))
       (eq? (car object) 'token)))

(define (ident-token? token)
  (type-token? token 'IDENTIFIER))
(define (digit-token? token)
  (type-token? token 'DIGIT))
(define (op-token? token)
  type-token? token 'OPERATOR)

(define (type-token? token type)
  (and(type-token? token)
      (eq? (name-token token) type)))

(define (name-token token)
  (cadr token))

(define (value-token token)
  (caddr token))

(define (spot-token)
  (cadddr token))

(define (assign-token name value spot)
  (list 'token name value spot))

(define *peek* '())

(define (check-token alist)
  (let ((token (if (null? *peek*)
                   (<token> in)
                   *peek)))
    (set! *peek* '())
    token))

(define (peek-token alist)
  (let ((token (if (null? *peek*)
                   (read-token in)
                   *peek*)))
    (set! *peek* (if token token '()))
    token))

(define (<token> alist)
  (let* (next-loc (next-spot in)) (next-char (peek-char in))
    (cond ((eof-object? next-char) #f)
          ((char-whitespace? next-char)
           (begin (check-char in)
                  (<token> in)))
          ((char-ident-initial? next-char)
           (<identifier> (list (check-char in)) next-loc in))
          (else
           (let ((next-char (check-char in)))
             (cond ((char=? #\( next-char)
                    (assign-token 'LEFT-PAREN "(" next-loc))
                   ((char=? #\) next-char)
                    (assign-token 'RIGHT-PAREN ")" next-loc))
                   ((char=? #\` next-char)
                    (assign-token 'ADD-OP "+" next-loc))
                   ((char=? #\] next-char)
                    (assign-token 'SUB-OP "-" next-loc))
                   ((char=? #\{ next-char)
                    (assign-token 'MULT-OP "*" next-loc))
                   ((char=? #\} next-char)
                    (assign-token 'DIV-OP "/" next-loc))
                   (else
                    (syntax-error next-loc next-char))))))))

Please guys I'm trying my best and nothing I try compiles. I have googled many things, but I couldn't find what could help me so.. even if you have tutorials or guides that would help me please share

Answer 1

I think you need a little guidance in reading the error messages.

When I run your program above in DrRacket, I get the error:

expand: unbound identifier in module in: token

At the same time token in this functions is colored red:

(define (spot-token)  
  (cadddr token))     ; <---

This means that the compiler hasn't seen the name token before.

Now since you didn't include descriptions of purpose for your functions, my guess is, that you forgot to use token in the list of arguments:

(define (spot-token token)   
  (cadddr token))     

It is important to catch mistakes as early as possible. My advice is each time you write a new function, write at least one to two tests of the function before beginning with the next function.