CODEWITHSUNDEEP
javalexerjava-compiler-api
Caffe Latte
Caffe Latte

Reputation: 1743

lexical analysis gives only one output?

I tested this example(lexe.java), but it gave me only one output.

I gave this text as a reader:

public class LexeTest{
 private int a = 14;
}

And the nextToken() function is :

 public Category nextToken () {
        if (inp.findWithinHorizon (tokenPat, 0) == null)
            return Category.EOF;
        else {
            lastLexeme = inp.match ().group (0);
            if (inp.match ().start (1) != -1)
                return nextToken ();
            else if (inp.match ().start (2) != -1) 
                return Category.IDENT;
            else if (inp.match ().start (3) != -1)
                return Category.NUMERAL;
            Category result = tokenMap.get (lastLexeme);
            if (result == null)
                return Category.ERROR;
            else
                return result;
        }
    }

Isdie the main method:

System.out.println(lexeObject.nextToken());

output is :

IDENT

Why? but the textfile contains multiple keywords? Anyone know what's the problem?

Upvotes: 1

Views: 55

Answers (2)

Ingo
Ingo

Reputation: 36329

Sorry to say that, but the problem seems to be your reading ability. Here is a part of the very first comment of the example you linked to (emphasis added by me):

A Lexer produces a stream of tokens taken from an arbitrary Reader supplied to the constructor, in response to repeated calls to the Lexer's nextToken routine.

Upvotes: 1

user207421
user207421

Reputation: 310860

If you only call nextToken() once, you will only get one token. I don't see why this is such a mystery. Try a loop.

Upvotes: 2

Related Questions