CODEWITHSUNDEEP
antlr3
Kalaivani
Kalaivani

Reputation: 21

Hidden token management in antlr3

I am using Antlr3.

print a;




print b;

This is my input file(Stored in DB) to the parser.And in lexer I kept the newline(\n) in hidden channel.So the parser will ignore all the newlines(\n).After executing it when I fetch the input file (from DB) it will change like

print a;
print b;

My new lines(\n) are missing!!!...How can I retrieve these newlines(\n)

Upvotes: 1

Views: 62

Answers (1)

Mike Lischke
Mike Lischke

Reputation: 53407

The token stream still has all of them. Simply iterate over all the tokens and you will see your newlines in the loop. Each token comes with a token index, which is the index in the token stream. So, when you see 2 normal tokens with indices that include a gap you know there are hidden tokens and you can retrieve them via your token stream.

Upvotes: 1

Related Questions