CODEWITHSUNDEEP
javacompiler-constructionabstract-syntax-tree
Code freak
Code freak

Reputation: 695

How to access comments from the java compiler tree api generated ast?

I've used the java compiler tree api to generate the ast for java source files. However, i'm unable to access th comments in the source files.

So far, i've been unable to find a way to extract comments from source file .. is there a way using the compiler api or some other tool ?

Upvotes: 4

Views: 5669

Answers (5)

enissay
enissay

Reputation: 300

Just for the record. Now with Java 8 you have a whole interface to play with comments and documentation details here.

Upvotes: 1

Unni Kris
Unni Kris

Reputation: 3095

The comments obtained through getCommentList method of CompilationUnit will not have the comment body. Also the comments will not be visited, during and AST Visit. Inorder to visit the comments we have call accept method for each comment in the Comment List.

for (Comment comment : (List<Comment>) compilationUnit.getCommentList()) {

    comment.accept(new CommentVisitor(compilationUnit, classSource.split("\n")));
}

The body of the comments can be obtained using some simple logic. In the below AST Visitor for comments, we need to specify the Complied class unit and the source code of the class during initialization.

import org.eclipse.jdt.core.dom.ASTNode;
import org.eclipse.jdt.core.dom.ASTVisitor;
import org.eclipse.jdt.core.dom.BlockComment;
import org.eclipse.jdt.core.dom.CompilationUnit;
import org.eclipse.jdt.core.dom.LineComment;

public class CommentVisitor extends ASTVisitor {

    CompilationUnit compilationUnit;

    private String[] source;

    public CommentVisitor(CompilationUnit compilationUnit, String[] source) {

        super();
        this.compilationUnit = compilationUnit;
        this.source = source;
    }

    public boolean visit(LineComment node) {

        int startLineNumber = compilationUnit.getLineNumber(node.getStartPosition()) - 1;
        String lineComment = source[startLineNumber].trim();

        System.out.println(lineComment);

        return true;
    }

    public boolean visit(BlockComment node) {

        int startLineNumber = compilationUnit.getLineNumber(node.getStartPosition()) - 1;
        int endLineNumber = compilationUnit.getLineNumber(node.getStartPosition() + node.getLength()) - 1;

        StringBuffer blockComment = new StringBuffer();

        for (int lineCount = startLineNumber ; lineCount<= endLineNumber; lineCount++) {

            String blockCommentLine = source[lineCount].trim();
            blockComment.append(blockCommentLine);
            if (lineCount != endLineNumber) {
                blockComment.append("\n");
            }
        }

        System.out.println(blockComment.toString());

        return true;
    }

    public void preVisit(ASTNode node) {

    }
}

Edit: Moved splitting of source out of the visitor.

Upvotes: 5

Ira Baxter
Ira Baxter

Reputation: 95334

Our SD Java Front End is a Java parser that builds ASTs (and optionally symbol tables). It captures comments directly on tree nodes.

The Java Front End is a member of a family of compiler langauge front ends (C, C++, C#, COBOL, JavaScript, ...) all of which are supported by DMS Software Reengineering Toolkit. DMS is designed to process languages for the purposes of transformation, and thus can capture comments, layout and formats to enable regeneration of code preserving the original layout as much as possible.

EDIT 3/29/2012: (in contrast to answer posted for doing this with ANTLR)

To get a comment on an AST node in DMS, one calls the DMS (lisp-like) function 

  (AST:GetComments <node>)

which provide access to the array of comments associated with the AST node. One can inquire about the length of this array (may be null), or for each array element, ask for any of these properties: (AST:Get... FileIndex, Line, Column, EndLine, EndColumn, String (exact Unicode comment content).

Upvotes: 5

Code freak
Code freak

Reputation: 695

Managed to solve the problem by using the getsourceposition() and some string manipulation (no regex was needed)

Upvotes: 0

erickson
erickson

Reputation: 269667

You might to use a different tool, like ANTLR's Java grammar. javac has no use for comments, and is likely to discard them completely. The parsers upon which tools like IDEs are built are more likely to retain comments in their AST.

Upvotes: 0

Related Questions