CODEWITHSUNDEEP
intellij-ideaantlr4
Dims
Dims

Reputation: 50989

Gradle can't find Antlr token file

I created a file MyLexer.g4 inside myproject/src/main/antlr/com/mypackage like:

lexer grammar MyLexer;

DIGIT : '0' .. '9' ;

...

WS  : [ \t\r\n]+ -> skip ;

and then trying to write parser in MyParser.g4 in the same directory:

grammar MyParser;

options
   { tokenVocab = MyLexer; }

SHORT_YEAR: DIGIT DIGIT;

unfortunately, when I run gradle task of generateGrammarSource, the following error occurs:

error(160): com\mypackage\MyParser.g4:4:18: cannot find tokens file MYPROJECT\build\generated-src\antlr\main\MyLexer.tokens

I.e. file is sought in incorrect place.

Actual file is created inside MYPROJECT\build\generated-src\antlr\main\com\mypackage\MyLexer.tokens

Upvotes: 16

Views: 4054

Answers (4)

Gleb
Gleb

Reputation: 33

The issue you're experiencing was addressed in ANTLR version 4.7.1. Starting from this version, there's an additional parameter -Xexact-output-dir that allows you to place the lexer and parser files in the exact directory you specify, such as src/main/antlr/com/mypackage. You can modify your build.gradle file like this:

gradleCopygenerateGrammarSource {
    outputDirectory = layout.buildDirectory.dir('generated-src/antlr/main/com/mypackage').get().asFile
    arguments = [
        '-visitor',
        '-package', 'com.mypackage',
        '-Xexact-output-dir'
    ]
}

By adding the -Xexact-output-dir argument, ANTLR will generate the .tokens file in the same directory as your grammar files, resolving the "cannot find tokens file" error.

Note: The outputDirectory setting explicitly defines where the generated files will be placed. This is particularly useful when you want to have more control over the location of your generated sources. In this case, it places the files in a subdirectory of your build directory, which is a common practice to keep generated sources separate from your main source code.

Upvotes: 1

Wenrui Meng
Wenrui Meng

Reputation: 1

As what suggested above, grammar file with package definition should be put under src/main/antlr folder. Once you compile it, it will generate files in the output folder. Then you can do the file move as what's desired based on the package path. Here is an relevant link how to move files based on your package

Generating ANTLR4 grammar files with package declaration in gradle

In the doLast method, it calls moveAntlrGeneratedFilesToTheirPackages to move the files to right folder to align with the package path.

Upvotes: 0

Sean
Sean

Reputation: 3015

As Steven Spungin said, you need to put your ANTLR source files in the directory src/main/antlr/ and not in a subdirectory of that directory.

You do not need to add the @header to your ANTLR source files. Use the following instead.

In your build.gradle you should have (modified for your version of ANTLR):

apply plugin: 'antlr'

dependencies {
    antlr "org.antlr:antlr4:4.7.1"
}

generateGrammarSource {
    arguments += ['-package', 'com.mypackage']
    outputDirectory = new File(buildDir.toString() + "/generated-src/antlr/main/com/mypackage/")
}

Upvotes: 13

Steven Spungin
Steven Spungin

Reputation: 29071

When generating your parser in a package by using:

@header {package org.acme.my.package;}

and declaring tokenVocab in your parser

options {tokenVocab = MyLanguage;}

The MyLanguageLexer.g4 and MyLanguageParser.g4 files must NOT BE in a package directory. due to a bug of sorts.

So this means /src/main/antlr/MyLanguageParser.g4 and not /src/main/antlr/com/acme/my/package/MyLanguageParser.g4.

The java files end up in the wrong directory in build/generated-src/antlr, but somehow make it to the correct directory in build/classes/java/main. And the .tokens file ends up where antlr expects it.

Keep in mind this will confuse your IDE; I had to add the compiled classes back to my compileClasspath to avoid visual class-not-found issues.

dependencies {
    testCompile fileTree('build/classes/java/main')
}

Upvotes: 5

Related Questions