Custom terminals in XText

Question

I've started to learn XText recently and tried to do a simple thing - have a custom terminals definition instead of importing org.eclipse.xtext.common.Terminals.

First I've created BlahLang.xtest via wizard in x.mvmn.blah.xtext package.

Then I've checked tutorials and done two things: a) Created custom terminals file in same package where my xtext file was. Namely Test.xtext in x.mvmn.blah.xtext.

The contents of the file were same as in Terminals.xtext, but with corrected package declaration:

grammar x.mvmn.blah.xtext.Test hidden(WS, ML_COMMENT, SL_COMMENT)

import "http://www.eclipse.org/emf/2002/Ecore" as ecore

terminal ID:
    '^'? ('a'..'z' | 'A'..'Z' | '_') ('a'..'z' | 'A'..'Z' | '_' | '0'..'9')*;

terminal INT returns ecore::EInt:
    ('0'..'9')+;

terminal STRING:
    '"' ('\\' . /* 'b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\' */ | !('\\' | '"'))* '"' |
    "'" ('\\' . /* 'b'|'t'|'n'|'f'|'r'|'u'|'"'|"'"|'\\' */ | !('\\' | "'"))* "'";

terminal ML_COMMENT:
    '/*'->'*/';

terminal SL_COMMENT:
    '//' !('\n' | '\r')* ('\r'? '\n')?;

terminal WS:
    (' ' | '\t' | '\r' | '\n')+;

terminal ANY_OTHER:
    .;

b) In my BlahLang.xtext file I've imported that file via 'with' declaration instead of org.eclipse.xtext.common.Terminals that was placed there by wizard:

grammar x.mvmn.blah.xtext.BlahLang with x.mvmn.blah.xtext.Test

generate blahLang "http://www.mvmn.x/blah/xtext/BlahLang"

Model:
    greetings+=Greeting*;

Greeting:
    'Hello' name=ID '!';

According to all documentation and tips I found online everything should've been fine. But nothing works - the files generated by XText are referencing non-existing classes:

x.mvmn.blah.xtext.services.TestGrammarAccess
x.mvmn.blah.xtext.TestStandaloneSetup
x.mvmn.blah.xtext.validation.TestValidator

I've tried different workarounds (played around with adding "generate" to Test.xtext etc) but nothing worked. I'm practically stuck and have no slightest idea what is wrong and how can I fix things.

Please help.

Answer 1

Thanks to Sebastian's suggestion I've finally managed to make XText produce code that won't show compile errors by adding this generator component to my GenerateBlahLang.mwe2 (mostly copied from existing generator, with some parts commented out due to error "You may not generate an ANTLR parser for a grammar without production rules" showing up during generation):

component = Generator auto-inject {
        pathRtProject = runtimeProject
        pathUiProject = "${runtimeProject}.ui"
        pathTestProject = "${runtimeProject}.tests"
        projectNameRt = projectName
        projectNameUi = "${projectName}.ui"
        encoding = encoding
        language = {
            uri = "classpath:/x/mvmn/blah/xtext/Test.xtext"// Java API to access grammar elements (required by several other fragments)
            fragment = grammarAccess.GrammarAccessFragment auto-inject {}

            // generates Java API for the generated EPackages
            fragment = ecore.EMFGeneratorFragment auto-inject {}

            // the old serialization component
            // fragment = parseTreeConstructor.ParseTreeConstructorFragment auto-inject {}    

            // serializer 2.0
            fragment = serializer.SerializerFragment auto-inject {
                generateStub = false
            }

            // a custom ResourceFactory for use with EMF
            fragment = resourceFactory.ResourceFactoryFragment auto-inject {}

            // The antlr parser generator fragment.
            //fragment = parser.antlr.XtextAntlrGeneratorFragment auto-inject {
            //  options = {
            //      backtrack = true
            //  }
            //}

            // Xtend-based API for validation
            fragment = validation.ValidatorFragment auto-inject {
            //    composedCheck = "org.eclipse.xtext.validation.ImportUriValidator"
            //    composedCheck = "org.eclipse.xtext.validation.NamesAreUniqueValidator"
            }

            // old scoping and exporting API
            // fragment = scoping.ImportURIScopingFragment auto-inject {}
            // fragment = exporting.SimpleNamesFragment auto-inject {}

            // scoping and exporting API
            fragment = scoping.ImportNamespacesScopingFragment auto-inject {}
            fragment = exporting.QualifiedNamesFragment auto-inject {}
            fragment = builder.BuilderIntegrationFragment auto-inject {}

            // generator API
            fragment = generator.GeneratorFragment auto-inject {}

            // formatter API
            fragment = formatting.FormatterFragment auto-inject {}

            // labeling API
            fragment = labeling.LabelProviderFragment auto-inject {}

            // outline API
            fragment = outline.OutlineTreeProviderFragment auto-inject {}
            fragment = outline.QuickOutlineFragment auto-inject {}

            // quickfix API
            fragment = quickfix.QuickfixProviderFragment auto-inject {}

            // content assist API
            fragment = contentAssist.ContentAssistFragment auto-inject {}

            // generates a more lightweight Antlr parser and lexer tailored for content assist
            //fragment = parser.antlr.XtextAntlrUiGeneratorFragment auto-inject {}

            // generates junit test support classes into Generator#pathTestProject
            fragment = junit.Junit4Fragment auto-inject {}

            // rename refactoring
            fragment = refactoring.RefactorElementNameFragment auto-inject {}

            // provides the necessary bindings for java types integration
            fragment = types.TypesGeneratorFragment auto-inject {}

            // generates the required bindings only if the grammar inherits from Xbase
            fragment = xbase.XbaseGeneratorFragment auto-inject {}

            // generates the required bindings only if the grammar inherits from Xtype
            fragment = xbase.XtypeGeneratorFragment auto-inject {}

            // provides a preference page for template proposals
            //fragment = templates.CodetemplatesGeneratorFragment auto-inject {}

            // provides a compare view
            fragment = compare.CompareFragment auto-inject {}
        }
    }

Answer 2

Sounds like you did not generate the proper infrastructure for your terminals grammar. You may want to adopt the same generator configuration as is used for Xtext's common.Terminals.

component = Generator auto-inject {
    pathRtProject = runtimeProject
    projectNameRt = runtimeProjectName
    language = {
        uri = "platform:/resource/x.mvmn.blah/src/x/mvmn/blah/xtext/Test.xtext"
        fragment = grammarAccess.GrammarAccessFragment auto-inject {}
        fragment = parseTreeConstructor.ParseTreeConstructorFragment {}
    }
}