Exception in generated editor: Xtext bug or error in grammar?

Question

After moving to Xtext 2.4.2 and Eclipse Kepler, a problem appeared in the editor of our DSL. Everything was perfectly working before (Xtext 2.3.x, Juno).

Our DSL editor throws a ClassCastException when we edit an initialization list: [1,2,3]

Steps to reproduce:

1. Create a new "Xtext State-Machine Example" project
2. Edit the grammar with the one below
3. Delete the package org.eclipse.xtext.example.fowlerdsl.generator (it will be regenerated at 4.)
4. Run the org.eclipse.xtext.example.fowlerdsl.GenerateStatemachine.mwe2 workflow
5. Run the eclipse application
6. Create a new file (example.statemachine) in an empty project and copy the folowing content

Grammar:

grammar org.eclipse.xtext.example.fowlerdsl.Statemachine
    with org.eclipse.xtext.common.Terminals
generate statemachine "http://www.eclipse.org/xtext/example/fowlerdsl/Statemachine"

Statemachine : {Statemachine}
    (vars += VarDeclWithOptionalInit)*
;
VarDeclWithOptionalInit returns Variable:
    VarDecl ('=' value=AstExpression)?
;
VarDecl returns Variable:
    'var' name = ID
;
AstExpression:
      ExpressionList
    | {AstExpression} INT
;
ExpressionList:
    '[' expressions+=AstExpression (',' expressions+=AstExpression)* ']'
;

Content of example.statemachine

//Adding space between chars inside [] cause Exception
var example1 = [1, 2, 3]

//Doing the same in [10, 20] or [30, 40] is Ok, but modifying
//the top-level list cause the Exception
var example2 = [[10, 20], 0, [30, 40], 1, 2]

I think the content of the example file is correct. But when I edit lists with new values, or when I add spaces around commas or values, the editor pop-up an error with a ClassCastException from the XtextReconcilierJob.

Do you think my grammar has something wrong, or is it a bug from Xtext side ?

Additional information

I fight against this for some days, and I collected some interesting information:

• In the example, editor throws only when editing elements from the top-level list, not from inner ones (see example2 variable)
• In my tests, sometimes a NullPointerException was throwed earlier in the call stack.
  • I suspect the method PartialParsingHelper.reparse(IParser parser, IParseResult prev, ReplaceRegion cr) to have a strange behavior
  • This method found the parent of the current modified element in the model, to call eSet() on this with the new value. But in our case, it try to call Statemachine.eSet() instead of Variable.eSet()
  • The ClassCastException comes from an equal feature id:
    • StateMachine.vars has a feature id == 0
    • Variable.value has a feature id == 0
    • But the 2 features have not the same type
  • If I add some feature BEFORE value in Variable, the Exception thrown (earlier) is a NullPointerException, because Variable.value has a feature id == 2 or 3 when StateMachine has only 2 features (0 and 1)

StackTrace of the error:

Thread [Worker-2] (Suspended (exception ClassCastException))
    StatemachineImpl.eSet(int, Object) line: 128
    StatemachineImpl(BasicEObjectImpl).eSet(EStructuralFeature, Object) line: 1071
    PartialParsingHelper.reparse(IParser, IParseResult, ReplaceRegion) line: 161
    StatemachineParser(AbstractAntlrParser).doReparse(IParseResult, ReplaceRegion) line: 136
    StatemachineParser(AbstractParser).reparse(IParseResult, ReplaceRegion) line: 48
    LazyLinkingResource(XtextResource).update(int, int, String) line: 220
    XtextDocumentReconcileStrategy.doReconcile(IRegion) line: 125
    XtextDocumentReconcileStrategy.reconcile(IRegion) line: 55
    XtextReconciler.doRun(XtextResource, IProgressMonitor) line: 329
    XtextReconciler.access$3(XtextReconciler, XtextResource, IProgressMonitor) line: 316
    XtextReconciler$1.process(XtextResource) line: 273
    XtextReconciler$1.process(Object) line: 1
    XtextReconciler$1(IUnitOfWork$Void<T>).exec(T) line: 36
    XtextDocument$XtextDocumentLocker(AbstractReadWriteAcces<P>).modify(IUnitOfWork<T,P>) line: 81
    XtextDocument$XtextDocumentLocker.modify(IUnitOfWork<T,XtextResource>) line: 201
    XtextDocument.internalModify(IUnitOfWork<T,XtextResource>) line: 98
    XtextReconciler.run(IProgressMonitor) line: 270
    Worker.run() line: 53

Answer 1

There is already a bugifix in the xText bugzilla. It is a patch to org.eclipse.parser.impl.PartialParsingHelper. To fix the issue:

1. Get the file from the patch.
2. Put the content in a separate class in your xText project
3. Register this class by overriding public Class bindIPartialParserHelper() in your RuntimeModule (a geenrated class in your project, extending org.eclipse.xtext.service.DefaultRuntimeModule)

The problem lies in the reparsing of an already parsed part of your language: the parent container of the old element was not correctly set. The new element was linked with the old element instead with the parent of the old element.

I guess you figured this already out on your own, but this instructions might help the next person having the same problem.

Answer 2

Xtext team confirmed that is is a bug with 2.4.x versions. See the related issue for more details and notification about the bugfix.