Xtext global auto-generation

Question

In Xtext, how does one auto-generate a single file containing information from multiple model files.

Consider the following simple Xtext grammar.

grammar org.example.people.People with org.eclipse.xtext.common.Terminals

generate people "http://www.example.org/people/People"

People:
    people+=Person*;

Person:
    'person' name=ID ';';

In the launched workspace I create a project with two files, friends.people

// friends
person Alice;
person Bob;

and enemies.people

// enemies
person Malice;
person Rob;

How do I auto-generate a single file listing everyone when the global index changes?

Alice
Bob
Malice
Rob

Answer 1

I added the validation code to the answer of fundagain to eliminate invalid resources. However, this will not work when last modified resource is invalid because doGenerate is not invoked when invalid. When any valid resource is saved invalid resources will be discarded from all.txt .

override doGenerate(ResourceSet rs, IFileSystemAccess2 fsa, IGeneratorContext context) {
    var valid_rs = new ArrayList<Resource>
    for(r : rs.resources)
        if (( r as XtextResource)
            .getResourceServiceProvider()
            .getResourceValidator()
            .validate(r,CheckMode.ALL, null)
            .map(issue | issue.severity)
            .filter[it === Severity.ERROR]
            .size == 0) 
                valid_rs.add(r)

    val types = valid_rs.map(r|r.allContents.toIterable.filter(Person)).flatten
    fsa.generateFile("all.txt", people.compile)
}

Answer 2

For ease of future reference, here is the solution obtained by combining the various references given by Christian Dietrich. Note that the solution is Eclipse dependent.

Anyone who finds themselves with this requirement should perhaps try to find a better way of modelling the problem. For example a singleton model element All that generates the required list by finding everyone in the model using the standard API. This is independent of Eclipse, and requires non of the following complexity.

In the generator package of the grammar project, create an Java interface IPeopleGenerator extending IGenerator2.

package org.example.people.generator;

import org.eclipse.emf.ecore.resource.ResourceSet;
import org.eclipse.xtext.generator.IFileSystemAccess2;
import org.eclipse.xtext.generator.IGenerator2;
import org.eclipse.xtext.generator.IGeneratorContext;

public interface IPeopleGenerator extends IGenerator2{
    public void doGenerate(ResourceSet input, IFileSystemAccess2 fsa, IGeneratorContext context);
}

and edit the existing generator PeopleGenerator as follows.

/*
 * generated by Xtext 2.14.0
 */
package org.example.people.generator

import org.eclipse.emf.ecore.resource.Resource
import org.eclipse.emf.ecore.resource.ResourceSet
import org.eclipse.xtext.generator.IFileSystemAccess2
import org.eclipse.xtext.generator.IGeneratorContext
import org.example.people.people.Person

/**
 * Generates code from your model files on save.
 * 
 * See https://www.eclipse.org/Xtext/documentation/303_runtime_concepts.html#code-generation
 */
class PeopleGenerator implements IPeopleGenerator {

    override doGenerate(ResourceSet rs, IFileSystemAccess2 fsa, IGeneratorContext context) {

    val people = rs.resources.map(r|r.allContents.toIterable.filter(Person)).flatten
    fsa.generateFile("all.txt", people.compile)
    }

    override afterGenerate(Resource input, IFileSystemAccess2 fsa, IGeneratorContext context) {
    }

    override beforeGenerate(Resource input, IFileSystemAccess2 fsa, IGeneratorContext context) {
    }

    override doGenerate(Resource input, IFileSystemAccess2 fsa, IGeneratorContext context) {
    }

  def compile (Iterable<Person> entities) '''
    «FOR e : entities»
    «e.name»
    «ENDFOR»
  '''

}

and add the method

def Class<? extends IPeopleGenerator> bindIPeopleGenerator () {
    return PeopleGenerator
}

to the existing runtime module PeopleRuntimeModule in the grammar project.

Work needs to be done in the UI project org.example.people.ui. Consequently this solution is Eclipse dependent.

Create a Java class org.example.people.ui.PeopleBuilderParticipant as follows (the complexity being the need to ensure that the global generated file is only created once).

package org.example.people.ui;

import java.util.List;

import org.eclipse.core.runtime.CoreException;
import org.eclipse.core.runtime.IProgressMonitor;
import org.eclipse.core.runtime.NullProgressMonitor;
import org.eclipse.emf.ecore.resource.Resource;
import org.eclipse.xtext.builder.BuilderParticipant;
import org.eclipse.xtext.builder.EclipseResourceFileSystemAccess2;
import org.eclipse.xtext.builder.MonitorBasedCancelIndicator;
import org.eclipse.xtext.generator.GeneratorContext;
import org.eclipse.xtext.resource.IContainer;
import org.eclipse.xtext.resource.IResourceDescription;
import org.eclipse.xtext.resource.IResourceDescription.Delta;
import org.eclipse.xtext.resource.IResourceDescriptions;
import org.eclipse.xtext.resource.impl.ResourceDescriptionsProvider;
import org.example.people.generator.IPeopleGenerator;

import com.google.inject.Inject;

public class PeopleBuilderParticipant extends BuilderParticipant {

    @Inject
    private ResourceDescriptionsProvider resourceDescriptionsProvider;

    @Inject
    private IContainer.Manager containerManager;

    @Inject(optional = true)
    private IPeopleGenerator generator;

    protected ThreadLocal<Boolean> buildSemaphor = new ThreadLocal<Boolean>();

    @Override
    public void build(IBuildContext context, IProgressMonitor monitor) throws CoreException {
        buildSemaphor.set(false);
        super.build(context, monitor);
    }

    @Override
    protected void handleChangedContents(Delta delta, IBuildContext context,
            EclipseResourceFileSystemAccess2 fileSystemAccess) throws CoreException {
        super.handleChangedContents(delta, context, fileSystemAccess);
        if (!buildSemaphor.get() && generator != null) {
            invokeGenerator(delta, context, fileSystemAccess);
        }
    }
    private void invokeGenerator(Delta delta, IBuildContext context, EclipseResourceFileSystemAccess2 access) {
        buildSemaphor.set(true);
        Resource resource = context.getResourceSet().getResource(delta.getUri(), true);
        if (shouldGenerate(resource, context)) {
            IResourceDescriptions index = resourceDescriptionsProvider.createResourceDescriptions();
            IResourceDescription resDesc = index.getResourceDescription(resource.getURI());
            List<IContainer> visibleContainers = containerManager.getVisibleContainers(resDesc, index);
            for (IContainer c : visibleContainers) {
                for (IResourceDescription rd : c.getResourceDescriptions()) {
                    context.getResourceSet().getResource(rd.getURI(), true);
                }
            }

            MonitorBasedCancelIndicator cancelIndicator = new MonitorBasedCancelIndicator(
                    new NullProgressMonitor()); //maybe use reflection to read from fsa
            GeneratorContext generatorContext = new GeneratorContext();
            generatorContext.setCancelIndicator(cancelIndicator);
            generator.doGenerate(context.getResourceSet(), access, generatorContext);
        }
    }

}

and bind this build participant by adding

override  Class<? extends IXtextBuilderParticipant>  bindIXtextBuilderParticipant() {       
    return PeopleBuilderParticipant;
}

to the existing UI module org.example.people.ui.PeopleUiModule.