Converting a simple AppEngine Java project to use modules

Question

This question is similar to this one, but that did not solve my problem.

I have a very simple Google AppEngine / Java application. It has been running since 2011, and does not use maven or other fancy stuff that I don't think I need. Recently, I added Cloud Endpoints to this application. I did not use generated endpoint-libs, because I did not seem to need that, and everything works fine without it.

The application has had a frontend and a backend for some time. I am now trying to convert these to modules: The frontend will become the default module, and the backend will become another module.

The structure of my old project is like this:

project
 |- src
 |   |- ... Java source files ...
 |
 |- war
 |   |- WEB-INF
 |   |   |- appengine.xml
 |   |   |- backends.xml
 |   |   |- cron.xml
 |   |   |- web.xml

I implemented cloud endpoints by providing Java classes with the right annotations. No fancy maven generating magic.

I understand that I need to create a directory for each module, like this:

project
 |- default
 |   |- WEB-INF
 |   |   |- appengine.xml
 |   |   |- cron.xml
 |   |   |- web.xml
 |
 |- module
 |   |- WEB-INF
 |   |   |- appengine.xml
 |   |   |- web.xml
 |
 |- META-INF
 |   |- appengine-application.xml
 |   |- application.xml

My questions are:

• Where should I put the src directory?
• Should I declare my cloud endpoint classes in default/WEB-INF/web.xml?
• Can each module have its own WEB-INF/cron.xml?

If it seems like I don't know what I am doing, that is probably right, but I don't want to have to put everything in a maven pom-file, write gradle scripts etcetera, and focus on the actual application instead. It's probably because I grew up with vi and emacs, in a time when we wrote code ourselves. ;)

Update:

I put the src directory under project at the same level as default and module. The compiled Java classes appear under default/WEB-INF/classes, which suggests that I did something right. GAE generates a *.api file in default/WEB-INF, which I did not see before when not using modules.

Locally, I can see my cloud endpoints APIs, and I can use them. When I deploy to AppEngine, and try to use the API explorer, I get an exception:

/_ah/spi/BackendService.getApiConfigs java.lang.NullPointerException at com.google.api.server.spi.SystemServiceServlet.execute(SystemServiceServlet.java:100) at com.google.api.server.spi.SystemServiceServlet.doPost(SystemServiceServlet.java:71) at javax.servlet.http.HttpServlet.service(HttpServlet.java:637) at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)

etcetera...

I did add OAuth2 credentials and set them in my cloud endpoint configuration. I could not find the code for SystemServiceServlet, but my guess would be that it cannot find my API classes (which are configured in default/WEB-INF/web.xml).

Another update:

I learned that AppEngine modules require an enterprise archive (ear) structure, and deploying like a simple GAE application is not going to work. There is no 'single press of a button' deployment. I followed the instructions in Programming Google App Engine with Java and ended up with a bunch of Eclipse projects. It is rather enterprisey, but I can get it to throw the same exception as the simple version I deployed earlier. I wonder if I made any progress at all.

Answer 1

Eventually, I followed the instructions given in Programming Google App Engine with Java (chapter 5) closely, and gradually added my old code. This worked, and I did not get the null-pointer exception any more (don't know what caused it).

Some points to be aware of:

• Modules do not share code. In Eclipse you can create a link in one project to a source folder in another project. This is the best way to share code that I could find.
• You must use the Java EE perspective in Eclipse. This has a 'Servers' tab, which includes a GAE button with a "Deploy to remote server" menu entry. This is the way to deploy your modules. The "Google / Deploy to AppEngine" context-menu entry does not work for module / enterprise projects.
• Special files like cron.xml and datastore-indexes.xml must be in the WEB-INF directory of the 'default' module.
• Routing URLs to modules on Google servers is very different from the development server.
• On the development server, the local datastore is buried somewhere in the Eclipse directory. Fix this by changing the working directory in the server configuration: Double-click on the server. This opens a ‘server editor’. Fill in Additional VM argument: -Ddatastore.backing_store=/C:/wherever/local_db.bin

The directory structure of my project now looks like this:

project
 |- default
 |   |- src
 |   |- WebContent
 |   |   |- WEB-INF
 |   |   |   |- appengine.xml
 |   |   |   |- cron.xml
 |   |   |   |- web.xml
 |
 |- module
 |   |- WebContent
 |   |   |- <HTML, CSS etcetera>
 |   |   |- WEB-INF
 |   |   |   |- appengine.xml
 |   |   |   |- web.xml
 |
 |- ear     (the EAR project)
 |   |- EarContent
 |   |   |- META-INF
 |   |   |   |- appengine-application.xml
 |   |   |   |- application.xml

Most of this was generated by following the instructions in the aforementioned book.

I hope this will help others struggling with making their AppEngine projects modular.

Answer 2

While the docs strongly suggest you need an ear to do this, you don't. You can just add a module definition in appengine-web.xml and your war will deploy to the specified module. Default module:

<module>default</module>

And for a backend

<module>backend</module>
<manual-scaling> ... </manual-scaling>

This has the benefit of using the same code on default and other modules, but requires some way of switching it at deploy time (without grinding your gears more I use maven profiles), you could just have another branch or do it manually. Then you should be able to deploy normally, but you'll need to do it twice.

This is the easiest migration path I've seen, setting up an ear is painful if you've already got a war.

Answer 3

I've created Appstart (https://github.com/omerio/appstart) a boilerplate maven based multi-module App Engine application that demonstrates the use of a few technologies including Cloud Endpoints and has 3 modules a fronend module, backend module and common module which includes all the common classes shared between the modules.

The projects are setup inside a parent folder 'appstart' with a parent maven POM. You can easily checkout the project, remove what you don't need, add your code and you should have a multi-module project. Alternatively you can structure your code similar to appstart. In this case you need to do the following:

• In your project folder add a parent pom with <packaging>pom</packaging> and add your modules to it (see the appstart parent pom as an example).
• For each of your modules include a pom which has a parent element <parent> (see the appstart-frontend pom as an example).
• You need to create a project with ear packaging (Enterprise ARchive), this facilitates the deployment of all your modules in one go (see appstart-ear as an example)

Your Cloud Endpoints can be declared in your default/WEB-INF/web.xml as with appstart's web.xml.

An example cron.xml is also included with the appstart-frontend. The cron.xml file must be added to the default module as mentioned in the App Engine docs. To invoke a cron job on a module simply include the <target>my-module</target> element in the cron.xml.