GAE Endpoints Exception: service name mismatch

Question

I'm trying to setup REST API for my project on Google App Engine, and the has been deployed. However, calling the API causes an error 500 to be returned to the client, and in GAE's log, the following stack trace is shown.

Stack trace:

com.google.apphosting.runtime.jetty9.JettyLogger warn: /_ah/api/handler/v1/fetchlatest/all (JettyLogger.java:29)
java.lang.IllegalArgumentException: service name mismatch
    at com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)
    at com.google.api.control.aggregator.CheckRequestAggregator.check(CheckRequestAggregator.java:223)
    at com.google.api.control.Client.check(Client.java:177)
    at com.google.api.control.ControlFilter.doFilter(ControlFilter.java:249)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
    at com.google.api.control.ConfigFilter.doFilter(ConfigFilter.java:120)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
    at com.googlecode.objectify.ObjectifyFilter.doFilter(ObjectifyFilter.java:48)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
    at com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlobUploadFilter.java:125)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
    at com.google.apphosting.runtime.jetty9.SaveSessionFilter.doFilter(SaveSessionFilter.java:37)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
    at com.google.apphosting.utils.servlet.JdbcMySqlConnectionCleanupFilter.doFilter(JdbcMySqlConnectionCleanupFilter.java:60)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
    at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:48)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at com.google.apphosting.runtime.jetty9.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:297)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
    at org.eclipse.jetty.server.Server.handle(Server.java:534)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
    at com.google.apphosting.runtime.jetty9.RpcConnection.handle(RpcConnection.java:202)
    at com.google.apphosting.runtime.jetty9.RpcConnector.serviceRequest(RpcConnector.java:81)
    at com.google.apphosting.runtime.jetty9.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:108)
    at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.dispatchServletRequest(JavaRuntime.java:680)
    at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.dispatchRequest(JavaRuntime.java:642)
    at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:612)
    at com.google.apphosting.runtime.JavaRuntime$NullSandboxRequestRunnable.run(JavaRuntime.java:806)
    at com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:274)
    at java.lang.Thread.run(Thread.java:745)

I am unsure of what the problem is, but guessing the from the stack trace, could it be due to how Objectify's filter is configured in web.xml, and as such, all request are hitting Objectify first? Though I have to admit, this doesn't seem likely, since other request are routed to the correct servlets.

Objectify Filter:

<filter>
    <filter-name>ObjectifyFilter</filter-name>
    <filter-class>com.googlecode.objectify.ObjectifyFilter</filter-class>
</filter>
<filter-mapping>
    <filter-name>ObjectifyFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>

Answer 1

You'll get this issue if you follow the official doc at https://cloud.google.com/endpoints/docs/frameworks/java/adding-api-management

There is a section telling you to copy in some code to your web.xml, showing:

...
<init-param>
    <param-name>endpoints.serviceName</param-name>
    <param-value>${endpoints.project.id}.appspot.com</param-value>
</init-param>
...

And it's unclear how that variable get's resolved. They don't instruct you to change it to your actual project Id, so you think it gets updated by the plugin or something...

But it actually doesn't. It turns out they are taking some of the code from the github project at https://github.com/GoogleCloudPlatform/java-docs-samples/tree/master/appengine-java8/endpoints-v2-backend

But they leave out that the variable actually gets updated by a custom task that is in that project's build.gradle file.

// this replaces the ${endpoints.project.id} in appengine-web.xml and web.xml
task replaceProjectId(type: Copy) {
    from 'src/main/webapp/WEB-INF/'
    include '*.xml'
    into "build/exploded-${archivesBaseName}/WEB-INF"
    expand(endpoints:[project:[id:projectId]])
    filteringCharset = 'UTF-8'
}
assemble.dependsOn replaceProjectId

Hope this helps!

Answer 2

It seems that the web.xml contained the wrong init-param (endpoint.serviceName instead of endpoints.serviceName). And to top that off, even the param-value for the property was wrong. Admittedly, it took us longer than expected to figure out something that should have been caught in a peer review.