Can't connect to local Datastore

Question

I created a local Google Database Emulator on my Windows machine. When I run the emulator I get the following output:

C:\Users\susan\AppData\Local\Google\Cloud SDK>gcloud beta emulators datastore start --project=susantestproject
WARNING: Reusing existing data in [C:\Users\susan\AppData\Roaming\gcloud\emulators\datastore].
Executing: cmd /c C:\Users\susan\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gcd\gcd.cmd start --host=localhost --port=8578 --store_on_disk=True --consistency=0.9 --allow_remote_shutdown C:\Users\susan\AppData\Roaming\gcloud\emulators\datastore
[datastore] API endpoint: http://localhost:8578/datastore
[datastore] If you are using a library that supports the DATASTORE_LOCAL_HOST environment variable, run:
[datastore]
[datastore]   export DATASTORE_LOCAL_HOST=localhost:8578
[datastore]
[datastore] Invoking dev_appserver with args: [--disable_update_check --jvm_flag=-Doauth.is_admin=true --property=datastore.index_configuration_format=yaml --port=8578 --address=localhost --property=datastore.force_is_high_replication=true --property=datastore.default_high_rep_job_policy_unapplied_job_pct=10.0 --allow_remote_shutdown C:\Users\susan\AppData\Roaming\gcloud\emulators\datastore].
[datastore] Executing [C:\Program Files\Java\jre1.8.0_74\bin\java, -Doauth.is_admin=true, -javaagent:C:\Users\susan\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gcd\.appengine\lib\agent\appengine-agent.jar, -Xbootclasspath/p:C:\Users\susan\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gcd\.appengine\lib\override\appengine-dev-jdk-overrides.jar, -classpath, C:\Users\susan\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gcd\CloudDatastore.jar;C:\Users\susan\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gcd\.appengine\lib\impl\appengine-api.jar;C:\Users\susan\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gcd\.appengine\lib\appengine-tools-api.jar;C:\Users\susan\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\gcd\.appengine\lib\impl\appengine-api-stubs.jar, com.google.appengine.tools.development.DevAppServerMain, --property=kickstart.user.dir=C:\Users\susan\AppData\Local\Google\Cloud SDK, --disable_update_check, --property=datastore.index_configuration_format=yaml, --port=8578, --address=localhost, --property=datastore.force_is_high_replication=true, --property=datastore.default_high_rep_job_policy_unapplied_job_pct=10.0, --allow_remote_shutdown, C:\Users\susan\AppData\Roaming\gcloud\emulators\datastore]
[datastore] Apr 08, 2016 8:14:51 PM java.util.prefs.WindowsPreferences <init>
[datastore] WARNING: Could not open/create prefs root node Software\JavaSoft\Prefs at root 0x80000002. Windows RegCreateKeyEx(...) returned error code 5.
[datastore] Apr 08, 2016 8:14:51 PM com.google.apphosting.utils.jetty.JettyLogger info
[datastore] INFO: Logging to JettyLogger(null) via com.google.apphosting.utils.jetty.JettyLogger                                                       [datastore] Apr 08, 2016 8:14:51 PM com.google.appengine.tools.development.DevAppServerImpl setServerTimeZone                                          [datastore] WARNING: Unable to set the TimeZone to UTC (this is expected if running on JDK 8)
[datastore] Apr 08, 2016 8:14:51 PM com.google.apphosting.utils.jetty.JettyLogger info
[datastore] INFO: jetty-6.1.x
[datastore] Apr 08, 2016 8:14:51 PM com.google.apphosting.utils.jetty.JettyLogger info
[datastore] INFO: Started SelectChannelConnector@localhost:8578
[datastore] Apr 08, 2016 8:14:52 PM com.google.appengine.tools.development.AbstractModule startup
[datastore] INFO: Module instance default is running at http://localhost:8578/
[datastore] Apr 08, 2016 8:14:52 PM com.google.appengine.tools.development.AbstractModule startup
[datastore] INFO: The admin console is running at http://localhost:8578/_ah/admin
[datastore] Apr 08, 2016 8:14:52 PM com.google.appengine.tools.development.DevAppServerImpl doStart
[datastore] INFO: Dev App Server is now running

I also set the following environment variables:

set DATASTORE_DATASET=susanproject
set DATASTORE_HOST=http://localhost:8578
set DATASTORE_EMULATOR_HOST=localhost:8578
set DATASTORE_PROJECT_ID=susanproject

In my code then I have the following code:

var gcloud = require('gcloud')({
  projectId: 'susantestproject',
});

var datastore = gcloud.datastore({
  apiEndpoint: 'http://localhost:8578'
});

app.get('/', function (req, res) {

// Save data to Datastore.
var blogPostData = {
  title: 'How to make the perfect homemade pasta',
  author: 'Andrew Chilton',
  isDraft: true
};

var blogPostKey = datastore.key('BlogPost');

datastore.save({
  key: blogPostKey,
  data: blogPostData
}, function(err) {
  // `blogPostKey` has been updated with an ID so you can do more operations
  // with it, such as an update.
  blogPostData.isDraft = false;

  datastore.save({
    key: blogPostKey,
    data: blogPostData
  }, function(err) {
    if (!err) {
      // The blog post is now published!
          res.status(200).send('Hello, world! An insert has been made!');

    }

  });
});

When I run the code and hit localhost:3000 (so it hits the '/' directory) I get the following output:

App listening at http://:::3000
E0408 20:14:36.906000000  8752 tcp_client_windows.c:106] on_connect error: No connection could be made because the target machine actively refused it.
E0408 20:14:36.910000000  8752 parsing.c:472] Unknown frame type 50
E0408 20:14:37.909000000  6356 tcp_client_windows.c:106] on_connect error: No connection could be made because the target machine actively refused it.
E0408 20:14:37.913000000  6356 tcp_client_windows.c:106] on_connect error: No connection could be made because the target machine actively refused it.
E0408 20:14:37.917000000  6356 parsing.c:472] Unknown frame type 50

I can connect to the admin console at http://localhost:8578/_ah/admin, but nothing is inserted into the datastore. Overall I would like to connect to my local emulator, but am unable to.

UPDATE:

For the heck of it I created a virtual machine, installed Ubuntu, and tried to install the Google Datastore in linux opposed to Windows to see if it was a platform issue.

I also revereted back to a gcloud version prior to 0.30.0 and if I use that api everything works. If I try 0.30.0+ these errors occur.

Answer 1

I had the same problem.

On linux I solved it by running:

~/apps/google-cloud-sdk/platform/gcd/gcd.sh start --host=localhost --port=8463 --store_on_disk=True --consistency=0.32 --allow_remote_shutdown ~/.config/gcloud/emulators/datastore

You get the admin page at

http://localhost:8463/_ah/admin

Don't know if the same works on other platforms too..

Answer 2

I found this mentioned in the release notes for v0.30.0:

Additionally, we are now using gRPC to give a performance boost to your Datastore calls. As of now, the gcloud SDK does not have a gRPC emulator, meaning if you wish to use a localhost backend instead of the upstream API, you will have to use the gcd.sh tool directly.

https://github.com/GoogleCloudPlatform/gcloud-node/releases/tag/v0.30.0

Have you tried using gcd.sh?

Brief guide:

• Download the archive from the link they give in the release notes
• Unpack the zip
• Run ./gcs.sh create mydatadir
• Run ./gcs.sh start mydatadir

In another shell run:

export DATASTORE_EMULATOR_HOST=localhost:8080
export DATASTORE_HOST=http://localhost:8080

Now run your app.

There's a gcs.bat that you can run if you are using windows.

I think this gRPC datastore is a work in progress. It presumably doesn't have the bells and whistles of the gcloud one, such as an admin console. I can't find any documentation for it.