DSE graph not able to create search index asText for a property

Question

I've just started my journey with DSE graph (had fair bit of understanding of Titan earlier). I've set-up DSE graph with Datastax 5.0.3.

When trying to create a search index for a property, I am getting following exception. schema.vertexLabel('Employee').index('search').search().by('story').asText().add()

org.apache.tinkerpop.gremlin.driver.exception.ResponseException: Cannot create search index with workload: Analytics

I was able to create properties, materialized and secondary indices. But when I tried to create the search index, I am facing this issue.

I realized that while bringing up my single node cluster, I've had to turn off -s flag because that is not letting me to bring up the DSE server. There was some exception when bringing up the node for the first time I was not supposed to set the -s flag as per some Datastax developer QAs.

entrypoint: ["/usr/local/bin/dse-entrypoint", "-k", "-g"]

Now when I tried to enable the -s flag, my node is not coming up and I am getting the following exception.

dse          | WARN  12:54:28,038  CLibrary.java:163 - Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.
dse          | WARN  12:54:28,038  StartupChecks.java:118 - jemalloc shared library could not be preloaded to speed up memory allocations
dse          | WARN  12:54:28,039  StartupChecks.java:150 - JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.
dse          | WARN  12:54:28,047  SigarLibrary.java:174 - Cassandra server running in degraded mode. Is swap disabled? : false,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : true
dse          | ERROR 12:54:28,710  CassandraDaemon.java:709 - Cannot start node if snitch's data center (SearchGraphAnalytics) differs from previous data center (GraphAnalytics). Please fix the snitch configuration, decommission and rebootstrap this node or use the flag -Dcassandra.ignore_dc=true.
dse          | INFO  12:54:28,717  DseDaemon.java:556 - DSE shutting down...
dse          | INFO  12:54:28,718  PluginManager.java:104 - All plugins are stopped.
dse          | Oct 27, 2016 12:54:28 PM org.apache.coyote.http11.Http11Protocol pause
dse          | INFO: Pausing Coyote HTTP/1.1 on http-172.19.0.3-8983
dse          | Oct 27, 2016 12:54:29 PM org.apache.catalina.core.StandardService stop
dse          | INFO: Stopping service Solr
dse          | INFO  12:54:29,907  SolrHttpAuditLogFilter.java:225 - Shutting down Solr audit logging filter
dse          | INFO  12:54:29,924  RepeatablePOSTQueryFilter.java:81 - Shutting down com.datastax.bdp.search.solr.RepeatablePOSTQueryFilter filter
dse          | Oct 27, 2016 12:54:29 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
dse          | SEVERE: The web application [/solr] appears to have started a thread named [Thread-3] but has failed to stop it. This is very likely to create a memory leak.
dse          | Oct 27, 2016 12:54:29 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
dse          | SEVERE: The web application [/solr] appears to have started a thread named [NonPeriodicTasks:1] but has failed to stop it. This is very likely to create a memory leak.
dse          | Oct 27, 2016 12:54:29 PM org.apache.coyote.http11.Http11Protocol destroy
dse          | INFO: Stopping Coyote HTTP/1.1 on http-172.19.0.3-8983
dse          | INFO  12:54:33,191  MessageServer.java:129 - internode-messaging message server finished shutting down.
dse          | INFO  12:54:37,209  MessageServer.java:129 - internode-messaging message server finished shutting down.
dse          | Exception in thread "Daemon shutdown" java.lang.AssertionError
dse          |  at org.apache.cassandra.gms.Gossiper.addLocalApplicationStateInternal(Gossiper.java:1427)
dse          |  at org.apache.cassandra.gms.Gossiper.addLocalApplicationStates(Gossiper.java:1451)
dse          |  at org.apache.cassandra.gms.Gossiper.addLocalApplicationState(Gossiper.java:1441)
dse          |  at com.datastax.bdp.gms.DseState.setActiveStatusSync(DseState.java:241)
dse          |  at com.datastax.bdp.server.DseDaemon.preStop(DseDaemon.java:576)
dse          |  at com.datastax.bdp.server.DseDaemon.safeStop(DseDaemon.java:587)
dse          |  at com.datastax.bdp.server.DseDaemon.lambda$getShutdownHook$226(DseDaemon.java:905)
dse          |  at java.lang.Thread.run(Thread.java:745)

Please suggest how I can rectify this situation and be able to add the searchIndex to my properties.

Answer 1

This error is stating that you are starting up the cluster with a different name than what the cluster expects based on a previous configuration/start.

By default, unless you override the cluster name in the cassandra.yaml file, your cluster will startup with a name based on the workloads you enable, i.e. -s, -t. In your case, since you started the cluster as an Analytics cluster and then restarted it as a SearchAnalytics cluster, the cluster name is defaulting to the new cluster name, which doesn't match the old cluster name.

The easiest thing to do here is to wipe your cassandra commit log, caches, and data directory and restart the node. That will wipe out the old cluster name from your system tables and will allow the cluster to start. Doing this will wipe out any data you had in the cluster.