why elasticsearch won't run on Ubuntu 14.04?

Question

I'm trying to determine if the elasticsearch instance is running, but it doesn't appear to be:

ubuntu@ubuntu:~$ sudo service elasticsearch status
 * elasticsearch is not running
ubuntu@ubuntu:~$ sudo service elasticsearch start
 * Starting Elasticsearch Server  [ OK ] 
ubuntu@ubuntu:~$ sudo service elasticsearch status
 * elasticsearch is not running
and

ubuntu@ubuntu:~$ sudo /etc/init.d/elasticsearch status
 * elasticsearch is not running
ubuntu@ubuntu:~$ sudo /etc/init.d/elasticsearch start
 * Starting Elasticsearch Server  [ OK ] 
ubuntu@ubuntu:~$ sudo /etc/init.d/elasticsearch status
 * elasticsearch is not running
ubuntu@ubuntu:/etc/elasticsearch# sudo service elasticsearch restart
 * Stopping Elasticsearch Server  [ OK ] 
 * Starting Elasticsearch Server  [ OK ] 
ubuntu@ubuntu:/etc/elasticsearch# sudo service elasticsearch status
 * elasticsearch is not running

and

ubuntu@ubuntu:~$ curl -XGET localhost:9200/_nodes/_all/process?pretty
curl: (7) Failed to connect to localhost port 9200: Connection refused

and

ubuntu@ubuntu:/etc/elasticsearch$ sudo netstat -nlp
tcp6       0      0 :::9300                 :::*                    LISTEN      4413/java       

UPD

My elasticsearch.log:

[2014-12-03 00:00:02,161][INFO ][cluster.metadata         ] [Zero] [logstash-2014.12.03] creating index, cause [auto(bulk api)], shards [5]/[1], mappings [_default_]
[2014-12-03 00:00:02,617][INFO ][cluster.metadata         ] [Zero] [logstash-2014.12.03] update_mapping [logs] (dynamic)
[2014-12-03 00:00:12,737][INFO ][cluster.metadata         ] [Zero] [logstash-2014.12.03] update_mapping [logs] (dynamic)
[2014-12-03 00:00:17,587][INFO ][cluster.metadata         ] [Zero] [logstash-2014.12.03] update_mapping [logs] (dynamic)
[2014-12-03 00:00:18,842][INFO ][cluster.metadata         ] [Zero] [logstash-2014.12.03] update_mapping [logs] (dynamic)
[2014-12-03 01:00:01,430][INFO ][cluster.metadata         ] [Zero] [logstash-2014.11.25] deleting index
[2014-12-03 09:46:57,461][INFO ][cluster.metadata         ] [Zero] [logstash-2014.12.03] update_mapping [logs] (dynamic)

Answer 1

If you are using Elasticsearch > 5.0

Min/max heap size requirements for Elasticsearch 5.0 are now both defaulted to 2gb

Check the ls /tmp/hs_err_pid*.log files, in the logs you'll see that JVM failed to start ES because of insufficient memory.

You can adjust the heap size settings in /etc/elasticsearch/jvm.options. Adjust the lines -Xms2g and -Xmx2g to -Xms1g and -Xmx1g respectively, if you're on a box with 2 GB of RAM. If you're going to use a box with 1 GB of RAM I would recommend using -Xms512m and -Xmx512m.

Reference

Answer 2

For me, this problem was caused by the Elasticsearch Data and/or Logs directory being at 100% disk usage. Run df -h to see whether the directory your Elasticsearch process is using for data and logs has free space or not.

Answer 3

This command resolved my issue:

sudo chown -R elasticsearch:elasticsearch /var/lib/elasticsearch/

Referenced from @imsaar github

Answer 4

I got to the same point after I've done apt-get dist-upgrade - JAVA got updated to "Java(TM) SE Runtime Environment (build 1.8.0_91-b14) Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode)" version.

My ES did not want to start. I found the cause here (tail -n100 /var/log/elasticsearch/elasticsearch.log):

java.lang.IllegalArgumentException: Plugin [license] is incompatible with Elasticsearch [2.3.2]. Was designed for version [2.3.1]

I just removed the plugin (bin/plugin remove license) and started ES successfully.

I hope this will help others.

Answer 5

While the accepted answer command worked for me using Elasticsearch 1.7.3, with some changes in Elasticsearch >2.0 running the accepted answer command would produce

es.default.config is no longer supported. elasticsearch.yml 
must be placed in the config directory and cannot be renamed

Github Issue

The command as specified in the above Github issue would now be:

sudo -u elasticsearch /usr/share/elasticsearch/bin/elasticsearch -d -p /var/run/elasticsearch.pid --path.conf=/etc/elasticsearch --default.path.home=/usr/share/elasticsearch --default.path.logs=/var/log/elasticsearch --default.path.data=/var/lib/elasticsearch --default.path.work=/tmp/elasticsearch --default.path.conf=/etc/elasticsearch

Answer 6

The command line parameters that @aleung refers to can be set in the configuration file. By default, the parameters are commented out.

Set the following in /etc/default/elasticsearch

################################
# Elasticsearch
################################

# Elasticsearch home directory
ES_HOME=/usr/share/elasticsearch

# Elasticsearch configuration directory
CONF_DIR=/etc/elasticsearch

# Elasticsearch data directory
DATA_DIR=/var/lib/elasticsearch

# Elasticsearch logs directory
LOG_DIR=/var/log/elasticsearch

# Elasticsearch PID directory
PID_DIR=/var/run/elasticsearch

Answer 7

I ran into the same issue this morning. After much digging, we found out it was caused by an unsuccessful Java 8 installation. All was fine after Java 8 installation had been fixed.

Answer 8

The elasticsearch user cannot write the PID file because it has no permissions to create a file in /var/run/:

FileNotFoundException[/var/run/elasticsearch.pid (Keine Berechtigung)]

The fix: create the directory /var/run/elasticsearch/, change its ownership to elasticsearch:elasticsearch, and change the PID file location to this directory in the init script.

mkdir -p /var/run/elasticsearch
chown elasticsearch: /var/run/elasticsearch
sed -e 's|^PID_FILE=.*$|PID_FILE=/var/run/$NAME/$NAME.pid|g' -i /etc/init.d/elasticsearch

Once you get that far, here's the next error you might see:

ElasticsearchIllegalStateException[Failed to obtain node lock, is the following location writable?: [/var/lib/elasticsearch/elasticsearch]]

Again a resource does not have the correct permissions for the elasticsearch user.

chown -R elasticsearch: /var/lib/elasticsearch

Not done yet. Now you have to edit /etc/init.d/elasticsearch and remove this line:

test "$START_DAEMON" == true || exit 0

This line is utter garbage and is guaranteed to cause an exit.

Now it should finally start.

Answer 9

Elasticsearch service init script doesn't print any error information on console or log file when it fails to startup, instead it ridiculously shows [OK].

You have to run elaticsearch manually with the same user and same parameters as what the init script does to check what's going wrong. The error message will be printed on console.

On my Ubuntu 14.10 with elasticsearch-1.4.1.deb installed, without any path changed, the command to run elastisearch is:

sudo -u elasticsearch /usr/share/elasticsearch/bin/elasticsearch -d -p /var/run/elasticsearch.pid --default.config=/etc/elasticsearch/elasticsearch.yml --default.path.home=/usr/share/elasticsearch --default.path.logs=/var/log/elasticsearch --default.path.data=/var/lib/elasticsearch --default.path.work=/tmp/elasticsearch --default.path.conf=/etc/elasticsearch

I just added a line into /etc/init.d/elasticsearch to print out the above command:

# Start Daemon
log_daemon_msg "sudo -u $ES_USER $DAEMON $DAEMON_OPTS"    # <-- Add this line
start-stop-daemon --start -b --user "$ES_USER" -c "$ES_USER" --pidfile "$PID_FILE" --exec $DAEMON -- $DAEMON_OPTS
log_end_msg $?