Elasticsearch start error:Native controller process has stopped - no new native processes can be started

Question

When I install elasticsearch on my virtual machine, elasticsearch does not start properly.

I tried to do these: vi /etc/security/limits.conf

• soft nofile 65536
• hard nofile 131072
• soft nproc 2048
• hard nproc 4096

vi /etc/security/limits.d/90-nproc.conf

• soft nproc 4096

vi /etc/sysctl.conf vm.max_map_count=655360

[es@test elasticsearch-7.3.1]$ ./bin/elasticsearch
Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[2019-09-10T22:52:12,836][INFO ][o.e.e.NodeEnvironment    ] [test] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [35.4gb], net total_space [39.9gb], types [rootfs]
[2019-09-10T22:52:12,850][INFO ][o.e.e.NodeEnvironment    ] [test] heap size [989.8mb], compressed ordinary object pointers [true]
[2019-09-10T22:52:12,856][INFO ][o.e.n.Node               ] [test] node name [test], node ID [G5dWJVSFTOWAVpNseEPcsQ], cluster name [elasticsearch]
[2019-09-10T22:52:12,857][INFO ][o.e.n.Node               ] [test] version[7.3.1], pid[19178], build[default/tar/4749ba6/2019-08-19T20:19:25.651794Z], OS[Linux/3.10.0-957.27.2.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/11.0.4/11.0.4+10-LTS]
[2019-09-10T22:52:12,859][INFO ][o.e.n.Node               ] [test] JVM home [/opt/jdk-11.0.4]
[2019-09-10T22:52:12,859][INFO ][o.e.n.Node               ] [test] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch-3827173686900523231, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -Dio.netty.allocator.type=unpooled, -XX:MaxDirectMemorySize=536870912, -Des.path.home=/usr/local/elasticsearch/elasticsearch-7.3.1, -Des.path.conf=/usr/local/elasticsearch/elasticsearch-7.3.1/config, -Des.distribution.flavor=default, -Des.distribution.type=tar, -Des.bundled_jdk=true]
[2019-09-10T22:52:23,967][INFO ][o.e.p.PluginsService     ] [test] loaded module [aggs-matrix-stats]
[2019-09-10T22:52:23,977][INFO ][o.e.p.PluginsService     ] [test] loaded module [analysis-common]
[2019-09-10T22:52:23,978][INFO ][o.e.p.PluginsService     ] [test] loaded module [data-frame]
[2019-09-10T22:52:23,979][INFO ][o.e.p.PluginsService     ] [test] loaded module [flattened]
[2019-09-10T22:52:23,979][INFO ][o.e.p.PluginsService     ] [test] loaded module [ingest-common]
[2019-09-10T22:52:23,980][INFO ][o.e.p.PluginsService     ] [test] loaded module [ingest-geoip]
[2019-09-10T22:52:23,981][INFO ][o.e.p.PluginsService     ] [test] loaded module [ingest-user-agent]
[2019-09-10T22:52:23,981][INFO ][o.e.p.PluginsService     ] [test] loaded module [lang-expression]
[2019-09-10T22:52:23,982][INFO ][o.e.p.PluginsService     ] [test] loaded module [lang-mustache]
[2019-09-10T22:52:23,983][INFO ][o.e.p.PluginsService     ] [test] loaded module [lang-painless]
[2019-09-10T22:52:23,984][INFO ][o.e.p.PluginsService     ] [test] loaded module [mapper-extras]
[2019-09-10T22:52:23,984][INFO ][o.e.p.PluginsService     ] [test] loaded module [parent-join]
[2019-09-10T22:52:23,985][INFO ][o.e.p.PluginsService     ] [test] loaded module [percolator]
[2019-09-10T22:52:23,986][INFO ][o.e.p.PluginsService     ] [test] loaded module [rank-eval]
[2019-09-10T22:52:23,986][INFO ][o.e.p.PluginsService     ] [test] loaded module [reindex]
[2019-09-10T22:52:23,987][INFO ][o.e.p.PluginsService     ] [test] loaded module [repository-url]
[2019-09-10T22:52:23,987][INFO ][o.e.p.PluginsService     ] [test] loaded module [transport-netty4]
[2019-09-10T22:52:23,988][INFO ][o.e.p.PluginsService     ] [test] loaded module [vectors]
[2019-09-10T22:52:23,988][INFO ][o.e.p.PluginsService     ] [test] loaded module [x-pack-ccr]
[2019-09-10T22:52:23,989][INFO ][o.e.p.PluginsService     ] [test] loaded module [x-pack-core]
[2019-09-10T22:52:23,989][INFO ][o.e.p.PluginsService     ] [test] loaded module [x-pack-deprecation]
[2019-09-10T22:52:23,990][INFO ][o.e.p.PluginsService     ] [test] loaded module [x-pack-graph]
[2019-09-10T22:52:23,990][INFO ][o.e.p.PluginsService     ] [test] loaded module [x-pack-ilm]
[2019-09-10T22:52:23,991][INFO ][o.e.p.PluginsService     ] [test] loaded module [x-pack-logstash]
[2019-09-10T22:52:23,991][INFO ][o.e.p.PluginsService     ] [test] loaded module [x-pack-ml]
[2019-09-10T22:52:23,992][INFO ][o.e.p.PluginsService     ] [test] loaded module [x-pack-monitoring]
[2019-09-10T22:52:23,992][INFO ][o.e.p.PluginsService     ] [test] loaded module [x-pack-rollup]
[2019-09-10T22:52:23,993][INFO ][o.e.p.PluginsService     ] [test] loaded module [x-pack-security]
[2019-09-10T22:52:23,993][INFO ][o.e.p.PluginsService     ] [test] loaded module [x-pack-sql]
[2019-09-10T22:52:23,994][INFO ][o.e.p.PluginsService     ] [test] loaded module [x-pack-voting-only-node]
[2019-09-10T22:52:23,994][INFO ][o.e.p.PluginsService     ] [test] loaded module [x-pack-watcher]
[2019-09-10T22:52:23,995][INFO ][o.e.p.PluginsService     ] [test] no plugins loaded
[2019-09-10T22:52:48,793][INFO ][o.e.x.s.a.s.FileRolesStore] [test] parsed [0] roles from file [/usr/local/elasticsearch/elasticsearch-7.3.1/config/roles.yml]
[2019-09-10T22:52:51,610][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [test] [controller/19279] [Main.cc@110] controller (64 bit): Version 7.3.1 (Build 1d93901e09ef43) Copyright (c) 2019 Elasticsearch BV
[2019-09-10T22:52:53,219][DEBUG][o.e.a.ActionModule       ] [test] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security
[2019-09-10T22:52:55,389][INFO ][o.e.d.DiscoveryModule    ] [test] using discovery type [zen] and seed hosts providers [settings]
[2019-09-10T22:53:03,862][INFO ][o.e.n.Node               ] [test] initialized
[2019-09-10T22:53:03,886][INFO ][o.e.n.Node               ] [test] starting ...
[2019-09-10T22:53:05,370][INFO ][o.e.t.TransportService   ] [test] publish_address {10.1.2.91:9301}, bound_addresses {[::]:9301}
[2019-09-10T22:53:05,397][INFO ][o.e.b.BootstrapChecks    ] [test] bound or publishing to a non-loopback address, enforcing bootstrap checks
ERROR: [2] bootstrap checks failed
[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured
[2019-09-10T22:53:05,451][INFO ][o.e.n.Node               ] [test] stopping ...
[2019-09-10T22:53:05,531][INFO ][o.e.n.Node               ] [test] stopped
[2019-09-10T22:53:05,531][INFO ][o.e.n.Node               ] [test] closing ...
[2019-09-10T22:53:05,588][INFO ][o.e.n.Node               ] [test] closed
[2019-09-10T22:53:05,593][INFO ][o.e.x.m.p.NativeController] [test] Native controller process has stopped - no new native processes can be started

I want it to start normally, but it always shows this error: [2019-09-10T22:53:05,593][INFO ][o.e.x.m.p.NativeController] [test] Native controller process has stopped - no new native processes can be startedenter code here

Answer 1

Here is the problem : [max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

That work for me :

 sudo sysctl -w vm.max_map_count=280530

Answer 2

May be specifying these three things correctly may solve your error trace.

1) Change the network.host in elastic configuration (elasticsearch.yml), look for the line containing network.host and uncomment it and set it to 0.0.0.0.

2) Change the JVM options, if you are using a zip file locate jvm.options in the config/ folder.

3) Set discovery.seed_hosts: also if any error still persists.

https://www.elastic.co/guide/en/elasticsearch/reference/master/jvm-options.html

Answer 3

I think this is related to this error:

[2019-09-10T22:53:05,397][INFO ][o.e.b.BootstrapChecks    ] [test] bound or publishing to a non-loopback address, enforcing bootstrap checks
ERROR: [2] bootstrap checks failed
[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured

see https://www.elastic.co/guide/en/elasticsearch/reference/7.1/discovery-settings.html