CODEWITHSUNDEEP
sphinxubuntu-10.04
PatrikAkerstrand
PatrikAkerstrand

Reputation: 45731

Issues with running two instances of searchd

I have just updated our Sphinx server from 1.10-beta to 2.0.6-release, and now I have run into some issues with searchd. Previously we were able to run two instances of searchd next to each other by specifying two different config-files, i.e:

searchd --config /etc/sphinx/sphinx.conf
searchd --config /etc/sphinx/sphinx.staging.conf

sphinx.conf listens to 9306:mysql41, and 9312, while sphinx.staging.conf listens to 9307:mysql41 and 9313.

After we updated to 2.0.6 however, a second instance is never started. Or rather.. the output makes it seem like it starts, and a pid-file is created etc. But for some reason only the first searchd instance keeps running, and the second seems to shutdown right away. So while trying to run searchd --config /etc/sphinx/sphinx.conf twice (if that was the first one started) complains that the pid-file is in use, trying to run searchd --config /etc/sphinx/sphinx.staging.conf (if that is the second started instance) "starts" the daemon again and again, only no new process is created.. Note that if I switch these commands around when first creating the process, then sphinx.conf is the instance not really started.

I have checked, and rechecked, that these ports are only used by searchd.

Does anyone have any idea of what I can do/try next? I've installed it from source on ubuntu 10.04 LTS with:

./configure --prefix /etc/sphinx --with-mysql --enable-id64 --with-libstemmer
make -j4 install

Upvotes: 3

Views: 4702

Answers (1)

PatrikAkerstrand
PatrikAkerstrand

Reputation: 45731

Note to self: Check the logs!

RT-indices use binary logs to enable crash recovery. Since my old config files did not specify a path for where these should be stored, both instances of searchd tried to write to the same binary logs. The instance started last was of course not permitted to manipulate these files, and thus exited with a fatal error:

[Fri Nov  2 17:13:32.262 2012] [ 5346] FATAL: failed to lock
    '/etc/sphinx/var/data/binlog.lock': 11 'Resource temporarily unavailable'
[Fri Nov  2 17:13:32.264 2012] [ 5345] Child process 5346 has been finished, 
    exit code 1. Watchdog finishes also. Good bye!

The solution was simple, ensure to specify a binlog_path inside the searchd configuration section of each configuration file:

searchd
{
[...]
   binlog_path = /path/to/writable/directory
[...]
}

Upvotes: 6

Related Questions