Mongo's stats not showing on munin

Question

I'm trying to add a MongoDB's plugin to munin. I followed this to install the plugin:

sudo apt-get install git munin-node
git clone git://github.com/erh/mongo-munin.git /home/ubuntu/mongo-munin
sudo ln -sf /home/ubuntu/mongo-munin/mongo_btree /etc/munin/plugins/mongo_btree
sudo ln -sf /home/ubuntu/mongo-munin/mongo_conn /etc/munin/plugins/mongo_conn
sudo ln -sf /home/ubuntu/mongo-munin/mongo_lock /etc/munin/plugins/mongo_lock
sudo ln -sf /home/ubuntu/mongo-munin/mongo_mem /etc/munin/plugins/mongo_mem
sudo ln -sf /home/ubuntu/mongo-munin/mongo_ops /etc/munin/plugins/mongo_ops
sudo service munin-node restart

(from https://ivan-site.com/2013/06/monitoring-mongodb-in-munin-on-ubuntu-1304)

But nothing was sent to munin's server (for Mongo only, all other date from the server are ok on Munin).

Then I found there is an error when runing the plugin:

# sudo -u munin python /usr/share/munin/plugins/mongo_ops
Traceback (most recent call last):
  File "/usr/share/munin/plugins/mongo_ops", line 56, in <module>
    doData()
  File "/usr/share/munin/plugins/mongo_ops", line 33, in doData
    ss = getServerStatus()
  File "/usr/share/munin/plugins/mongo_ops", line 28, in getServerStatus
    raw = urllib2.urlopen(req).read()
  File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 400, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 418, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 1207, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/lib/python2.7/urllib2.py", line 1177, in do_open
    raise URLError(err)
urllib2.URLError: <urlopen error [Errno 111] Connection refused>

I've checked this question: Munin Mongodb Plugin Not Showing. . .? but the problem is not the same, I'm running pyhton 2, as seen in the output above:

File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen

And so, plugins are active:

# munin-node-configure | grep "mongo"
mongo_btree                | yes   |
mongo_conn                 | yes   |
mongo_lock                 | yes   |
mongo_mem                  | yes   |
mongo_ops                  | yes   |

Is it a permission from Mongo? Maybe I need to setup a user for Munin? Do you have any idea how to fix that?

Answer 1

This seems broken or not working when using credentials:

$ munin-run -d mongo_docs 
# Processing plugin configuration from /etc/munin/plugin-conf.d/munin-node
# Setting /rgid/ruid/ to /126/65534/
# Setting /egid/euid/ to /126 126/65534/
# Setting up environment
# Environment MONGO_DB_URI = mongodb://MY_USER:MY_PASSWORD@localhost:27017/admin
# About to run '/etc/munin/plugins/mongo_docs'
Traceback (most recent call last):
  File "/etc/munin/plugins/mongo_docs", line 53, in <module>
    doData()
  File "/etc/munin/plugins/mongo_docs", line 20, in doData
    ss = getServerStatus()
  File "/etc/munin/plugins/mongo_docs", line 14, in getServerStatus
    return c.admin.command('serverStatus', workingSet=True)
  File "/usr/local/lib/python2.7/dist-packages/pymongo/database.py", line 733, in command
    codec_options, session=session, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pymongo/database.py", line 630, in _command
    client=self.__client)
  File "/usr/local/lib/python2.7/dist-packages/pymongo/pool.py", line 613, in command
    user_fields=user_fields)
  File "/usr/local/lib/python2.7/dist-packages/pymongo/network.py", line 167, in command
    parse_write_concern_error=parse_write_concern_error)
  File "/usr/local/lib/python2.7/dist-packages/pymongo/helpers.py", line 159, in _check_command_response
    raise OperationFailure(msg % errmsg, code, response)
pymongo.errors.OperationFailure: not authorized on admin to execute command { serverStatus: 1, workingSet: true, $db: "admin", $readPreference: { mode: "primaryPreferred" } }

But works fine when executing manually:

$ mongo -u MY_USER admin --eval 'db.serverStatus().metrics.document'
MongoDB shell version v3.6.16
Enter password: 
connecting to: mongodb://127.0.0.1:27017/admin?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("b2d5afa5-8bc9-4ed7-9a6e-a52e8d660876") }
MongoDB server version: 3.6.15
{
    "deleted" : NumberLong(1137),
    "inserted" : NumberLong(342220),
    "returned" : NumberLong(3429651),
    "updated" : NumberLong(1364343)
}
$

Probably pymongo bug?

Another two that are failing due to missing keys:

$ munin-run mongo_btree
Traceback (most recent call last):
  File "/etc/munin/plugins/mongo_btree", line 59, in <module>
    doData()
  File "/etc/munin/plugins/mongo_btree", line 22, in doData
    for k,v in get().iteritems():
  File "/etc/munin/plugins/mongo_btree", line 19, in get
    return getServerStatus()["indexCounters"]
KeyError: 'indexCounters'

$  munin-run mongo_lock 
Traceback (most recent call last):
  File "/etc/munin/plugins/mongo_lock", line 52, in <module>
    doData()
  File "/etc/munin/plugins/mongo_lock", line 21, in doData
    print name + ".value " + str( 100 * (getServerStatus()["globalLock"]["lockTime"]/getServerStatus()["globalLock"]["totalTime"]) )
KeyError: 'lockTime'

Answer 2

OK, so first off - those plugins were written a LONG time ago and have not been updated in several years, so they may or may not work at this point. Second, they use the HTTP console of MongoDB to gather data. Since that now defaults to disabled it may not be running at all for your instance (and is generally not recommended for production systems - if you do run it, you should firewall it off).

EDIT: After a bit more checking, the plugins require more than the console, they require the REST interface to be enabled, and that is definitely off by default, and again is not generally recommended for production.

Finally, the plugins are hard coded to connect to port 28017 (default port for MongoDB +1000), so if you have customized the port MongoDB is running on, you will need to manually adjust each plugin to connect to the right port.

The easiest way to check all this is to point your browser at the host on port 28017 (http://hostame/28017) or whatever the appropriate port is and see if you can access the console.

Update

I forked Eliot's plugins to see how hard it would be to port to pymongo, removing the dependency on the REST API, and I managed to do it fairly easily (no auth support yet). You can find that version here, same installation and test commands per the page you linked worked for me:

https://github.com/comerford/mongo-munin

As it turns out, the changes to the serverStatus() command since the plugins were written means that several of them no longer work as-is with a current version of MongoDB, so I had to go back and fix them up for 2.4 and later (this along with the new pymongo dependency is listed in the README).