CODEWITHSUNDEEP
pythoncassandramesospheredcos
Hugo Matinho
Hugo Matinho

Reputation: 21

dcos cassandra subcommand error

Can't seem to install the Cassandra package, marathon get's stuck in deployment in phase 1/2 and dcos cassandra subcommand issues the following stacktrace, any help appreciated.

Traceback (most recent call last):
  File "/home/azureuser/.dcos/subcommands/cassandra/env/bin/dcos-cassandra", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/opt/mesosphere/lib/python3.4/site-packages/pkg_resources.py", line 2701, in <module>
    parse_requirements(__requires__), Environment()
  File "/opt/mesosphere/lib/python3.4/site-packages/pkg_resources.py", line 572, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: requests


Python version: Python 3.4.2
requests version : 1.8.1

Upvotes: 1

Views: 528

Answers (2)

Hugo Matinho
Hugo Matinho

Reputation: 21

Unfortunately we're still having the same problem, though we've managed to get a workaround. It seems there are more than one distinct issues with DC/OS on Azure, anyway I'll provide further feedback. If using the Marketplace version of DC/OS 1.7.0, Cassandra doesn't deploy, it get's stuck in Marathon on phase 1/2, upon inspection of the logs it seems to have a problem with accessing the default ports.

Pastebin to log file

On the other hand that problem doesn't appear on ACS DC/OS, Cassandra deploys correctly appearing in the DC/OS Service tab as well as on Marathon. The DCOS Cassandra CLI doesn't work on any. Upon a not very thorough inspection, it seems that when we installed DCOS CLI using the method above there are some issues with the dependencies specially taking into account the $PYTHONPATH variable

/opt/mesosphere/lib/python3.4/site-packages

We were able to solve the dependencies issue by taking two actions:

  • First Dependency issue was with requests module, which was solved with the following actions after installing cli for the Cassandra subcommand.

    cd ~/.dcos/subcommands/cassandra
source env/bin/activate
pip install -Iv requests

We used -Iv since the usual update procedure fails with external dependency in $PYTHONPATH path, so requests dependency solved.

  • Second dependency which the cassandra subcommand was requiring was docopt, again by using the same method we were able to solve the issue and now the subcommand works as per the documentation

    pip install -Iv docopt

This does seem a bit hackish, wondering if there's anything more appropriate to be done.

output of dcos cassandra connection after taking above steps

{
"address": [
    "10.32.0.9:9042",
    "10.32.0.6:9042",
    "10.32.0.8:9042"
],
"dns": [
    "node-0.cassandra.mesos:9042",
    "node-1.cassandra.mesos:9042",
    "node-2.cassandra.mesos:9042"
]
}

The same happens for other DC/OS subcommands like for example the Kafka one.

Upvotes: 1

nickbp
nickbp

Reputation: 66

I'm on the team that's building the Cassandra service. Thanks for trying it out!

We've just updated the Cassandra CLI package to better define its pip dependencies. In your case it looks like it was trying to reuse an old version of the requests library? To kick your CLI's Cassandra module to the latest version, try running dcos package uninstall --cli cassandra; dcos package install --cli cassandra. Note that the --cli is important; omitting it can result in uninstalling the Cassandra service itself, while all we want is to reinstall the local CLI module.

Keep in mind that you should also be able to access the Cassandra service directly over HTTP. The CLI module is effectively a thin interface around the service's HTTP API. For example, curl -H "Authorization:token=$(dcos config show core.dcos_acs_token)" http://<your-dcos-host>/service/cassandra/v1/plan | jq '.'. See the curl examples in the Cassandra 1.7 docs for other endpoints.

Once you've gotten the CLI up and running, that should give more insight into the state of the service, but logs may give more thorough information, particularly if the service is failing to start. You can access the service logs directly by visiting the dashboard at http://<your-dcos-host>/:

  1. Click Services on the left, then select marathon from the list. The Cassandra service manager is run as a Marathon task.
  2. A panel will come up showing a list of all tasks being managed by Marathon. Click cassandra on this list to show its working directory, including the available log files.
  3. When hovering over files, a magnifying glass will appear. Click a magnifying glass to display the corresponding file in-line.

Upvotes: 2

Related Questions