CODEWITHSUNDEEP
cassandradatastaxdatastax-enterprise
user374374
user374374

Reputation: 343

Connecting opscenter but no data

I have deployed a datastax cassandra cluster on google cloud. The cluster size is 4 nodes on two different datacenters.

Servers are running fine and able to view opscenter but no data on it and getting the following errors.

This is a fresh deployment and all nodes are up and running. No tables were created yet.

Error Message from opscenter log: 

[opscenterd] ERROR: Unhandled error in Deferred: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ONE (1 required but only 0 alive)

address.yaml from one of the node:

stomp_interface: 10.128.x.x [ ops center ip ]
agent_rpc_interface: 10.132.x.x [ one of the node ip]
agent_rpc_broadcast_address: 10.132.x.x

**cassandra.yaml from one of the node:**

cluster_name: 'Test Cluster'

num_tokens: 64



hinted_handoff_enabled: true
max_hint_window_in_ms: 10800000 # 3 hours

hinted_handoff_throttle_in_kb: 1024

max_hints_delivery_threads: 2

hints_directory: /var/lib/cassandra/hints

hints_flush_period_in_ms: 10000

max_hints_file_size_in_mb: 128


batchlog_replay_throttle_in_kb: 1024

authenticator: AllowAllAuthenticator

authorizer: AllowAllAuthorizer

role_manager: com.datastax.bdp.cassandra.auth.DseRoleManager

roles_validity_in_ms: 2000


permissions_validity_in_ms: 2000


partitioner: org.apache.cassandra.dht.Murmur3Partitioner

data_file_directories:
     - /mnt/data

commitlog_directory: /mnt/commitlog

disk_failure_policy: stop

commit_failure_policy: stop

key_cache_size_in_mb:

key_cache_save_period: 14400

row_cache_size_in_mb: 0

row_cache_save_period: 0


counter_cache_size_in_mb:

counter_cache_save_period: 7200


saved_caches_directory: /mnt/saved_caches

commitlog_sync: periodic
commitlog_sync_period_in_ms: 10000

commitlog_segment_size_in_mb: 32


seed_provider:
    # Addresses of hosts that are deemed contact points. 
    # Cassandra nodes use this list of hosts to find each other and learn
    # the topology of the ring.  You must change this if you are running
    # multiple nodes!
    - class_name: org.apache.cassandra.locator.SimpleSeedProvider
      parameters:
          # seeds is actually a comma-delimited list of addresses.
          # Ex: "<ip1>,<ip2>,<ip3>"
          - **seeds: "XX.XXX.X.X" [ The node IP from different Data Center]**

concurrent_reads: 32
concurrent_writes: 32
concurrent_counter_writes: 32

concurrent_materialized_view_writes: 32

memtable_allocation_type: heap_buffers



index_summary_capacity_in_mb:

index_summary_resize_interval_in_minutes: 60

trickle_fsync: true
trickle_fsync_interval_in_kb: 10240

storage_port: 7000

ssl_storage_port: 7001

listen_address: **SAME NODE IP**

broadcast_address: **SAME NODE IP**



start_native_transport: true
native_transport_port: 9042



start_rpc: true

rpc_address: 0.0.0.0

rpc_port: 9160

broadcast_rpc_address: SAME NODE IP

rpc_keepalive: true

rpc_server_type: sync




thrift_framed_transport_size_in_mb: 15

incremental_backups: false

snapshot_before_compaction: false

auto_snapshot: true

tombstone_warn_threshold: 1000
tombstone_failure_threshold: 100000

Opscenter.conf from opscenter: 

[webserver]

port = 8888

interface = 0.0.0.0

[authentication]

enabled = False

[stat_reporter]

nodetool status:

Datacenter: europe-west1-b

Status=Up/Down

|/ State=Normal/Leaving/Joining/Moving

-- Address Load Tokens Owns Host ID Rack

UN 10.XXX.0.2 220 KB 64 ? 5381a371-c532-4be7-a7f5-80162bef4541 europe-west1-b

UN 10.XXX.0.3 148.6 KB 64 ? e5b3dfec-f9bf-4952-b48f-b5a82749a9fe europe-west1-b

Datacenter: us-east1-b

Status=Up/Down

|/ State=Normal/Leaving/Joining/Moving

-- Address Load Tokens Owns Host ID Rack

UN 10.XXX.0.2 158.72 KB 64 ? b93980e0-3c28-4bb0-9e49-220cb009d44c us-east1-b

UN 10.XXX.0.3 192.85 KB 64 ? af242b71-b7b4-4345-9888-5b89eb5e6199 us-east1-b

Note: Non-system keyspaces don't have the same replication settings, effective ownership information is meaningless

Upvotes: 0

Views: 524

Answers (1)

Chris Lohfink
Chris Lohfink

Reputation: 16400

You have a DC in your replication factor ($dc) that doesnt exist, so when OpsCenter sends a request to read data the coordinator responds with an UnavailableException.

Try updating your keyspace like ALTER KEYSPACE "OpsCenter" WITH replication = {'class': 'NetworkTopologyStrategy', 'us-east1-b': '2'} and doing a full repair if you want to keep the data thats there.

If it hasnt run yet so you dont have data you want to save, just delete the opscenter keyspace (drop keyspace "OpsCenter") and restart opscenterd and the agents (after opscenter service has started back up).

Upvotes: 1

Related Questions