CODEWITHSUNDEEP
logstashibm-cloudrethinkdb
Thor_Bux
Thor_Bux

Reputation: 1157

Connect Logstash with RethinkDB on IBM BlueMix

I try and connect Logstash with RethinkDB on IBM BlueMix. I have provisioned the RethinkDB-Service from BlueMix (with is from Compose I believe) and also provisioned a Virtual Machine (CentOS). Then I installed Logstash (using the yum package) and the plugin from here (https://github.com/wayann/logstash-input-rethinker). So far so good. Now I struggle to get a connection from Logstash to RethinkDB. I have sent host and port and auth_key in a similar config file (https://github.com/wayann/logstash-input-rethinker/blob/master/rethinker.conf) and started Logstash with bin/logstash -f rethinker.conf However, Logstash is unable to connect to RethinkDB. Has anyone experience using Logstash with RethinkDB on Bluemix? I have sent host and port according to the credentials available on the BlueMix dashboard (same as the admin UI) but I'm unsure what to use for auth_key. admin-password didn't work. BlueMix runs RethinkDB version 2.3.5

Any hint highly apprechiated

Edit1, Error message:

05:46:12.049 [[main]<rethinker] ERROR logstash.pipeline - A plugin 
had an unrecoverable error. Will restart this plugin.
 Plugin: <LogStash::Inputs::Rethinker host=>"sl-eu-lon-2-
portal.3.dblayer.com", port=>15216, auth_key=>"********", 
watch_dbs=>["test", "MyDB"], watch_tables=>["MyTable"], 
backfill=>"true", id=>"f1c07e0332787a22600c0835d2aa6ad61ca9b22b-1", 
enable_metric=>true, codec=><LogStash::Codecs::JSONLines 
id=>"json_lines_e24c0744-3983-4001-9629-d93e266c5ffb", 
enable_metric=>true, charset=>"UTF-8", delimiter=>"\n">, 
squash=>true, user=>"admin">
 Error: Connection closed by server.

Edit2, config file:

input { 
  rethinker {
    host => 'sl-eu-lon-2-portal.3.dblayer.com'
    port => 15216
    auth_key => '*****'
    watch_dbs => ['test','MyDB']
    watch_tables => ['MyTable']
    backfill => true
  }
}

output { 
  stdout { 
    codec => json_lines 
  } 
}

Edit3:

Ok, it was my error. I need to use the 'username', 'password' and 'ca_cert' parameters and not 'auth_key'. Using this configuration connects to the rethinkdb instance:

input { 
  rethinker {
    host => 'sl-eu-lon-2-portal.3.dblayer.com'
    port => 15216
    watch_dbs => ['MyDB']
    watch_tables => ['MyTable']
    backfill => true
    user => 'admin'
    password => '******'
    ca_certs => '<<cert-string>>'
  }
}

output { 
  stdout { 
    codec => json_lines 
  } 
}

Now, however, I get the following error:

03:54:37.252 [[main]<rethinker] ERROR logstash.pipeline - A plugin 
had an unrecoverable error. Will restart this plugin.
  Plugin: <LogStash::Inputs::Rethinker host=>"sl-eu-lon-2-
portal.3.dblayer.com", port=>15216, watch_dbs=>["MyDB"], 
watch_tables=>["MyTable"], backfill=>"true", user=>"admin", 
password=>"*****", ca_certs=>"**cert-string**", 
id=>"46aa34ee0917060057d4e9a0c657ee327df730ee-1", 
enable_metric=>true, codec=><LogStash::Codecs::JSONLines 
id=>"json_lines_c43cdbc3-1646-4e96-aadc-834e727949a6", 
enable_metric=>true, charset=>"UTF-8", delimiter=>"\n">, 
squash=>true>
  Error: No message available

How would I resolved this Error: No message available error? Logstash version is 5.5.2

Upvotes: 2

Views: 106

Answers (1)

Thor_Bux
Thor_Bux

Reputation: 1157

I finally figured it out, thanks a lot for your help @whites11.

I needed to set the correct settings in the configuration file. That includes user, password and ca-cert. For ca-cert it is important that this is the path to the certificate file and not the certificate content as a string.

My setting looks like this now:

input { 
  rethinker {
    host => 'sl-eu-lon-2-portal.3.dblayer.com'
    port => 15216
    watch_dbs => ['MyDB']
    watch_tables => ['MyTable']
    backfill => true
    user => 'admin'
    password => '******'
    ca_certs => '<<path-to-cert-file>>'
  }
}

output { 
  stdout { 
    codec => json_lines 
  } 
}

Upvotes: 1

Related Questions