CODEWITHSUNDEEP
mysqlelasticsearchjdbclogstashlogstash-jdbc
Kelvin Baumgart
Kelvin Baumgart

Reputation: 161

Can't import mysql data into elastic search

I'm trying to follow along with a course example haven't recieved any help in the FAQ's, tried everything I could find on google and here.

I'm not using docker just running this demo on my local machine(Ubunutu 18.04), both elastic search and mysql are running.

When I run "sudo bin/logstash -f /etc/logstash/conf.d/mysql.conf --path.settings /etc/logstash" I get the following Error: com.mysql.jdbc.Driver not loaded. Are you sure you've included the correct jdbc driver in :jdbc_driver_library? The driver does exist and path is correct.

when I use sudo bin/logstash --config.test_and_exit -f /etc/logstash/conf.d/mysql.conf It returns with configuration ok.

I'm using mysql-connector-java-5.1.47 openjdk version "1.8.0_181" OpenJDK Runtime Environment (build 1.8.0_181-8u181-b13-0ubuntu0.18.04.1-b13) OpenJDK 64-Bit Server VM (build 25.181-b13, mixed mode)

Elasticsearch-6.4.2 Logstash-6.4.2

My mysql.conf is

input {
    jdbc {
        jdbc_connection_string => "jdbc:mysql://localhost:3306/movielens"
        jdbc_user => "grunt"
        jdbc_password => "password"
        jdbc_driver_library => "/home/alarik/mysql-connecter-java-5.1.47/mysql-connector-java-5.1.47-bin.jar"
        jdbc_driver_class => "com.mysql.jdbc.Driver"
        statement => "SELECT * FROM movies"
    }
}

output {
    stdout { codec => json_lines }
    elasticsearch {
        "hosts" => "localhost:9200"
        "index" => "movielens-sql"
        "document_type" => "data"
    }
}

Upvotes: 0

Views: 213

Answers (1)

mkop
mkop

Reputation: 91

I solved the problem:

First check your java version:

root@xxxxxx:/# java -version
openjdk version "1.8.0_181"

If you are using 1.8 then you should use the JDBC42 version.

If you are using 1.7 then you should use the JDBC41 version.

If you are using 1.6 then you should use the JDBC43 version.

mysql setup:

mysql-connector-java-5.1.46.jar

jdbc_driver_library => "//path_to_jar/mysql-connector-java-5.1.46.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"

Upvotes: 0

Related Questions