CODEWITHSUNDEEP
neo4j
shree11
shree11

Reputation: 535

Migrating mysql data to neo4j database

I wanted to migrate data from Mysql to neo4j. I'm using Neo4j 2.1.2 64 bit installer on 64 bit windows machine. I followed the blog in the link http://maxdemarzi.com/2012/02/28/batch-importer-part-2/#more-660 where migrating data from postgreSQL is well well explained.

Even I took the same example and created the sames tables in mysql. After creating nodes and relationship tables in mysql, i exported them as a csv file . So that I can use them in the batch import command. Here all my fields are varchar and row_number() fiels is also a varchar field.

I used the below command to export mysql's relationship table into myrels.csv file (same thing for nodes table):

SELECT  *
INTO OUTFILE 'D:/Tech_Explorations/BigData_Related/Neo4j/mqytoneo4j/myrels.csv'
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
ESCAPED BY '"'
LINES TERMINATED BY '\n'
FROM
(
SELECT 'start' AS `start`, 'end' AS `end`,'type' AS `type`,'status' AS `status`
UNION ALL

SELECT `start`, `end`,`type`,`status`
FROM `vouch_rels`
) `sub_query`;

Used below query to load the mynodes.csv and myrels.csv o neo4j:

java -server -Xms1024M -jar D:/Neo4j/target/batch-import-jar-with-dependencies.jar    
neo4j/data/graph.db mynodes.csv myrels.csv

When i executed the above batch import query , it's giving me an error saying

Exception in thread "main" java.lang.NumberFormatException: For input string: "1
,"1","python,confirmed"

Where "1,"1","python,confirmed" is row in the myrels.csv.

The above error might be because of some datatype or csv file issue but I'm not able to figure it out. Even I tried with changing different csv load options while loading from mysql to csv file. But still getting the same error.

Upvotes: 1

Views: 2887

Answers (2)

Pranjal Mittal
Pranjal Mittal

Reputation: 11422

MySQL to Neo4j migration is not a straightforward export-load problem. The property graph needs to be clear for Neo4j and should be consistent with the MySQL schema. There is no way to automatically generate Neo4j property graph from MySQL schema to my knowledge. After the 2 schemas are well defined you can write your own migrations in any programming language.

The python way to do the migration

py2neo is a Python library that makes it easy to write migrations as it provides a ton of useful functions, option to run cypher queries, transaction support, etc.

I used py2neo in a project to migrate around 100MB data from MySQL to Neo4j. Here is the sample code for reference along with documentation. The data is not provided but the schema of both MySQL and Neo4j property graph is given.

P.S: I might have digressed from trying to address your problem. But I have written this answer as it might help readers who are looking to solve the MySQL to Neo4j migration problem using Python.

Upvotes: 3

stephenmuss
stephenmuss

Reputation: 2445

I'd suggest looking at the LOAD CSV Cypher option. There are detailed docs on the Neo4j website.

Basically, you can use a Cypher query like the following to import your data.

USING PERIODIC COMMIT
LOAD CSV WITH HEADERS FROM "file:/path/to/your.csv" AS csvLine
MATCH (person:Person { id: toInt(csvLine.personId)}),(movie:Movie { id: toInt(csvLine.movieId)})
CREATE (person)-[:PLAYED { role: csvLine.role }]->(movie)

If you wish to proceed with the Java batch import tool then I believe your file needs to be tab delimited not comma delimited.

Upvotes: 0

Related Questions