CODEWITHSUNDEEP
sql-servercsvneo4jssissql-graph
Adam
Adam

Reputation: 2031

Export CSV data from SQL Server for import into Neo4j Graph Database

I need to bulk import some highly connected data from SQL Server into Neo4j for analysis.

There are 2 links in the Neo4j developer guide which discusss this:

My first attempt to import data from a CSV file generated by SQL Server Import and Export Wizard failed to load the columns from my tables into Node attributes.

Does anyone know of a guide to configuring an SSIS package to produce a CSV export compliant with Neo4j's requirements listed at the link above?

Upvotes: 1

Views: 625

Answers (1)

Mark Wojciechowicz
Mark Wojciechowicz

Reputation: 4477

I don't have a guide to point you to, but I can point you in the right direction to solve this problem yourself:

  • Do not use the import export wizard -- this is very limiting in your ability to configure the flat file destination and it will not allow you to fix data in the pipeline

  • Sounds like they want a header row. Which means you need to make sure that the first row in your data is that. This can be done with a UNION, if your data is from SQL for example. i.e. 

    SELECT 'Column1' as Column1
     , 'Column2' as Column2 
UNION 
SELECT <actual data from my table>

  • Configure the flat file connection manager to have a header row by checking "Column names in the first data row"

  • You can use a derived column or script task to put quotes around special characters or quotes in text:

     - "Special character in non-quoted text ← make sure unusual text is always quoted"
 - "stray quotes – standalone double or single quote in the middle 
of non-quoted text, or non-escaped quotes in quoted text ← escape 
or remove stray quotes"

Per the suggestions in the link, you could use CSVkit or Papa Parse to examine issues in your final CSV file before trying to upload it to neo4j. Often these data quality problems are specific to your domain and once the handful of issues are resolved, it will not require further care and feeding.

Upvotes: 2

Related Questions