CODEWITHSUNDEEP
csvbulk-loadcrate
user3769537
user3769537

Reputation: 13

Importing data from csv files into Cratedb

I have created a table in Crate 0.38.x with columns having integer, string and timestamp data types. I want to load data into this table from delimited text files. Is there a utility to do a bulk import? Sorry, but I could not find one in the documentation or on Github

Upvotes: 1

Views: 1541

Answers (3)

Ashutosh Kumar
Ashutosh Kumar

Reputation: 1

Read this: https://crate.io/docs/crate/reference/en/latest/general/dml.html#import-and-export

just make sure that you have created the table with schema beforehand using the copy function to import dataset from json or csv.

Upvotes: 0

mfussenegger
mfussenegger

Reputation: 3971

In order to do bulk imports from file the COPY FROM statement can be used (see https://crate.io/docs/stable/sql/reference/copy_from.html). But there is only support for JSON formatted files so you'll probably need to convert the text files first.

Not sure if there are any plans to add support for other formats, but if you create a github issue requesting the feature you'll get feedback once it has been implemented.

There are also docs available on how to migrate from mysql and mongodb

Upvotes: 2

Andres Palacios
Andres Palacios

Reputation: 23

I have quickly imported data from MySQL to Crate 0.40 installing Ruby on Rails in the same server of the MySQL DB, and then using the Mysql2JSON gem (See the Mysql2xxx part).

Crate requires a one line per register JSON file. So, you have to edit the output replacing the [", ",", "] with ", "/n", " in the mysql2xxXX gem source, in order to have a format like this in the output:

{"id": 1, "quote": "Don't panic"}
{"id": 2, "quote": "Would it save you a lot of time if I just gave up and went mad now?"}

After exporting the MySQL JSON info with the Mysql2Json gem you have to upload the file to the Create server and put in the Crate console:

COPY table_name FROM 'file:///tmp/import_data/quotes.json'

Upvotes: 1

Related Questions