CODEWITHSUNDEEP
mysqlcsvpentahokettlepdi
Deepesh
Deepesh

Reputation: 840

Creating a table from .csv file using PDI Kettle

I'm new to PDI and I'm working with PDI Kettle, I have 40 .csv files with different number of columns ,I want to create tables out of those files in a single transformation, I have used a "CSV File Input" step to select a file and "Table Output" step to create table but for creating 40 tables out of those 40 files I again need to select these two steps,so is there any way to create all 40 tables in one go in a single transformation is it possible,pls help me with the same Thanks in advance

Upvotes: 0

Views: 3882

Answers (1)

bolav
bolav

Reputation: 6998

To do this in Pentaho with the standard steps is a bit involved. To read the CSV and get the headings, and then read the data, you need to use ETL Metadata injection.

First read the header with columns name, and use ETL Metadata injection to read the data in another transformation.

To auto create the databases is not straight forward, as this is something the main developer of Pentaho discourages.

Here is the answer and an example of how to auto create a table: Perform an auto CREATE TABLE to store the output of a transformation.

So you would run a job, that passes the filename and tablename to a transformation. The transformation will use ETL Metadata injection to read the CSV into correct fields and meta.getSQLStatementsString(); to get the DDL of the database to store the data.

Upvotes: 1

Related Questions