CODEWITHSUNDEEP
etldata-integration
Kaushal
Kaushal

Reputation: 938

ETL Tool Which is most configurable

I am looking for best suited ETL Tool for the following criteria.

  1. Supports MongoDB
  2. Accepts Metadata as input (Or accepts file and builds its metadata on the fly)
  3. provides configurable Mapping. (mapping can be defined from outside development, using some file ot table)

Please suggest the tool which caters to the above needs.

Upvotes: 2

Views: 864

Answers (1)

fenix
fenix

Reputation: 1746

Hmm, your questing is looking most configurable ETL tool. From past years of experience in ETL process, I can inform you that you will never find such tool that meets all your demands. Especially when you have Enterprise level data warehouse (needed because of high and complex reporting needs), the only one software solution is to build your own custom project based ETL software, which is often ungrateful.

But (big BUT), you can achieve at least 80% of needs with existing tools. Plugins, smart usage of scripts, good data-flow design and (if needed) small custom software in pair with scheduling could help you out to fulfill imagined process. ETL process doesn't seem to be different in compare to any other work - 80% of the work is done in 20% of time, and the rest of work (20%) is done in 80% of time.

My suggestion for you:

Pentaho Data Integration - free and open source

PDI is powerfull ETL tool, and surley can meet your demands. There is a plenty of plugins, solid level community and fine API if you're going to develop more plugins.

Pentaho Data Integration + Integration Server - Enterprise Edition - "cheap enough" for almost every medium size projects

Enterprise edition has everything like free edition, including more plugins (JMS producer for example), version control system, instaview's and ect. Beside, it has it own Server so scheduling is software based (not OS based), logging, better management and most important thing - support!

Informatica or Microsoft SSIS - expensive and brilliant

I would not wasting words for this tools. Informatica is primary ETL oriented company that using Informatica on high level require deep understanding of DB/DWH design, ETL process, PL/SQL, dimensional modeling ect.

SSIS is primary constructed for SQL Server, so I don't see high usage needs if at least one of your source db or target db (DWH) is not running on SQL Server.

Conclusion

This is just a scratch of plenty tools that market provide to us. Someone else would probably not even mention these tools. Please look one of the lists.

Almost each BI system has it own ETL tool. Maybe the good choice would be to use it together, in that way you will be in possibility to use maximum from both.

Note: Good ETL project manager, or ETL developer can extend tool advantages to level that better/more expensive have!

Upvotes: 1

Related Questions