CODEWITHSUNDEEP
apachelucenecompass-lucene
Valath
Valath

Reputation: 920

Can Apache Lucene replace ETL tools for text processing

I got a flat file extract of 15GB and its spec which helps to identify each row record and how to split each row to gather required information. I am planning to make use of ETL tools, because I thought text processing of this bulk file cannot be achieved in java. But I came to read about Lucene now. Now am really confused. Following are my doubts:

  1. Can Apache Lucene can process a 15GB flat text file without memory issues.
  2. What is the performance benchmark for 15GB flat file to read each row, identify the identifier and based on the identifier split the row data and load it into mapped db.
  3. Can I use Apache Lucene for this task OR go for ETL tools?

Upvotes: 0

Views: 113

Answers (0)

Related Questions