CODEWITHSUNDEEP
indexingsolrweb-crawler
user1420373
user1420373

Reputation: 33

How can I exclude large files while indexing?

Is there a way to config Solr to ignore large files while indexing?

I'm trying to index a network drive but can't figure out how to ignore large files (>20MB).

Thanks

Upvotes: 1

Views: 351

Answers (1)

rleir
rleir

Reputation: 821

Try something of this nature:

$ find /mnt -type f -size -20M -exec /opt/solr/bin/post -c wizbang {} \;

If you use Tika, it has a file size limit though it is not 'ignore'ing the file:

Apache Tika and character limit when parsing documents

How to read large files using TIka?

Upvotes: 1

Related Questions