CODEWITHSUNDEEP
csvsolrlucenetoken
Robert Watts
Robert Watts

Reputation: 179

Store and export raw tokens in a solr index

I'd like to store tokens generated by Solr during indexing, such as DictionaryCompoundWordToken and then export them, hopefully using CSVResponseWriter. Is there a way to do that?

I know it's possible to use the Analysis tool to provide values and see how they are tokenized, but I am unaware of how to do this for entirety of the index, or at least on a query basis.

Upvotes: 0

Views: 559

Answers (2)

Persimmonium
Persimmonium

Reputation: 15771

Let's see, I think what you want is to store, alongside the original content of some field, the field value but after it goes through some analysis chain, right?

You would think copyFields would help, but they don't as if you store them, the original field value is stored. You need to use an updateProcessor. Look at this talk Erik Hatcher gave, minutes 7:30 to 20:00 aprox, and you will see exactly this case explained very well, with examples and all.

Once you have that stored in the index, you can return it and do anything you like.

Upvotes: 2

Arun
Arun

Reputation: 1787

One way to look at this is this, you will index your document content into a field "mytext" with your DictionaryCompoundWordToken or any other analysis that fits your needs. Then you can facet on "mytext" with q=*:* , your query would look like this : http://localhost:8983/solr/collection1/select?q=*%3A*&start=0&rows=1&wt=xml&indent=true&facet=true&facet.field=mytext That should give all the tokens that went into mytext. But i am not 100% sure of your expectation with what you said in the question. Let me know this helps.

Upvotes: 0

Related Questions