CODEWITHSUNDEEP
lucenelucene.net
Xodarap
Xodarap

Reputation: 11859

Lucene field from TokenStream with stored values

I have a field which needs to come from a token stream; it cannot be instantiated with a string and then analyzed into tokens. For example, I might want to combine the data from multiple columns (in my RDBMS) into a single Lucene field, but I want to analyze each column in its own way. So I cannot simply concat them all as a single string then analyze the resulting string.

The problem I am running into now is that fields created from token streams cannot be stored, which makes sense in the general case since the stream may not have an obvious string representation. However, I know the string representation, and I would like to store that.

I tried adding the same field twice, once with it being stored and having string data and once with it coming from a token stream, but it seems that this can't be done. Apart from some hack like adding a field with a name of "myfield__stored" is there a way to do this?

I am using 2.9.2.

Upvotes: 4

Views: 1378

Answers (1)

Xodarap
Xodarap

Reputation: 11859

I found a way. You can sneak it in by instantiating it as a normal field but calling SetTokenStream later:

Field f = new Field(Name, StringValue, Store, Analyzed, TV);
f.SetTokenStream(TokenStreamValue);

Because the reader/string value is only indexed if the token stream value is null, the token stream value will be indexed. The store methods look at string/reader regardless of token stream, so it will be this value which is stored.

Upvotes: 3

Related Questions