how to implement full text search in database

Question

I understand that full text indexing and search for a database can be enabled by a lot of pre-packaged products. However, just out of academical curiosity, I wonder how are those full text indexes actually implemented. I have tried to google for results with little answer. Please any feedback would be much appreciated.

Answer 1

Full text searches are supported by quite a few database engines these days as a core feature.

As for implementation I think your best bet is to check out postgres full text searches, as you can

• find a lot of material on how it is implemented
• actually change and play with the parsers (for example optimize for certain domain)

There are further details and concept explained on wikipedia:

• full text indexes, and you can also check out
• open source and free full text search engines as normally you will find supporting documentation explaining inner workings of those too (I have heard good things about Lucene/Solr from this list)

Answer 2

Probably by creating dictionaries of "words" and maybe a bit of lexical analysis. (Note that fulltext searches whole words and not parts of words, so indexing may be constrained to that.)