How is site specific search done?

Question

Can someone please tell how site specific searches are done. For example, search in the stackoverflow.com

Thanks

Answer 1

Search Google for "site:stackoverflow.com [SEARCH STRING]". This will only return pages that are hosted through *.stackoverflow.com.

Or, like Rob said, if you're looking to setup your own custom in-site search bar, you can use Google CSE.

If you're interested in the nuts-and bolts of "how site-specific searches are done" (your words), then you can take heart in that it is very easy; the URLs' host names ("http://www.example.com/path/to/page" => "www.example.com") are matched against the domain being searched.

Something that will really mess this up, though is the existence of DNS aliases and duplicate entries.

Answer 2

The search here on Stackoverflow is, as I can recall, done using Lucene.

You build an index with details you want to search through and link these to the posts / other things you want the search to find.

More details here: http://en.wikipedia.org/wiki/Lucene

They have used the SQL server search algorithm but were not really satisfied with it, if I remember correctly.

Here is the blog post by Jeff: https://blog.stackoverflow.com/2008/11/sql-2008-full-text-search-problems/

Answer 3

Google, Bing, and Yahoo it is done as:

c++ site:stackoverflow.com

Answer 4

Depending on what you want you can build your own implementation to search trough a database or use Google Custom Search on Google CSE