CODEWITHSUNDEEP
javaoptimizationsearch
Ace
Ace

Reputation: 4443

Optimizing a simple search algorithm

I have been playing around a bit with a fairly simple, home-made search engine, and I'm now twiddling with some relevancy sorting code.

It's not very pretty, but I'm not very good when it comes to clever algorithms, so I was hoping I could get some advice :)

Basically, I want each search result to get scoring based on how many words match the search criteria. 3 points per exact word and one point for partial matches

For example, if I search for "winter snow", these would be the results:

Here's the code:

String[] resultWords = result.split(" ");
String[] searchWords = searchStr.split(" ");
int score = 0;
for (String resultWord : resultWords) {
    for (String searchWord : searchWords) {
        if (resultWord.equalsIgnoreCase(searchWord))
            score += 3;
        else if (resultWord.toLowerCase().contains(searchWord.toLowerCase()))
            score++;
    }
}

Upvotes: 8

Views: 3364

Answers (5)

ATorras
ATorras

Reputation: 4293

You can use regular expressions for finding patterns and lengths of matched patterns (for latter classification/scoring).

Upvotes: 0

bruno conde
bruno conde

Reputation: 48265

Your code seems ok to me. I suggest little changes:

Since your are going through all possible combinations you might get the toLowerCase() of your back at the start.

Also, if an exact match already occurred, you don't need to perform another equals.

    result = result.toLowerCase();
    searchStr = searchStr.toLowerCase();

    String[] resultWords = result.split(" ");
    String[] searchWords = searchStr.split(" ");
    int score = 0;
    for (String resultWord : resultWords) {
        boolean exactMatch = false;
        for (String searchWord : searchWords) {
            if (!exactMatch && resultWord.equals(searchWord)) {
                exactMatch = true;
                score += 3;
            } else if (resultWord.contains(searchWord))
                score++;
        }
    }

Of course, this is a very basic level. If you are really interested in this area of computer science and want to learn more about implementing search engines start with these terms:

Upvotes: 3

dfa
dfa

Reputation: 116334

  • stemming
  • for acronyms case sensitivity is important, i.e. SUN; any word that matches both content and case must be weighted more than 3 points (5 or 7)?
  • use the strategy design pattern

For example, consider this naive score model:

interface ScoreModel {
     int startingScore();
     int partialMatch();
     int exactMatch();
}

...

int search(String result, String searchStr, ScoreModel model) {
    String[] resultWords = result.split(" ");
    String[] searchWords = searchStr.split(" ");
    int score = model.startingScore();

    for (String resultWord : resultWords) {
        for (String searchWord : searchWords) {
            if (resultWord.equalsIgnoreCase(searchWord)) {
                score += model.exactMatch();
            } else if (resultWord.toLowerCase().contains(searchWord.toLowerCase())) {
                score += model.partialMatch();
            }
        }
    }

    return score;
}

Upvotes: 1

Sean Owen
Sean Owen

Reputation: 66886

1) You can sort searchWords first. You could break out of the loop once your result word was alphabetically after your current search word.

2) Even better, sort both, then walk along both lists simultaneously to find where any matches occur.

Upvotes: 0

user106014
user106014

Reputation:

Basic optimization can be done by preprocessing your database: don't split entries into words every time.

Build words list (prefer hash or binary tree to speedup search in the list) for every entry during adding it into DB, remove all too short words, lower case and store this data for further usage.

Do the same actions with the search string on search start (split, lower case, cleanup) and use this words list for comparing with every entry words list.

Upvotes: 0

Related Questions