CODEWITHSUNDEEP
vespa
Kexin Wang
Kexin Wang

Reputation: 25

Vespa response: Summary data is incomplete: Timed out waiting for summary data

I am deploying a simple text retrieval system with Vespa. However, I found when setting topk to some large number, e.g. 40, the response will include the error message "Summary data is incomplete: Timed out waiting for summary data." and also some unexpected ids. The system works fine for some small topk like 10. The response was as follows:

{'root': {'id': 'toplevel', 'relevance': 1.0, 'fields': {'totalCount': 1983140}, 'coverage': {'coverage': 19, 'documents': 4053984, 'degraded': {'match-phase': False, 'timeout': True, 'adaptive-timeout': False, 'non-ideal-state': False}, 'full': False, 'nodes': 1, 'results': 1, 'resultsFull': 0}, 'errors': [{'code': 12, 'summary': 'Timed out', 'message': 'Summary data is incomplete: Timed out waiting for summary data. 1 responses outstanding.'}], 'children': [{'id': 'index:square_datastore_content/0/34b46b2e96fc0aa18ed4941b', 'relevance': 44.44359956427316, 'source': 'square_datastore_content'}, {'id': 'index:square_datastore_content/0/16dbc34c5e77684cd6f554fd', 'relevance': 43.94371735208669, 'source': 'square_datastore_content'}, {'id': 'index:square_datastore_content/0/9f2fd93f6d74e88f96d7014f', 'relevance': 43.298002713993384, 'source': 'square_datastore_content'}, {'id': 'index:square_datastore_content/0/76c4e3ee15dc684a78938a9d', 'relevance': 40.908658368905485, 'source': 'square_datastore_content'}, {'id': 'index:square_datastore_content/0/c04ceee4b9085a4d041d8c81', 'relevance': 36.13561898237115, 'source': 'square_datastore_content'}, {'id': 'index:square_datastore_content/0/13806c518392ae7b80ab4e4c', 'relevance': 35.688377118163714, 'source': 'square_datastore_content'}, {'id': 'index:square_datastore_content/0/87e0f13fdef1a1c404d3c8c6', 'relevance': 34.74150232183567, 'source': 'square_datastore_content'}, ...]}}

I am using the schema:

schema wiki {
document wiki {
    field title type string {
        indexing: summary | index
        index: enable-bm25
    }
    field text type string {
        indexing: summary | index
        index: enable-bm25
    }
    field id type long {
        indexing: summary | attribute
    }
    field dpr_embedding type tensor<bfloat16>(x[769]) {
        indexing: attribute | index
        attribute {
            distance-metric: euclidean
        }
    }
}
fieldset default {
    fields: title, text
}
rank-profile bm25 inherits default {
    first-phase {
        expression: bm25(title) + bm25(text)
    }
}
rank-profile dpr inherits default {
    first-phase {
        expression: closeness(dpr_embedding)
    }
}}

This error occurred for both BM25 and DPR dense retrieval. So what is wrong? And what can I do? Thanks.

Upvotes: 1

Views: 626

Answers (1)

Jo Kristian Bergum
Jo Kristian Bergum

Reputation: 3184

The default Vespa timeout is 500 ms and can be adjusted by &timeout=x where x is given in seconds, e.g &timeout=2 would use an overall request timeout of 2 seconds.

A query is executed in two protocol phases:

  1. Find the top k matches given the query/ranking profile combination, each node returns up to k results
  2. The stateless container merges the results and finally asks for summary data (e.g the contents of only the top k results)

See https://docs.vespa.ai/en/performance/sizing-search.html for an explanation of this.

In your case you are hit by two things

  1. A soft timeout at the content node (coverage is reported to be only 19%) so within the default timeout of 500ms it could retrieve and rank 19% of the available content. At 500ms minus a factor it timed out and returned what it had retrieved and rank up til the.
  2. When trying to use the time left it also timed out waiting for the hits data for those documents which it managed to retrieve and rank within the soft timeout, this is the incomplete summary data response.

Generally, if you want cheap BM25 search use WAND (https://docs.vespa.ai/en/using-wand-with-vespa.html) If you want to search using embeddings, use ANN instead of brute force NN. We also have a complete sample application reproducing the DPR (Dense Passage Retrieval) here https://github.com/vespa-engine/sample-apps/tree/master/dense-passage-retrieval-with-ann

Upvotes: 1

Related Questions