CODEWITHSUNDEEP
langchainsimilarity-search
Uno_Count
Uno_Count

Reputation: 3

Does the question/query vectorized in similaritySearch (Langchain) Does it use tokens

const simpleRes = await vectorStore.similaritySearchWithScore(question);
console.log(simpleRes);

How can I know if it uses tokens in similarity search, does my question converted to vector and uses OpenAIEmbeddings before it search the closest vector in the database?

Upvotes: 0

Views: 377

Answers (2)

Juan Manuel Martínez
Juan Manuel Martínez

Reputation: 1

As far as I can see on the code, langchain's similaritySearch method embeds your query so it can perform the search. You can confirm that by looking at model usage report at OpenAI dashboard, where you can find the calls to the embedding model.

It does not use tokens as defined in completion generations, though. You will find model usage as "n_context_tokens_total", which I find quite confusing.

HTH.

Juan Manuel.

Upvotes: 0

Yilmaz
Yilmaz

Reputation: 49571

when you do a similarity search in the database there is no token usage.

Tokens are the basic units of text or code that an LLM AI uses to process and generate language.

You consume tokens when you send the most similar chunks to the LLM

Upvotes: 0

Related Questions