CODEWITHSUNDEEP
langchainvector-databasefaiss
Rasik
Rasik

Reputation: 2420

How to excludes the excludes the terms in Query while Similarity Search

I'm building a recipe search system using FAISS vector store, LangChain, and the Hugging Face sentence-transformers/all-MiniLM-L6-v2 model for embeddings. The system is designed to find recipes based on various parameters such as ingredients, dietary restrictions, cuisine, and meal type.

To make the search query I have use the following prompt.

query_prompt = ChatPromptTemplate.from_template(
    """Analyze the following search parameters for a recipe search query:
    Dietary Restrictions: {dietary_restrictions}    
    Ingredients: {ingredients}
    Dietary Restrictions: {dietary_restrictions}
    Cuisine: {cuisine}
    Meal Type: {meal_type}

    Transform these parameters into a natural language query that can be used to search for recipes. Include the definition of the dietary restrictions. 

    Provide your analysis in the following format:
    Query: [Transformed natural language query]
    Filter Terms: [List of key terms for filtering, including ingredients, dietary restrictions, cuisine, and meal type]
    """
)

This generates a search query like:

Search Query: "Dairy-free American snack recipes, excluding all products derived from milk such as cheese, yogurt, and butter."

and i used this query for the similarity_search:

search_results = vector_store.similarity_search(query, k=1000)  # Retrieve top 1000 results initially

However, The similarity search is not properly excluding terms specified in the query.

For example:

When i used the following search parameter:

find_recipes(
    ingredients=[],
    dietary_restrictions=["dairy free"],
    cuisine="American",
    meal_type="snacks"
)

Some of the results still include recipes with dairy products:

### 3. Milk Peda Recipe
### 12. Yogurt Cheese (Labneh)

How can I modify my similarity search to properly exclude terms or concepts mentioned in the "excluding" part of the query? Is there a way to implement negative matching?

Any insights on how to improve the search to respect exclusion terms?

Upvotes: 0

Views: 59

Answers (1)

Arindam
Arindam

Reputation: 81

I think the design is wrong. Similarity search will give you those vectors similar to your query. But it may not work.

You need to build your vectors using Chroma() and use some retrieval methods to use "filter" condition.

With this design of yours, it may not work.

Upvotes: 0

Related Questions