Is semantic search the same as querying a vector database?

Question

RAG (Retrieval augmented generation) is a technique to search for relevant documents before asking the LLM to generate a natural answer.

For what I understand:

• load the documents (from files, scraping, sql queries, etc)
• split into chunks of text
• generate synthetic questions for each chunk
• embed the questions (e.g., using OpenAI embeddings)
• save to a vector database

Then when the user sends a message, we embed the question (user message) and search the embeddings (e.g., cosine similarity, knn).

So I heard about Semantic Search, apparently it understands the context of what the user is asking, can find the perfect amount of context to answer. I searched about it, I found this research on github: Semantic Graphs for Generating Deep Questions and found Graph Databases and several articles about Semantic Search using Knowledge Graphs.

I'm assuming that semantic search is some kind of search in graph database that knows how to retrieve the right amount of context.

I tried to find some existing services for semantic search and AWS came up: Semantic search in Amazon OpenSearch Service. But it seems to be just a vector search with extra sauce, not really a semantic search using knowledge graph in a graph database.

Now I'm confused if semantic search is actually the same as vector search. And I have no idea on how to use a Graph Database and why they're not using for semantic search in AWS

Answer 1

Semantic Search and Vector Search do have a lot of overlap and both terms will bring different opinions and interpretations, so I will share mine and realize they may differ from others. Also this is a very deep topic but I will try to answer as succinctly as possible.

Semantic Search often involves encoded domain expertise rules in the graph itself using an ontology or other rules based system, such as Mammal is a subset of Animal. So if someone is searching for an Animal, anything labeled as a Mammal should also be considered. Answering this often involves a graph traversal, such as Animal => Mammal => (whales, seals, primates, ...)

Vector Search requires a pre-process to generate a series of vectors from a document or term that, when considered holistically, will place items with similar meaning together in distance. Consider if an Animal is encoded as vectors [1.0,1.0,1.0] and a Rock is encoded as [0.0, 0.0, 0.0], then a Mammal might be [0.95,0.9,0.98], so if you plotted these 3-dimensionally then you would see that the Mammal is much closer to an Animal than a Rock. The real challenge is generating the vectors in a meaningful way so a Mammal is closer to a Rock and the same idea applies to every concept in your data.

To add my opinion on an additional value of a Knowledge Graph for RAG, consider patents. The first patent for 3D printing is titled "stereolithography". No boolean search or vector encoding process I've encountered will place "stereolithography" near "3D printing" and therefore you'd be unlikely to get that result. But later 3D printing patents do mention that term and have references back to the original patent via citations, and therefore you would find it via boolean/semantic search + graph traversal. Now if you capture that entire family of patents and feed it as prompts into the LLM, theoretically you will get a much more accurate response.

[Disclaimer: This example is provided as an understandable example, but many LLM models will identify the above referenced patent when prompted on the topic because there is significant existing literature it was trained upon that explicitly states that connection, not because it did any meaningful semantic or vector search]

Answer 2

Starting from version 2.9+, OpenSearch does support semantic searches using models deployed either within the cluster, pre-trained models, and models running outside the cluster such as Amazon SageMaker, Amazon Bedrock, etc. To use this feature, the first step is registering the model that will augment the search with a semantic (meaning oriented) value, which will be used during the search execution to find which documents match the search criteria, using the vectors from the text embedding.

For this to work, you don't need to use graph-oriented databases. Graph databases are useful for finding data over complex relationships on highly connected datasets, like trees and heaps. Any dataset provided by the model would work. With this said, you can surely use a graph database to enrich your model with complex relationships.

For more information about the support of semantic search in OpenSearch:

https://opensearch.org/docs/latest/search-plugins/semantic-search/