Tushar Kulange
Tushar Kulange

Reputation: 1

I am not getting proper response during similarity search

I am trying to do similarity search using chroma locally, and ollama embeddings used for creating embeddings

from langchain_community.embeddings import OllamaEmbeddings
from langchain.document_loaders import TextLoader, UnstructuredMarkdownLoader, CSVLoader 
import chromadb
from langchain.text_splitter import MarkdownTextSplitter, CharacterTextSplitter,       MarkdownHeaderTextSplitter  
from langchain.vectorstores import Chroma
from langchain_community.llms import Ollama

embeddings = OllamaEmbeddings(model="nomic-embed-text")
llm = Ollama(model="phi")
loader = TextLoader("./try.md")
data = loader.load()
text_splitter=CharacterTextSplitter.from_tiktoken_encoder(separator="##",chunk_size=500,chunk_overlap=0)
docs=text_splitter.split_documents(data)
db = Chroma.from_documents(docs, embeddings,persist_directory="./try_few_show",  normalize_embeddings=False)
db.persist()
db_new_connection=Chroma(persist_directory='./try_few_show',embedding_function=embeddings)
retriever=db_new_connection.as_retriever()
result=retriever.get_relevant_documents("Pdetails")

I tried above text which i mention in problem details.. and i am expecting the exact similarity search. like suppose i search "pdetails". i need exact 'pdetails' related embedding not other which containing only "p" or "details"

Upvotes: 0

Views: 201

Answers (0)

Related Questions