Langchain4j pgvector implementation as an EmbeddingStore?

Question

I'm building a RAG based AI service using Langchain4j. I have one microservice that is ingesting and saving my documents (pdfs, csv, words...) in my PostgreSQL DB (with vector extension) as embeddings.

From the other hand I'm building another microservice to hold the AI conversation logic.

To do this I'm creating the next beans

    @Bean
public EmbeddingStore<TextSegment> embeddingStore() {
    return new InMemoryEmbeddingStore<>();
}

    @Bean
public ContentRetriever contentRetriever() {
    return EmbeddingStoreContentRetriever.builder()
            .embeddingStore(embeddingStore())
            .embeddingModel(bedrockTitanEmbeddingModel())
            .maxResults(10) // on each interaction we will retrieve the 5 most relevant segments
            .minScore(0.2) // we want to retrieve segments very similar to the user query
            .build();
}
    @Bean
public RetrievalAugmentor retrievalAugmentor() {
    return DefaultRetrievalAugmentor.builder()
            .queryTransformer(queryTransformer())
            .contentRetriever(contentRetriever())
            .build();
}

    @Bean
public AiAgent aiAgent() {
    return AiServices.builder(ErekyAiAgent.class)
            .retrievalAugmentor(retrievalAugmentor())
            .chatLanguageModel(bedrockAnthropicChatModel())
            .contentRetriever(contentRetriever())
            .build();
}

The ContentRetriever is asking me as a mandatory parameter the embeddingStore. Now for testing I'm using the memory one but I saw that Langchain4j has an implementation with pgvector.

In the flow what I'm doing is:

1. Doing the query to my PostgreSQL database with the user asked question
2. Returning the document text list found
3. Transforming the List of Strings containing the document text I got to a list of List<TextSegment> that is a type of langchain4j library.
4. Then I need to transform the List<TextSegment> to embeddings again and add them along with the List<TextSegment> without embedding them to the embedding store I'm using.

The logic is

List<String> documentTexts = getDocumentTextsFromUserQuestion(promptDto);
        List<TextSegment> textSegments = getTextSegments(documentTexts);
        embeddingStore.addAll(embedComponent.getEmbeddingsFromTextSegments(textSegments), textSegments);
        return new PromptDTO(aiAgent.answer(documentTexts, promptDto.getText()));

I saw that for some reason the logic always need for me to add that data to the embedding store to be able to give a correct answer based on my data. When I used the pgvector implementation of Langchain4j and did the same thing I saw that the implementation is creating a table in my DB with the data I already had saved before inserted in this new table to give the answer. And the data is being duplicated, there is a way to make this work without that?

And since I already have the data saved in the DB, I can't directly do the call to the AI with the data found from the user question + the user question?

I did it like this in Python calling chain.run being documents the data found in the DB and the question being the user question and it works and I don't need this intermmediate embedding store.

chain = load_qa_chain(llm, chain_type="stuff")
    # Call to the model
    # response = st.session_state.conversation({'question': user_question})
    response = chain.run(input_documents=document, question=user_question)

Answer 1

Not sure if you've already fixed your blockers. I ran into something similar with MongoDb, and although it's not a new collection being created in the database when using the MongoDB implementation of the Embedded Store, it happens to be another issue with ObjectId's. My project was underway, until I realized I could use langchain4j to implement various AI actions. So, I already had various queries and aggregations to my MongoDB Collections in service classes, with data already stored as embeddings. What I did was reuse these queries and store them in a InMemoryEmbeddingStore. My query already received the most relevant embeddings from original prompt. Or you can store the information you want to embed as plain text in your DB and use a langchain4j EmbeddingModel to convert data into embeddings to push into that InMemoryEmbeddingStore. Here's the first iteration of my work-around.

    // We will use initialize and use the ADA_002 to create embeddings on text field from database
    EmbeddingModel embeddingModel = new OpenAiEmbeddingModel.OpenAiEmbeddingModelBuilder()
            .modelName(OpenAiEmbeddingModelName.TEXT_EMBEDDING_ADA_002)
            .apiKey(openAiKey)
            .maxRetries(2)
            .build();

    InMemoryEmbeddingStore<TextSegment> inMemoryEmbeddingStore = new InMemoryEmbeddingStore<>();

    // we can use query service to get most relevant search results
    List<QueryResults> querySearchResult = queryService.getDataFromVectorSearch(prompt);

    querySearchResult.forEach(result -> {
        TextSegment textSegment = TextSegment.from(result.getText());
        Embedding embedding = embeddingModel.embed(textSegment).content();
        inMemoryEmbeddingStore.add(embedding, textSegment);
    });

    // use embedding model and in memory store as retriever for optimal answer
    EmbeddingStoreContentRetriever retriever = EmbeddingStoreContentRetriever.builder()
            .embeddingModel(embeddingModel)
            .embeddingStore(inMemoryEmbeddingStore)
            .build();

    String question = prompt;

    // given the prompt with the added context and user question, we can now build our model
    ChatLanguageModel chatLanguageModel = OpenAiChatModel.builder()
            .apiKey(openAiKey)
            .modelName(GPT_4o)
            .timeout(Duration.ofSeconds(60))
            .build();

    // using our AI system interface build the prompt
    Bot bot = AiServices.builder(Bot.class)
            .chatLanguageModel(chatLanguageModel)
            .contentRetriever(retriever)
            .build();

    return bot.chat(1, question);

Something great about langchain4j's InMemoryEmbeddingStore is that you can save messages with an Id. There are few things still being updated frequently to langchain4j, so I'm sure you'll be able to find a suitable solution in no time. Hope this helps!