CODEWITHSUNDEEP
springartificial-intelligenceollamapgvectorspring-ai
Rishabh Raghwendra
Rishabh Raghwendra

Reputation: 41

Mistral Model not found issue in Spring AI

I am exploring Spring AI. I created a RAG application using PgVector store but when I hit the endpoint it's showing me the below error:

Not Found - {"error":"model \"mistral\" not found, try pulling it first"}

Even though I have mentioned in the application.properties file that I want to use llama3.2:1b model but still it's looking for mistral I don't know why.

Below is my controller:

@GetMapping("/ai/generate")
    public Map<String,String> generate(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
        List<Document> similarDocuments = vectorStore.similaritySearch(SearchRequest.query(message).withTopK(2));
        List<String> contentList = similarDocuments.stream().map(Document::getContent).toList();
        PromptTemplate promptTemplate = new PromptTemplate(prompt);
        Map<String, Object> promptParameters = new HashMap<>();
        promptParameters.put("input", message);
        promptParameters.put("documents", String.join("\n", contentList));
        Prompt prompt =promptTemplate.create(promptParameters);
        return Map.of("generation", chatModel.call(prompt).getResult().getOutput().getContent());
    }

application.properties file:

spring.application.name=codeAI

spring.main.allow-bean-definition-overriding=true

spring.datasource.url=jdbc:postgresql://localhost:5432/postgres
spring.datasource.username=postgres
spring.datasource.password=1

spring.ai.vectorstore.pgvector.index-type=HNSW
spring.ai.vectorstore.pgvector.distance-type=COSINE_DISTANCE
spring.ai.vectorstore.pgvector.dimensions=1536

spring.ai.ollama.base-url=http://localhost:11434
spring.ai.ollama.chat.options.model=llama3.2:1b
spring.ai.ollama.chat.options.temperature=0.7

I searched regarding this on interent but didn't find any solution. I am not getting why it's fetching mistral model even I have mentioned that I want to use the llama3.2:1b model in the application.properties file.

Upvotes: 4

Views: 500

Answers (1)

Andre Kapp
Andre Kapp

Reputation: 43

The problem is the missing embedding model property. You only have the chat model configured.

Example:

ai:
ollama:
  embedding:
    model: llama3.2
  chat:
    options:
      model: llama3.2
      temperature: 0.5
vectorstore:
  pgvector:
    index-type: hnsw
    distance-type: cosine_distance
    dimensions: 1536

Upvotes: 2

Related Questions