CODEWITHSUNDEEP
spring-ai
user2693135
user2693135

Reputation: 1316

How to generate image description with GPT-4o with SpringAI 1.0.0 Milestone

I am using Spring AI 1.0.0-M1 trying to submit an image to GPT-4o for description.

Below is the code:

@Autowired
final ChatClient chatClient;

@Override
public String getDescription(MultipartFile file) throws IOException {
OpenAiChatOptions chatOptions = OpenAiChatOptions.builder()
        .withModel(OpenAiApi.ChatModel.GPT_4_O.getValue())
        .build();

Resource resource = new ByteArrayResource(file.getBytes());

var userMessage = new UserMessage(
        "Explain what do you see in this picture?", 
        List.of(new Media(MimeTypeUtils.IMAGE_JPEG, resource)));
var response = chatClient.prompt().user(userMessage.getContent()).call().chatResponse();
return response.getResult().getOutput().getContent();

}

I keep getting this message:

"I'm sorry, but I can't see pictures or images. If you describe the picture to me, I'll do my best to help you understand or analyze it!"

Am I doing something wrong?

Upvotes: 1

Views: 48

Answers (1)

jonghoonpark
jonghoonpark

Reputation: 31

Although a long time has passed, I wanted to leave a comment for the record.

I tested this with Spring AI 1.0.0-M1 and found that it worked fine. I suspect the problem might have been related to the resource handling process.

Currently, the version has been upgraded to 1.0.0-M6. I recommend trying again with the latest version!

Upvotes: 0

Related Questions