Quarkus Langchain4j Image Support For Local Ollama Models

[myfear]

Description

I want to work with MoE models like qwen2.5vl locally.
Using the quarkus-langchain4j-ollama extensions already let's me use the following code:

  Image langchainImage = Image.builder()
                    .base64Data(base64String) // Use the base64 encoded string
                    .mimeType(mimeType)
                    .build();

But using it as input for the AIService right now:

String imageDescription = imageService.describeImage(langchainImage);

Leads to:

dev.langchain4j.exception.InternalServerException: {"error":"Failed to create new sequence: failed to process inputs: image: unknown format\n"}

Example Code that I would like to see work: https://github.com/myfear/ejq_substack_articles/blob/main/quarkus-image-describer/src/main/java/org/acme/ImageDescriberResource.java

Implementation ideas

No response

[myfear]

Moved to Quarkus Langchain4j quarkiverse/quarkus-langchain4j#1531

[geoand]

🙏🏽