聊聊Spring AI的Image Model

序

本文主要研究一下Spring AI的Image Model

ImageModel

org/springframework/ai/image/ImageModel.java

@FunctionalInterfacepublic interface ImageModel extends Model<ImagePrompt, ImageResponse> {    ImageResponse call(ImagePrompt request);}

ImageModel继承了Model接口，其call方法入参为ImagePrompt，返回ImageResponse

ImagePrompt

org/springframework/ai/image/ImagePrompt.java

public class ImagePrompt implements ModelRequest<List<ImageMessage>> {   private final List<ImageMessage> messages;    private ImageOptions imageModelOptions;   public ImagePrompt(List<ImageMessage> messages) {       this.messages = messages; }   public ImagePrompt(List<ImageMessage> messages, ImageOptions imageModelOptions) {       this.messages = messages;     this.imageModelOptions = imageModelOptions;   }   public ImagePrompt(ImageMessage imageMessage, ImageOptions imageOptions) {        this(Collections.singletonList(imageMessage), imageOptions);  }   public ImagePrompt(String instructions, ImageOptions imageOptions) {      this(new ImageMessage(instructions), imageOptions);   }   public ImagePrompt(String instructions) {     this(new ImageMessage(instructions), ImageOptionsBuilder.builder().build());  }   @Override    public List<ImageMessage> getInstructions() {       return this.messages;   }   @Override    public ImageOptions getOptions() {        return this.imageModelOptions;  }   }

ImagePrompt实现了ModelRequest接口，它定义了messages及imageModelOptions属性，其getInstructions返回的是List<ImageMessage>类型

ImageMessage

org/springframework/ai/image/ImageMessage.java

public class ImageMessage {  private String text;   private Float weight; public ImageMessage(String text) {     this.text = text; }   public ImageMessage(String text, Float weight) {       this.text = text;     this.weight = weight; }   }

ImageMessage定义了text及weight属性

ImageOptions

org/springframework/ai/image/ImageOptions.java

public interface ImageOptions extends ModelOptions {    @Nullable    Integer getN();  @Nullable    String getModel();   @Nullable    Integer getWidth();  @Nullable    Integer getHeight(); @Nullable    String getResponseFormat();  @Nullable    String getStyle();}

ImageOptions继承了ModelOptions，它定义了getN、getModel、getWidth、getHeight、getResponseFormat、getStyle方法

ImageResponse

org/springframework/ai/image/ImageResponse.java

public class ImageResponse implements ModelResponse<ImageGeneration> {  private final ImageResponseMetadata imageResponseMetadata;      private final List<ImageGeneration> imageGenerations;     public ImageResponse(List<ImageGeneration> generations) {        this(generations, new ImageResponseMetadata()); }       public ImageResponse(List<ImageGeneration> generations, ImageResponseMetadata imageResponseMetadata) {       this.imageResponseMetadata = imageResponseMetadata;      this.imageGenerations = List.copyOf(generations);    }   }

ImageResponse实现了ModelResponse接口，其getResult返回的是ImageGeneration类型

ImageGeneration

org/springframework/ai/image/ImageGeneration.java

public class ImageGeneration implements ModelResult<Image> {  private ImageGenerationMetadata imageGenerationMetadata;  private Image image;  public ImageGeneration(Image image) {     this.image = image;   }   public ImageGeneration(Image image, ImageGenerationMetadata imageGenerationMetadata) {        this.image = image;       this.imageGenerationMetadata = imageGenerationMetadata;   }   @Override    public Image getOutput() {        return this.image;  }   @Override    public ImageGenerationMetadata getMetadata() {        return this.imageGenerationMetadata;    }   @Override    public String toString() {        return "ImageGeneration{" + "imageGenerationMetadata=" + this.imageGenerationMetadata + ", image=" + this.image                + '}'; }}

ImageGeneration实现了ModelResult接口，其getOutput返回的是Image类型，其getMetadata返回ImageGenerationMetadata

Image

org/springframework/ai/image/Image.java

public class Image {       private String url;        private String b64Json;    }

Image定义了url与b64Json属性

ImageGenerationMetadata

org/springframework/ai/image/ImageGenerationMetadata.java

public interface ImageGenerationMetadata extends ResultMetadata {}

ImageGenerationMetadata目前仅仅是继承了ResultMetadata

示例

pom.xml

        <dependency>         <groupId>org.springframework.ai</groupId>         <artifactId>spring-ai-starter-model-zhipuai</artifactId>      </dependency>

配置

spring:  ai:    model:      image: zhipuai    zhipuai:      api-key: xxxx      base-url: https://open.bigmodel.cn/api/paas      images:        options:          model: cogview-3

代码

    @Test    public void testGenImage() {        ImagePrompt imagePrompt = new ImagePrompt("A light cream colored mini golden doodle",                ZhiPuAiImageOptions.builder()                        .model("cogview-3").build());        ImageResponse response = imageModel.call(imagePrompt);        log.info("resp:{}", JSON.toJSONString(response));    }

输出示例

resp:{"metadata":{"created":1743578654097,"empty":true,"rawMap":{}},"result":{"output":{"url":"https://sfile.chatglm.cn/testpath/xxx.png"}},"results":[{"$ref":"$.result"}]}

源码

ZhiPuAiImageAutoConfiguration

org/springframework/ai/model/zhipuai/autoconfigure/ZhiPuAiImageAutoConfiguration.java

@AutoConfiguration(after = { RestClientAutoConfiguration.class, SpringAiRetryAutoConfiguration.class })@ConditionalOnClass(ZhiPuAiApi.class)@ConditionalOnProperty(name = SpringAIModelProperties.IMAGE_MODEL, havingValue = SpringAIModels.ZHIPUAI,      matchIfMissing = true)@EnableConfigurationProperties({ ZhiPuAiConnectionProperties.class, ZhiPuAiImageProperties.class })public class ZhiPuAiImageAutoConfiguration {    @Bean    @ConditionalOnMissingBean    public ZhiPuAiImageModel zhiPuAiImageModel(ZhiPuAiConnectionProperties commonProperties,         ZhiPuAiImageProperties imageProperties, RestClient.Builder restClientBuilder, RetryTemplate retryTemplate,          ResponseErrorHandler responseErrorHandler) {        String apiKey = StringUtils.hasText(imageProperties.getApiKey()) ? imageProperties.getApiKey()             : commonProperties.getApiKey();     String baseUrl = StringUtils.hasText(imageProperties.getBaseUrl()) ? imageProperties.getBaseUrl()              : commonProperties.getBaseUrl();        Assert.hasText(apiKey, "ZhiPuAI API key must be set");     Assert.hasText(baseUrl, "ZhiPuAI base URL must be set");       var zhiPuAiImageApi = new ZhiPuAiImageApi(baseUrl, apiKey, restClientBuilder, responseErrorHandler);        return new ZhiPuAiImageModel(zhiPuAiImageApi, imageProperties.getOptions(), retryTemplate); }}

ZhiPuAiImageAutoConfiguration在spring.ai.model.image为zhipuai时启用，它会根据ZhiPuAiConnectionProperties及ZhiPuAiImageProperties自动配置ZhiPuAiImageModel

ZhiPuAiConnectionProperties

@ConfigurationProperties(ZhiPuAiConnectionProperties.CONFIG_PREFIX)public class ZhiPuAiConnectionProperties extends ZhiPuAiParentProperties {    public static final String CONFIG_PREFIX = "spring.ai.zhipuai";    public static final String DEFAULT_BASE_URL = "https://open.bigmodel.cn/api/paas"; public ZhiPuAiConnectionProperties() {     super.setBaseUrl(DEFAULT_BASE_URL); }}class ZhiPuAiParentProperties {  private String apiKey; private String baseUrl;    public String getApiKey() {     return this.apiKey;   }   public void setApiKey(String apiKey) {       this.apiKey = apiKey;   }   public String getBaseUrl() {        return this.baseUrl;  }   public void setBaseUrl(String baseUrl) {     this.baseUrl = baseUrl; }}

ZhiPuAiConnectionProperties继承了ZhiPuAiParentProperties属性，ZhiPuAiParentProperties定义了apiKey、baseUrl属性

ZhiPuAiImageProperties

org/springframework/ai/model/zhipuai/autoconfigure/ZhiPuAiImageProperties.java

@ConfigurationProperties(ZhiPuAiImageProperties.CONFIG_PREFIX)public class ZhiPuAiImageProperties extends ZhiPuAiParentProperties {  public static final String CONFIG_PREFIX = "spring.ai.zhipuai.image";       @NestedConfigurationProperty private ZhiPuAiImageOptions options = ZhiPuAiImageOptions.builder().build(); public ZhiPuAiImageOptions getOptions() {      return this.options;   }   public void setOptions(ZhiPuAiImageOptions options) {        this.options = options;  }}

ZhiPuAiImageProperties继承了ZhiPuAiParentProperties，它主要配置spring.ai.zhipuai.image.options

ZhiPuAiImageOptions

org/springframework/ai/zhipuai/ZhiPuAiImageOptions.java

@JsonInclude(JsonInclude.Include.NON_NULL)public class ZhiPuAiImageOptions implements ImageOptions {      @JsonProperty("model")  private String model = ZhiPuAiImageApi.DEFAULT_IMAGE_MODEL;      @JsonProperty("user_id")    private String user;   public static Builder builder() {     return new Builder();    }   @Override    @JsonIgnore  public Integer getN() {     return null;    }   @Override    public String getModel() {      return this.model;    }   public void setModel(String model) {     this.model = model; }   @Override    @JsonIgnore  public Integer getWidth() {     return null;    }   @Override    @JsonIgnore  public Integer getHeight() {        return null;    }   @Override    @JsonIgnore  public String getResponseFormat() {     return null;    }   @Override    @JsonIgnore  public String getStyle() {      return null;    }   public String getUser() {       return this.user; }   public void setUser(String user) {       this.user = user;   }   }

ZhiPuAiImageOptions实现了ImageOptions接口，主要支持了getModel、getUser方法

小结

Spring AI提供了ImageModel接口，它继承了Model接口，其call方法入参为ImagePrompt，返回ImageResponse。spring-ai-starter-model-openai提供了OpenAiImageModel，spring-ai-starter-model-azure-openai提供了AzureOpenAiImageModel，spring-ai-starter-model-qianfan提供了QianFanImageModel，spring-ai-starter-model-stability-ai提供了StabilityAiImageModel，spring-ai-starter-model-zhipuai提供了ZhiPuAiImageModel。

序