feat(vertexai): add Virtual Try-On model support

[CorieW]

Adds Virtual Try On model to VertexAI plugin.

Testing

The following puts a shirt (image A) onto a person (image B):

Image A

Image B

Result

Params

[gemini-code-assist]

Code Review

This pull request introduces support for Virtual Try-On models in the Vertex AI plugin, including the implementation of model options, request/response mapping, and integration into the plugin's model resolution and discovery logic. Feedback focuses on improving integration with the Genkit framework by defining a JSON schema for VirtualTryOnOptions and ensuring it is passed as customOptions in model references, metadata, and constructors. Additionally, it was suggested to disable systemRole support as the current implementation only processes media parts and does not handle text instructions.

I am having trouble creating individual review comments. Click here to see my feedback.

packages/genkit_vertexai/lib/genkit_vertexai.dart (69)

The virtualTryOn method should pass VirtualTryOnOptions.$schema as customOptions to modelRef. This ensures that the model configuration is properly typed and validated by the Genkit framework, especially when used through the CLI or developer UI.

    return modelRef(
      'vertexai/$name',
      config: config,
      customOptions: VirtualTryOnOptions.$schema,
    );

packages/genkit_vertexai/lib/src/virtual_try_on.dart (15-18)

Add the schemantic import to support defining the JSON schema for VirtualTryOnOptions.

import 'dart:convert';

import 'package:genkit/plugin.dart';
import 'package:genkit_google_genai/common.dart';
import 'package:schemantic/schemantic.dart';

packages/genkit_vertexai/lib/src/virtual_try_on.dart (28)

The Virtual Try-On model does not appear to support system instructions or text prompts based on the implementation of _toVirtualTryOnPredictRequest, which only extracts media parts. Setting systemRole: true might be misleading; it should likely be false unless the model actually processes text instructions.

    'systemRole': false,

packages/genkit_vertexai/lib/src/virtual_try_on.dart (33-44)

To integrate properly with Genkit's schema validation and UI, VirtualTryOnOptions should provide a JSON schema via a static $schema field.

class VirtualTryOnOptions {
  VirtualTryOnOptions({
    this.outputOptions,
    this.sampleCount,
    this.storageUri,
    this.seed,
    this.baseSteps,
    this.safetySetting,
    this.personGeneration,
    this.addWatermark,
    this.enhancePrompt,
  });

  static final $schema = SchemanticType.object(
    properties: {
      'outputOptions': SchemanticType.any(),
      'sampleCount': SchemanticType.integer(),
      'storageUri': SchemanticType.string(),
      'seed': SchemanticType.integer(),
      'baseSteps': SchemanticType.integer(),
      'safetySetting': SchemanticType.string(),
      'personGeneration': SchemanticType.string(),
      'addWatermark': SchemanticType.boolean(),
      'enhancePrompt': SchemanticType.boolean(),
    },
  );
}

packages/genkit_vertexai/lib/src/virtual_try_on.dart (75-80)

Include customOptions in the model metadata to ensure the framework is aware of the configuration schema when listing models.

ActionMetadata virtualTryOnModelMetadata(String pluginName, String modelName) {
  return modelMetadata(
    '$pluginName/$modelName',
    modelInfo: virtualTryOnModelInfo,
    customOptions: VirtualTryOnOptions.$schema,
  );
}

packages/genkit_vertexai/lib/src/virtual_try_on.dart (92)

The Model constructor should include customOptions so that the framework can validate and parse the request configuration against the VirtualTryOnOptions schema.

    name: '$pluginName/$modelName',
    customOptions: VirtualTryOnOptions.$schema,
    metadata: {'model': virtualTryOnModelInfo.toJson()},

[CorieW]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces support for Virtual Try-On models within the Genkit Vertex AI plugin. Key changes include the addition of the virtualTryOn method to the VertexAiPluginHandle, implementation of the model logic in a new virtual_try_on.dart file, and the inclusion of comprehensive unit tests. Feedback was provided to improve the validation of image URIs by explicitly checking for unsupported URI schemes to prevent invalid API requests.

[CorieW]

• Test changes