Decouple Planning & Execution via Interface-Based Plugin + Workaround

[bdetry]

Context : Exploring different solutions on multi-agent (MAS) sample projects.

Problem : I realized that there is no native way to create a plan and execute it later in the process. Imagine a scenario where one agent creates a plan for other agents across different geolocations (or any other distributed system).

Potential solution : Add possibility to depend on abstraction instead of implementation for the plugins' setup.

Current way to implement tool calling :

From docs

Kernel kernel = builder.Build();
kernel.Plugins.AddFromType<LightsPlugin>("Lights");

How it looks like with my workaround :

Kernel kernel = builder.Build();
kernel.ImportPluginFromInterface(typeof(IFlightSearchPlugin) , "FlightSearch");

The extension method (not refactored, inspired from the actual source code of SK) is at the bottom of this note, please note the use of a DummyExecutionLogicAsync

IFlightSearchPlugin.cs

public interface IFlightSearchPlugin
{
    [KernelFunction]
    [Description("Search for available flights between cities based on specified criteria.")]
    [return: Description("A result object containing a list of flight options and their details.")]
    Task<FlightSearchResults> SearchFlightsAsync(
        [Description("Departure city or airport code (e.g., 'NYC', 'CDG').")]
        string origin,
        [Description("Arrival city or airport code (e.g., 'PAR', 'LHR').")]
        string destination,
        [Description("Desired departure date in YYYY-MM-DD format.")]
        string date,
        CancellationToken cancellationToken = default);
}

Getting the plan (without the tools being called) :

 ChatMessageContent result = await _chatCompletionService.GetChatMessageContentAsync(
     _chatHistory,
     _modelSettings,
     _kernel.Kernel,
     cancellationToken
 );

 string llmResponseText = result.Content ?? string.Empty;

 if (result is OpenAIChatMessageContent openAiChatMessageContent)
 {
     var toolCalls = openAiChatMessageContent.ToolCalls;
     _tasks.AddRange(toolCalls.Select(toolCall =>
             new AgentTask(toolCall.FunctionName,
                           toolCall.FunctionArguments.ToString(),
                           toolCall.Id,
                            AgentTaskStatus.InProgress)));


 }

Example :

Query : I need to plan a trip to Montpellier from Chicago In the next hour this is urgent.

KernelInterfaceExtensions.cs

public static class KernelInterfaceExtensions
{
    public static Kernel ImportPluginFromInterface(
        this Kernel kernel,
        Type interfaceType,
        string pluginName,
        ILoggerFactory? loggerFactory = null)
    {
        var logger = loggerFactory?.CreateLogger(typeof(KernelInterfaceExtensions)) ?? NullLogger.Instance;

        if (!interfaceType.IsInterface)
        {
            throw new ArgumentException($"{interfaceType.FullName} is not an interface.", nameof(interfaceType));
        }

        pluginName ??= interfaceType.Name.StartsWith("I") && interfaceType.Name.Length > 1
            ? interfaceType.Name.Substring(1).Replace("Plugin", "")
            : interfaceType.Name.Replace("Plugin", "");

        logger.LogInformation("Importing functions from interface {InterfaceType} into plugin '{PluginName}'", interfaceType.FullName, pluginName);

        List<KernelFunction> kernelFunctions = new List<KernelFunction>();

        foreach (var methodInfo in interfaceType.GetMethods(BindingFlags.Public | BindingFlags.Instance | BindingFlags.DeclaredOnly))
        {
            HandleInterfaceMethod(interfaceType, pluginName, loggerFactory, logger, kernelFunctions, methodInfo);
        }

        var plugin = KernelPluginFactory.CreateFromFunctions(pluginName , kernelFunctions);

        kernel.Plugins.Add(plugin);

        return kernel;
    }

    private static void HandleInterfaceMethod(Type interfaceType, string pluginName, ILoggerFactory? loggerFactory, ILogger logger, List<KernelFunction> kernelFunctions, MethodInfo methodInfo)
    {
        var kernelFunctionAttribute = methodInfo.GetCustomAttribute<KernelFunctionAttribute>();
        if (kernelFunctionAttribute == null)
        {
            logger.LogTrace("Method {MethodName} on {InterfaceType} is missing KernelFunctionAttribute.", methodInfo.Name, interfaceType.Name);
            throw new Exception($"Method {methodInfo.Name} on {interfaceType.Name} is missing KernelFunctionAttribute.");
        }

        logger.LogDebug("Processing method {InterfaceType}.{MethodName} for plugin '{PluginName}'", interfaceType.Name, methodInfo.Name, pluginName);

        var descriptionAttribute = methodInfo.GetCustomAttribute<DescriptionAttribute>();
        var returnDescriptionAttribute = methodInfo.ReturnParameter?.GetCustomAttribute<DescriptionAttribute>();

        var parametersMetadata = new List<KernelParameterMetadata>();
        foreach (var paramInfo in methodInfo.GetParameters())
        {
            if (paramInfo.ParameterType == typeof(CancellationToken))
            {
                continue;
            }

            var paramDescAttr = paramInfo.GetCustomAttribute<DescriptionAttribute>();
            parametersMetadata.Add(new KernelParameterMetadata(paramInfo.Name!)
            {
                Description = paramDescAttr?.Description,
                ParameterType = paramInfo.ParameterType,
                IsRequired = !paramInfo.IsOptional,
                DefaultValue = paramInfo.IsOptional ? paramInfo.DefaultValue : null
            });
        }

        Type metadataReturnType = methodInfo.ReturnType;
        if (methodInfo.ReturnType.IsGenericType && methodInfo.ReturnType.GetGenericTypeDefinition() == typeof(Task<>))
        {
            metadataReturnType = methodInfo.ReturnType.GetGenericArguments()[0];
        }
        else if (methodInfo.ReturnType == typeof(Task))
        {
            metadataReturnType = typeof(void);
        }

        var returnParameterMetadata = new KernelReturnParameterMetadata
        {
            Description = returnDescriptionAttribute?.Description,
            ParameterType = metadataReturnType
        };

        Task<object?> DummyExecutionLogicAsync(KernelArguments args, CancellationToken ct)
        {
            if (metadataReturnType == typeof(void))
            {
                return Task.FromResult<object?>(null);
            }

            object? defaultValue = metadataReturnType.IsValueType ? Activator.CreateInstance(metadataReturnType) : null;
            return Task.FromResult(defaultValue);
        }

        var kernelFunction = KernelFunctionFactory.CreateFromMethod(DummyExecutionLogicAsync,
            new KernelFunctionFromMethodOptions()
            {
                Description = descriptionAttribute?.Description,
                FunctionName = methodInfo.Name,
                ReturnParameter = returnParameterMetadata,
                LoggerFactory = loggerFactory,
                Parameters = parametersMetadata,
            });


        kernelFunctions.Add(kernelFunction);


        logger.LogInformation("Successfully imported metadata for {PluginName}.{FunctionName} from interface {InterfaceType}",
            pluginName, methodInfo.Name, interfaceType.FullName);
    }
}

[moonbox3]

We have some new orchestration patterns. Please have a look at this one, which is related to the "plan and execute" notion: https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/GettingStartedWithAgents/Orchestration/Step05_Magentic.cs