Merge pull request #1 from optuna/add-simple-auto-matplotlib-example

Add a simple matplotlib example

Author: HideakiImamura

examples/auto-matplotlib/README.md

@@ -0,0 +1,88 @@
+# Configure Matplotlib with Optuna MCP
+
+This example aims to show the combination of qualitative evaluation by an LLM agent and the black-box optimization by Optuna MCP.
+As a simple example, we use Matplotlib.
+
+## Overview
+
+Simply put, an LLM agent qualitatively evaluates the legend position of images generated by `target_generator` in the self-implemented Matplotlib MCP ([matplotlib_server.py](./matplotlib_server.py)) based on the instruction detailed later and searches for the optimal legend position using Optuna based on the qualitative judge.
+To keep the example simple, we consider only one parameter.
+
+> [!NOTE]
+> There is room for improvement in the qualitative evaluation scheme and the optimization itself, but such enhancement is not the scope of this example. 
+
+<table>
+    <caption>Default and optimized figures. <b>Left</b>: The default figure. The legend overlaps with the x-label, making it hard to recognize the label. <b>Right</b>: The optimized figure. The overlap is successfully removed by the optimization using Optuna, making the x-label visible.</caption>
+    <tr>
+        <td><img src="./images/first-plot.png" alt=""></td>
+        <td><img src="./images/best-plot.png" alt=""></td>
+    </tr>
+</table>
+
+## Workflow
+
+The example works as follows:
+1. Suggest the position controlling parameter, i.e., `bbox_to_anchor_y`, using Optuna MCP.
+2. Generate an image file using Matplotlib MCP.
+3. Upload the image file manually and let the LLM agent qualitatively evaluate the image file based on the instruction described in `Prompt`.
+4. Report the qualitative score using Optuna MCP.
+5. Repeat 1. -- 4.
+
+## How to Reproduce Demo
+
+### Setups
+
+Please first install the dependencies for this example:
+
+```shell
+$ uv pip install matplotlib numpy "mcp[cli]>=1.5.0"
+```
+
+The directory structure of this example should look like:
+
+```shell
+$ tree ./auto-matplotlib
+auto-matplotlib/
+└── matplotlib_server.py
+```
+
+To enable the MCP server for Matplotlib in Claude Desktop, go to `Claude > Settings > Developer > Edit Config > claude_desktop_config.json` and add the following inside `mcpServers`:
+
+
+```json
+"AutoMatplotlib": {
+    "command": "/path/to/uv",
+    "args": [
+        "--directory",
+        "/path/to/auto-matplotlib",
+        "run",
+        "matplotlib_server.py"
+    ]
+}
+```
+
+### Prompt
+
+Use the following prompt to iterate the routine described in `Workflow`:
+
+```txt
+Create a study named `auto-matplotlib-demo` to maximize the qualitative score of plot figures.
+
+Our task is to optimize the legend position in the figure.
+The legend MUST be located below the `xlabel`.
+We will optimize the position by controlling `bbox_to_anchor_y` in range of `(-0.1, 0.1)`.
+
+After that, repeat the following five times:
+1. Sample a trial from Optuna MCP.
+2. Generate a plot given the trial using `target_generator`.
+3. Ask me to upload the generated image.
+4. Look at the generated image and evaluate `bbox_to_anchor_y` qualitatively from 1 (worst) to 9 (best).
+5. Tell the qualitative assessment to Optuna MCP.
+
+Note that `bbox_to_anchor_y` is not good if:
+- the legend hides the `xlabel` of the figure,
+- the legend hides the main plots or is located above `xlabel`,
+- there is an insufficient margin between the upper part of the legend and the lower part of the `xlabel`.
+
+Please evaluate each criterion qualitatively from 1 (worst) to 3 (best).
+```

examples/auto-matplotlib/images/best-plot.png

@@ -0,0 +1,59 @@
+from __future__ import annotations
+
+import os
+
+import matplotlib.pyplot as plt
+from mcp.server.fastmcp import FastMCP
+import numpy as np
+
+
+mcp = FastMCP("AutoMatplotlib")
+plt.rcParams["font.family"] = "Times New Roman"
+plt.rcParams["font.size"] = 16
+rng = np.random.RandomState(42)
+X1 = rng.random(size=(3, 100, 40)) * 10 - 5
+X2 = np.clip(rng.normal(size=(3, 100, 40)) * 2.5, -5, 5)
+os.makedirs("figs/", exist_ok=True)
+
+
+@mcp.tool()
+def target_generator(trial_number: int, bbox_to_anchor_y: float) -> str:
+    """
+    Generate a plot figure based on the trial suggested by Optuna MCP.
+
+    Args:
+        trial_number: The trial number.
+        bbox_to_anchor_y:
+            The `bbox_to_anchor_y` stored in `params` of a `trial` suggested by Optuna MCP.
+    """
+    fig, axes = plt.subplots(ncols=2, nrows=2, figsize=(10, 5), sharex=True)
+    dx = np.arange(100) + 1
+    for i, d in enumerate([5, 10, 20, 40]):
+        ax = axes[i // 2][i % 2]
+
+        def _subplot(ax: plt.Axes, values: list[list[float]]) -> plt.Line2D:
+            cum_values = np.minimum.accumulate(values, axis=-1)
+            mean = np.mean(cum_values, axis=0)
+            stderr = np.std(cum_values, axis=0) / np.sqrt(len(values))
+            (line,) = ax.plot(dx, mean)
+            ax.fill_between(dx, mean - stderr, mean + stderr, alpha=0.2)
+            return line
+
+        lines = []
+        ax.set_title(f"{d}D")
+        lines.append(_subplot(ax, np.sum((X1[..., :d] - 2) ** 2, axis=-1)))
+        lines.append(_subplot(ax, np.sum((X2[..., :d] - 2) ** 2, axis=-1)))
+
+    fig.supxlabel("Number of Trials")
+    fig.supylabel("Objective Values")
+    labels = ["Uniform", "Gaussian"]
+    loc = "lower center"
+    bbox_to_anchor = (0.5, bbox_to_anchor_y)
+    fig.legend(handles=lines, labels=labels, loc=loc, ncols=2, bbox_to_anchor=bbox_to_anchor)
+    fig_path = f"figs/fig{trial_number}.png"
+    plt.savefig(fig_path, bbox_inches="tight")
+    return f"{fig_path} generated for Trial {trial_number} with {bbox_to_anchor_y=}"
+
+
+if __name__ == "__main__":
+    mcp.run()