Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 88 additions & 0 deletions examples/auto-matplotlib/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# Configure Matplotlib with Optuna MCP

This example aims to show the combination of qualitative evaluation by an LLM agent and the black-box optimization by Optuna MCP.
As a simple example, we use Matplotlib.

## Overview

Simply put, an LLM agent qualitatively evaluates the legend position of images generated by `target_generator` in the self-implemented Matplotlib MCP ([matplotlib_server.py](./matplotlib_server.py)) based on the instruction detailed later and searches for the optimal legend position using Optuna based on the qualitative judge.
To keep the example simple, we consider only one parameter.

> [!NOTE]
> There is room for improvement in the qualitative evaluation scheme and the optimization itself, but such enhancement is not the scope of this example.

<table>
<caption>Default and optimized figures. <b>Left</b>: The default figure. The legend overlaps with the x-label, making it hard to recognize the label. <b>Right</b>: The optimized figure. The overlap is successfully removed by the optimization using Optuna, making the x-label visible.</caption>
<tr>
<td><img src="./images/first-plot.png" alt=""></td>
<td><img src="./images/best-plot.png" alt=""></td>
</tr>
</table>

## Workflow

The example works as follows:
1. Suggest the position controlling parameter, i.e., `bbox_to_anchor_y`, using Optuna MCP.
2. Generate an image file using Matplotlib MCP.
3. Upload the image file manually and let the LLM agent qualitatively evaluate the image file based on the instruction described in `Prompt`.
4. Report the qualitative score using Optuna MCP.
5. Repeat 1. -- 4.

## How to Reproduce Demo

### Setups

Please first install the dependencies for this example:

```shell
$ uv pip install matplotlib numpy "mcp[cli]>=1.5.0"
```

The directory structure of this example should look like:

```shell
$ tree ./auto-matplotlib
auto-matplotlib/
└── matplotlib_server.py
```

To enable the MCP server for Matplotlib in Claude Desktop, go to `Claude > Settings > Developer > Edit Config > claude_desktop_config.json` and add the following inside `mcpServers`:


```json
"AutoMatplotlib": {
"command": "/path/to/uv",
"args": [
"--directory",
"/path/to/auto-matplotlib",
"run",
"matplotlib_server.py"
]
}
```

### Prompt

Use the following prompt to iterate the routine described in `Workflow`:

```txt
Create a study named `auto-matplotlib-demo` to maximize the qualitative score of plot figures.

Our task is to optimize the legend position in the figure.
The legend MUST be located below the `xlabel`.
We will optimize the position by controlling `bbox_to_anchor_y` in range of `(-0.1, 0.1)`.

After that, repeat the following five times:
1. Sample a trial from Optuna MCP.
2. Generate a plot given the trial using `target_generator`.
3. Ask me to upload the generated image.
4. Look at the generated image and evaluate `bbox_to_anchor_y` qualitatively from 1 (worst) to 9 (best).
5. Tell the qualitative assessment to Optuna MCP.

Note that `bbox_to_anchor_y` is not good if:
- the legend hides the `xlabel` of the figure,
- the legend hides the main plots or is located above `xlabel`,
- there is an insufficient margin between the upper part of the legend and the lower part of the `xlabel`.

Please evaluate each criterion qualitatively from 1 (worst) to 3 (best).
```
Binary file added examples/auto-matplotlib/images/best-plot.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added examples/auto-matplotlib/images/first-plot.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
59 changes: 59 additions & 0 deletions examples/auto-matplotlib/matplotlib_server.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
from __future__ import annotations

import os

import matplotlib.pyplot as plt
from mcp.server.fastmcp import FastMCP
import numpy as np


mcp = FastMCP("AutoMatplotlib")
plt.rcParams["font.family"] = "Times New Roman"
plt.rcParams["font.size"] = 16
rng = np.random.RandomState(42)
X1 = rng.random(size=(3, 100, 40)) * 10 - 5
X2 = np.clip(rng.normal(size=(3, 100, 40)) * 2.5, -5, 5)
os.makedirs("figs/", exist_ok=True)


@mcp.tool()
def target_generator(trial_number: int, bbox_to_anchor_y: float) -> str:
"""
Generate a plot figure based on the trial suggested by Optuna MCP.

Args:
trial_number: The trial number.
bbox_to_anchor_y:
The `bbox_to_anchor_y` stored in `params` of a `trial` suggested by Optuna MCP.
"""
fig, axes = plt.subplots(ncols=2, nrows=2, figsize=(10, 5), sharex=True)
dx = np.arange(100) + 1
for i, d in enumerate([5, 10, 20, 40]):
ax = axes[i // 2][i % 2]

def _subplot(ax: plt.Axes, values: list[list[float]]) -> plt.Line2D:
cum_values = np.minimum.accumulate(values, axis=-1)
mean = np.mean(cum_values, axis=0)
stderr = np.std(cum_values, axis=0) / np.sqrt(len(values))
(line,) = ax.plot(dx, mean)
ax.fill_between(dx, mean - stderr, mean + stderr, alpha=0.2)
return line

lines = []
ax.set_title(f"{d}D")
lines.append(_subplot(ax, np.sum((X1[..., :d] - 2) ** 2, axis=-1)))
lines.append(_subplot(ax, np.sum((X2[..., :d] - 2) ** 2, axis=-1)))

fig.supxlabel("Number of Trials")
fig.supylabel("Objective Values")
labels = ["Uniform", "Gaussian"]
loc = "lower center"
bbox_to_anchor = (0.5, bbox_to_anchor_y)
fig.legend(handles=lines, labels=labels, loc=loc, ncols=2, bbox_to_anchor=bbox_to_anchor)
fig_path = f"figs/fig{trial_number}.png"
plt.savefig(fig_path, bbox_inches="tight")
return f"{fig_path} generated for Trial {trial_number} with {bbox_to_anchor_y=}"


if __name__ == "__main__":
mcp.run()