feat: replace verbose tdml docstrings with compact curated summaries (#309)

* feat: replace verbose tdml docstrings with compact curated summaries (#265)

Convert TD_ANALYTIC_FUNCS from list to dict[str, str] mapping each teradataml
function to a one-line curated summary. Replace convert_tdml_docstring_to_mcp_docstring()
with build_tdml_tool_docstring() that generates compact parameter descriptions
directly from live function metadata, significantly reducing MCP tool description size.
Adds scripts/seed_tdml_summaries.py utility for regenerating summaries from a live DB.

* style: ruff format utils/__init__.py

Author: dtehan-td

CLAUDE.md

@@ -66,6 +66,15 @@ Tools are organized into domain modules under `src/teradata_mcp_server/tools/`:
 
 Profiles (defined in `config/profiles.yml`) control which modules load. The `module_loader.py` uses regex pattern matching against tool name prefixes to determine which modules to import. Available profiles: `all`, `dba`, `dataScientist`, `eda`, `bar`, `llmUser`, `tester`.
 
+### teradataml Analytic Function Tools (`tdml_*`)
+
+The ~89 `tdml_*` tools (e.g., `tdml_KMeans`, `tdml_XGBoost`) are registered dynamically in `app.py`, separate from the `handle_*` module pattern. Key files:
+
+- **`tools/constants.py`** — `TD_ANALYTIC_FUNCS`: a `dict[str, str]` mapping teradataml function name → curated one-line summary. This is the authoritative list of which functions to register. To add a new function, add one entry here.
+- **`tools/utils/__init__.py`** — `build_tdml_tool_docstring(summary, func_metadata, partition_order_cols)`: builds the compact MCP tool description at registration time by reading parameter names, descriptions, and types from the live teradataml JSON store.
+
+Tools are only registered when `enable_analytic_functions` is true, teradataml is installed, and a database connection is available. Functions missing from the connected system are skipped with a warning.
+
 ### Configuration System
 
 Layered config loading (`config_loader.py`):

docs/developer_guide/DEVELOPER_GUIDE.md

@@ -373,6 +373,19 @@ This section explains how the pieces fit together at runtime.
   - The wrapper delegates execution to `execute_db_tool` which:
     - Injects a DB connection (SQLAlchemy `Connection` preferred)
     - Sets QueryBand based on request context (`tools/utils/queryband.py`)
+- Dynamically registers `tdml_*` analytic function tools when teradataml is installed and a database connection is available (see below).
+
+### Dynamic teradataml Analytic Function Registration
+
+The ~89 `tdml_*` tools (e.g., `tdml_KMeans`, `tdml_XGBoost`) are not defined as `handle_*` functions. Instead, `app.py` generates and registers them at startup when `enable_analytic_functions` is true:
+
+1. **`tools/constants.py`** — `TD_ANALYTIC_FUNCS` is a `dict[str, str]` mapping each teradataml function name to a curated one-line summary (e.g., `"KMeans": "Groups observations into k clusters..."`). This dict is the authoritative list of which functions to register.
+
+2. **`tools/utils/__init__.py`** — `build_tdml_tool_docstring(summary, func_metadata, partition_order_cols)` builds the compact MCP tool description at registration time. It reads parameter names, descriptions, Required/Optional, and types directly from `func_metadata.arguments` (teradataml's live JSON store, populated from the database), combining them with the curated summary.
+
+3. **`app.py`** — Iterates `TD_ANALYTIC_FUNCS.items()`, queries the live JSON store for each function's metadata, generates a Python function string via `exec()`, and registers it with `mcp.tool()`. If a function from the dict is not present in the connected database's function list, it is skipped with a warning.
+
+**To add a new analytic function:** add one entry to `TD_ANALYTIC_FUNCS` in `tools/constants.py` with a concise one-line description. No other code changes are needed.
 
 ## Project Layout
 
@@ -394,11 +407,12 @@ teradata-mcp-server/
    │  └─ profiles.yml
    └─ tools/
       ├─ __init__.py               # Lazy module loader + explicit exports (e.g., TDConn)
+      ├─ constants.py              # TD_ANALYTIC_FUNCS dict: teradataml function name → one-line summary
       ├─ module_loader.py          # Profiles → load only needed tool modules (+ YAMLs)
       ├─ td_connect.py             # SQLAlchemy connection + auth validation helpers
       ├─ utils/
-      │  ├─ __init__.py            # JSON helpers, auth header parsing, exports queryband
-+     │  └─ queryband.py           # Build Teradata QueryBand from request context
+      │  ├─ __init__.py            # JSON helpers, auth header parsing, tdml docstring builder
+      │  └─ queryband.py           # Build Teradata QueryBand from request context
       ├─ base/ ...                 # Tool groups (base, dba, sec, qlty, rag, fs, tdvs, ...)
       └─ fs/...                    # Optional extras; imported only if profile enables them
 ```

docs/developer_guide/HOW_TO_ADD_YOUR_FUNCTION.md

@@ -126,4 +126,33 @@ Use MCP Inspector or your client (Claude Desktop) to call the tool once it’s e
 | MCP wrapper (auto)          | Auto-generated MCP wrapper around your handler (built at startup).           |
 | `execute_db_tool` (internal)  | Central adapter: sets QueryBand, handles errors/formatting, reconnects.    |
 
-Let me know if you'd like this as a template or reusable decorator for many functions. 
+---
+
+## ➕ Adding a teradataml Analytic Function (`tdml_*`)
+
+The `tdml_*` tools (e.g., `tdml_KMeans`, `tdml_XGBoost`) are registered dynamically from the teradataml library. They do **not** follow the `handle_*` pattern — instead, they are driven by a curated dictionary.
+
+### How it works
+
+1. `tools/constants.py` contains `TD_ANALYTIC_FUNCS`, a `dict[str, str]` mapping each teradataml function name to a curated one-line summary.
+2. At startup, `app.py` iterates this dict, queries the live teradataml JSON store for each function's parameter metadata, and generates + registers a `tdml_<FunctionName>` MCP tool automatically.
+3. `build_tdml_tool_docstring()` in `tools/utils/__init__.py` assembles the compact description (summary + one line per parameter with Required/Optional and types).
+
+### Steps to add a new function
+
+1. Open `src/teradata_mcp_server/tools/constants.py`.
+2. Add one entry to `TD_ANALYTIC_FUNCS`:
+
+```python
+TD_ANALYTIC_FUNCS = {
+    ...
+    "MyNewFunction": "One-sentence description of what this function does.",
+}
+```
+
+That's it — no other code changes are needed. The server will register `tdml_MyNewFunction` automatically on next startup, provided the function exists in the connected database's teradataml version.
+
+### Notes
+- The summary should be one sentence, no longer than ~120 characters.
+- If the function is not present in the connected database, it is skipped with a warning — no error.
+- The `fs` extra (`uv sync --extra fs`) must be installed for any `tdml_*` tools to register.

scripts/seed_tdml_summaries.py

@@ -0,0 +1,198 @@
+"""
+Seed script: connects to Teradata, extracts one-line summaries from teradataml
+__init__.__doc__ for all TD_ANALYTIC_FUNCS, then prints the new dict[str, str]
+block ready to paste into constants.py.
+
+Requires DATABASE_URI env var:
+  export DATABASE_URI="teradata://user:pass@host:1025/db"
+  uv run python scripts/seed_tdml_summaries.py
+"""
+
+import os
+import re
+import textwrap
+import warnings
+
+warnings.filterwarnings("ignore")
+
+import teradataml as tdml  # noqa: E402
+
+# Connect so that teradataml populates __init__.__doc__ on each class
+_uri = os.environ.get("DATABASE_URI", "")
+if _uri:
+    _m = re.match(r"teradata://([^:]+):([^@]+)@([^:]+):(\d+)/(.+)", _uri)
+    if _m:
+        tdml.create_context(
+            host=_m.group(3),
+            username=_m.group(1),
+            password=_m.group(2),
+            database=_m.group(5),
+        )
+    else:
+        raise ValueError(f"Cannot parse DATABASE_URI: {_uri}")
+else:
+    raise EnvironmentError("DATABASE_URI not set — docstrings require a live connection")
+
+FUNCS = [
+    "ANOVA",
+    "Attribution",
+    "Antiselect",
+    "Apriori",
+    "BincodeFit",
+    "BincodeTransform",
+    "CFilter",
+    "CategoricalSummary",
+    "ChiSq",
+    "ClassificationEvaluator",
+    "ColumnSummary",
+    "ColumnTransformer",
+    "ConvertTo",
+    "DecisionForest",
+    "FTest",
+    "FillRowId",
+    "Fit",
+    "GetFutileColumns",
+    "GetRowsWithMissingValues",
+    "GetRowsWithoutMissingValues",
+    "GLM",
+    "GLMPerSegment",
+    "Histogram",
+    "KMeans",
+    "KMeansPredict",
+    "KNN",
+    "MovingAverage",
+    "NERExtractor",
+    "NGramSplitter",
+    "NaiveBayesTextClassifierPredict",
+    "NaiveBayesTextClassifierTrainer",
+    "NonLinearCombineFit",
+    "NonLinearCombineTransform",
+    "NumApply",
+    "NPath",
+    "OneClassSVM",
+    "OneClassSVMPredict",
+    "OneHotEncodingFit",
+    "OneHotEncodingTransform",
+    "OrdinalEncodingFit",
+    "OrdinalEncodingTransform",
+    "OutlierFilterFit",
+    "OutlierFilterTransform",
+    "Pack",
+    "PolynomialFeaturesFit",
+    "PolynomialFeaturesTransform",
+    "Pivoting",
+    "QQNorm",
+    "ROC",
+    "RandomProjectionFit",
+    "RandomProjectionMinComponents",
+    "RandomProjectionTransform",
+    "RegressionEvaluator",
+    "RoundColumns",
+    "RowNormalizeFit",
+    "RowNormalizeTransform",
+    "SMOTE",
+    "SVM",
+    "SVMPredict",
+    "ScaleFit",
+    "ScaleTransform",
+    "Sessionize",
+    "SentimentExtractor",
+    "Shap",
+    "Silhouette",
+    "SimpleImputeFit",
+    "SimpleImputeTransform",
+    "StrApply",
+    "StringSimilarity",
+    "TDDecisionForestPredict",
+    "TDGLMPredict",
+    "TDNaiveBayesPredict",
+    "TFIDF",
+    "TargetEncodingFit",
+    "TargetEncodingTransform",
+    "TextMorph",
+    "TextParser",
+    "TrainTestSplit",
+    "Transform",
+    "UnivariateStatistics",
+    "Unpack",
+    "Unpivoting",
+    "VectorDistance",
+    "WhichMax",
+    "WhichMin",
+    "WordEmbeddings",
+    "XGBoost",
+    "XGBoostPredict",
+    "ZTest",
+]
+
+
+def extract_summary(func_name: str) -> str:
+    """Pull the first meaningful sentence from the teradataml __init__ docstring."""
+    func_obj = getattr(tdml, func_name, None)
+    if func_obj is None:
+        return f"Teradata ML analytic function {func_name}."
+
+    raw = getattr(func_obj.__init__, "__doc__", None) or ""
+    # Dedent and strip leading blank lines
+    raw = textwrap.dedent(raw).strip()
+
+    # The teradataml pattern is:
+    #   DESCRIPTION:
+    #       <summary text, may span multiple lines>
+    #
+    #   PARAMETERS:
+    # Try to grab the DESCRIPTION block first.
+    desc_match = re.search(r"DESCRIPTION\s*:\s*\n(.*?)(?:\n\s*\n|\n\s*PARAMETERS\s*:)", raw, re.DOTALL)
+    if desc_match:
+        block = desc_match.group(1)
+    else:
+        # Fallback: take the first non-empty paragraph
+        block = raw.split("\n\n")[0]
+
+    # Collapse internal whitespace / newlines into a single line
+    block = re.sub(r"\s+", " ", block).strip()
+
+    # Replace teradataml-specific terminology
+    block = block.replace("teradataml DataFrame", "table name")
+    block = block.replace("DataFrame", "table name")
+
+    # Truncate at the first sentence boundary (period followed by space or end)
+    # Keep the trailing period.
+    sent_match = re.search(r"^(.*?\.)\s", block)
+    if sent_match:
+        summary = sent_match.group(1)
+    else:
+        # No sentence boundary — use the whole block but cap length
+        summary = block[:200].rstrip()
+        if not summary.endswith("."):
+            summary += "."
+
+    return summary
+
+
+def main():
+    results: list[tuple[str, str]] = []
+    missing: list[str] = []
+
+    for name in FUNCS:
+        summary = extract_summary(name)
+        results.append((name, summary))
+        if "analytic function" in summary and name in summary:
+            missing.append(name)
+
+    # Print the dict literal ready to paste into constants.py
+    print("TD_ANALYTIC_FUNCS = {")
+    for name, summary in results:
+        # Escape any quotes inside the summary
+        safe = summary.replace('"', '\\"')
+        print(f'    "{name}": "{safe}",')
+    print("}")
+
+    if missing:
+        print(f"\n# WARNING: {len(missing)} functions had no extractable docstring — fallback used:")
+        for m in missing:
+            print(f"#   {m}")
+
+
+if __name__ == "__main__":
+    main()

src/teradata_mcp_server/app.py

@@ -38,7 +38,7 @@
 from teradata_mcp_server.tools import ContextCatalog
 from teradata_mcp_server.tools.graph.graph_edge_contract import GRAPH_EDGE_CONTRACT
 from teradata_mcp_server.tools.utils import (
-    convert_tdml_docstring_to_mcp_docstring,
+    build_tdml_tool_docstring,
     execute_analytic_function,
     get_anlytic_function_signature,
     get_dynamic_function_definition,
@@ -587,7 +587,7 @@ def execute_tool(
     if enable_analytic_functions:
         tdml_processed_funcs = set(tdml.analytics.json_parser.json_store._JsonStore._get_function_list()[0].keys())
 
-        for func_name in funcs:
+        for func_name, summary in funcs.items():
             # Before adding the function, check if function is existed or not.
             # Connection is not mandatory for MCP server. If connection is not there, then
             # functions can not be added.
@@ -596,30 +596,28 @@ def execute_tool(
                 continue
 
             func_metadata = tdml.analytics.json_parser.json_store._JsonStore.get_function_metadata(func_name)
-            func_obj = getattr(tdml, func_name, None)
             func_params = func_metadata.function_params
 
             inp_data = [t.get_lang_name() for t in func_metadata.input_tables]
             # Add partition_by parameters for func parameters.
-            additional_args_docs = []
+            partition_order_cols = []
             for table in inp_data:
                 func_params[f"{table}_partition_column"] = None
                 func_params[f"{table}_order_column"] = None
-                additional_args_docs.append(get_partition_col_order_col_doc_string(table))
+                partition_order_cols.append(get_partition_col_order_col_doc_string(table))
 
             # Generate function argument string.
             func_args_str = get_anlytic_function_signature(func_params)
 
             full_func_name = "tdml_" + func_name
-            init_doc = func_obj.__init__.__doc__  # type: ignore[misc]
             func_str = get_dynamic_function_definition().format(
                 analytic_function=full_func_name,
-                doc_string=init_doc,
+                doc_string=summary,
                 func_args_str=func_args_str,
                 tables_to_df=json.dumps(inp_data),
             )
 
-            doc_string = convert_tdml_docstring_to_mcp_docstring(init_doc, additional_args_docs)
+            doc_string = build_tdml_tool_docstring(summary, func_metadata, partition_order_cols)
 
             # Execute the generated function definition in the global scope.
             # Global scope will have all other functions. So reference to other functions will work.