Merge origin/main into db-tool-registry

Resolves conflicts preserving both feature sets:
- Progressive disclosure of tools (this branch)
- Dynamic DB-defined objects via registry (this branch)
- SHOW command DDL extraction (main)
- Ruff formatting / snake_case renames (main)
- CI workflow, CLAUDE.md, auth improvements (main)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Author: remi-td

.github/README.md

@@ -0,0 +1,50 @@
+# CI/CD
+
+## GitHub Actions Workflow
+
+The CI workflow (`.github/workflows/ci.yml`) runs on every push to `main` and on pull requests targeting `main`. It consists of three parallel jobs:
+
+### Jobs
+
+| Job | What it does |
+|-----|-------------|
+| **Lint** | Runs `ruff check` and `ruff format --check` against `src/` |
+| **Type Check** | Runs `mypy` against `src/` (installs the `dev` extra for type stubs) |
+| **Integration Tests** | Runs the test suite against a live Teradata database |
+
+All jobs use `uv sync --frozen` to ensure the lock file is up to date — if `uv.lock` is stale relative to `pyproject.toml`, the job will fail.
+
+### Running Checks Locally
+
+```bash
+# Lint
+uv run ruff check src/
+uv run ruff format --check src/
+
+# Type check
+uv sync --extra dev
+uv run mypy src/
+
+# Integration tests (requires a live Teradata connection)
+export DATABASE_URI="teradata://user:pass@host:1025/database"
+uv run python tests/run_mcp_tests.py "uv run teradata-mcp-server"
+```
+
+### Configuring the `DATABASE_URI` Secret
+
+The integration test job requires a `DATABASE_URI` repository secret to connect to a Teradata instance. Without it, the test job logs a warning and skips.
+
+To configure:
+
+1. Go to **Settings > Secrets and variables > Actions** in the GitHub repository
+2. Click **New repository secret**
+3. Name: `DATABASE_URI`
+4. Value: a Teradata connection URI, e.g. `teradata://user:pass@host:1025/database`
+
+You need to be on the Teradata VPN to access the test database, so this secret should only be added by authorized Teradata personnel. If you don't have access, the tests will simply be skipped.
+
+The test job is automatically skipped on fork PRs (where secrets are unavailable) to avoid failures.
+
+### Concurrency
+
+The workflow uses concurrency groups scoped to the branch/PR ref. If a new commit is pushed while a previous run is still in progress, the older run is cancelled to save CI minutes and avoid database contention.

.github/workflows/ci.yml

@@ -0,0 +1,65 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  lint:
+    name: Lint (Ruff)
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v4
+        with:
+          enable-cache: true
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+      - run: uv sync --frozen --extra dev
+      - run: uv run ruff check src/
+      - run: uv run ruff format --check src/
+
+  typecheck:
+    name: Type Check (Mypy)
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v4
+        with:
+          enable-cache: true
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+      - run: uv sync --frozen --extra dev
+      - run: uv run mypy src/
+
+  test:
+    name: Integration Tests
+    runs-on: ubuntu-latest
+    if: ${{ github.event_name == 'push' || github.event.pull_request.head.repo.full_name == github.repository }}
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v4
+        with:
+          enable-cache: true
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+      - run: uv sync --frozen
+      - name: Run integration tests
+        if: ${{ env.DATABASE_URI != '' }}
+        env:
+          DATABASE_URI: ${{ secrets.DATABASE_URI }}
+        run: uv run python tests/run_mcp_tests.py "uv run teradata-mcp-server"
+      - name: Warn if DATABASE_URI not configured
+        if: ${{ env.DATABASE_URI == '' }}
+        env:
+          DATABASE_URI: ${{ secrets.DATABASE_URI }}
+        run: echo "::warning::DATABASE_URI secret is not configured — integration tests were skipped. See .github/README.md for setup instructions."

.gitignore

@@ -8,10 +8,12 @@ working/
 *.egg-info/
 **/.ipynb_checkpoints/
 /env
-CLAUDE.md
+docker-compose.dev.yml
 var/
 /# Only ignore root-level overrides
 /profiles.yml
 /*_objects.yml
 examples/app-flowise/.*
 test_*.py
+.planning/
+.ruff_cache/

CLAUDE.md

@@ -0,0 +1,101 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+Teradata MCP Server — a Model Context Protocol server for interacting with Teradata databases. Built on FastMCP, distributed on PyPI as `teradata-mcp-server`. Python 3.11+.
+
+## Build & Development Commands
+
+```bash
+# Install dependencies
+uv sync
+
+# Install with optional modules
+uv sync --extra fs       # Feature Store (teradataml)
+uv sync --extra tdvs     # Vector Store (teradatagenai)
+uv sync --extra bar      # Backup & Restore
+uv sync --extra dev      # Dev tools (ruff, mypy)
+
+# Run the server locally (stdio mode)
+uv run teradata-mcp-server --database_uri "teradata://user:pass@host:1025/db"
+
+# Run with specific profile and transport
+uv run teradata-mcp-server --profile dba --mcp_transport streamable-http --mcp_port 8001
+
+# Lint
+uv run ruff check src/
+uv run ruff format --check src/
+
+# Type check
+uv run mypy src/
+
+# Run tests (requires a live Teradata connection)
+export DATABASE_URI="teradata://user:pass@host:1025/database"
+uv run python tests/run_mcp_tests.py "uv run teradata-mcp-server"
+
+# Docker
+docker compose up teradata-mcp-server
+```
+
+## Architecture
+
+### Request Flow
+
+1. **`server.py`** — CLI argument parsing, calls `create_mcp_app()`
+2. **`app.py`** — FastMCP app factory. Creates the MCP instance, registers tools/prompts/resources based on profile, configures middleware
+3. **`middleware.py`** — `RequestContextMiddleware` extracts per-request headers, auth, and session info. Sets Teradata QueryBand for tracing
+4. **Tool handlers** — Plain sync functions (`handle_*`) in `tools/` subdirectories. Wrapped to async via `asyncio.to_thread`
+
+### Profile-Based Module Loading
+
+Tools are organized into domain modules under `src/teradata_mcp_server/tools/`:
+- **base/** — Core queries (readQuery, tableList, columnDescription, etc.)
+- **dba/** — Administration & monitoring
+- **sec/** — Security & permissions
+- **rag/** — Retrieval-augmented generation workflows
+- **fs/** — Feature Store (optional, requires teradataml)
+- **tdvs/** — Vector Store operations
+- **bar/** — Backup & restore (optional)
+- **chat/** — Chat completion
+- **qlty/** — Data quality / EDA
+- **plot/** — Visualization (charts)
+- **sql_opt/** — SQL optimization
+- **tmpl/** — Template tools
+
+Profiles (defined in `config/profiles.yml`) control which modules load. The `module_loader.py` uses regex pattern matching against tool name prefixes to determine which modules to import. Available profiles: `all`, `dba`, `dataScientist`, `eda`, `bar`, `llmUser`, `tester`.
+
+### Configuration System
+
+Layered config loading (`config_loader.py`):
+1. Packaged defaults from `src/teradata_mcp_server/config/*.yml`
+2. User overrides from working directory or `CONFIG_DIR` env var
+
+Settings dataclass in `config/__init__.py` merges CLI args, environment variables, and defaults.
+
+### Database Connectivity
+
+`TDConn` class in `tools/td_connect.py` manages SQLAlchemy engine creation for Teradata. Supports connection pooling (`TD_POOL_SIZE`, `TD_MAX_OVERFLOW`, `TD_POOL_TIMEOUT`), auth modes (TD2, LDAP via `LOGMECH`), and rate-limited authentication.
+
+Tool handlers receive either a SQLAlchemy `Connection` or raw `TeradataConnection` as their first parameter — the wrapper in `app.py` handles injection.
+
+### Transport Modes
+
+Set via `MCP_TRANSPORT` env var or `--mcp_transport` flag:
+- **stdio** (default) — for Claude Desktop and CLI clients
+- **streamable-http** — HTTP with streaming on configurable host/port
+- **sse** — Server-Sent Events, this will be merged into streamable-http as the MCP standard is depricating SSE as a separate transport
+
+For stdio transport, logs go to file only (to avoid polluting MCP stdout). Log locations: macOS `~/Library/Logs/TeradataMCP/`, Linux `~/.local/state/teradata_mcp_server/logs/`.
+
+### Testing
+
+Tests require a live Teradata database. Test cases are JSON files in `tests/cases/` (e.g., `core_test_cases.json`). The test runner (`tests/run_mcp_tests.py`) dynamically discovers available tools and only runs matching test cases. Results are saved as timestamped JSON in `var/test-reports/`.
+
+## Code Conventions
+
+- **Ruff** for linting/formatting: line length 120, target py311, rules: E, W, F, I, N, B, UP, C4, SIM, PIE, PL
+- Tool handler functions are named `handle_<tool_name>` and stay synchronous
+- Tool names are prefixed by module: `base_*`, `dba_*`, `sec_*`, `rag_*`, etc.
+- Structured JSON logging via `CustomJSONFormatter` in `utils.py`

docs/server_guide/CUSTOMIZING.md

@@ -217,7 +217,7 @@ my-teradata-config/
 ```
 
 ### Complete Example
-See the provided [`custom_objects.yml`](../custom_objects.yml) in the repository for a complete working example.
+See the provided [`custom_objects.yml`](../../examples/server-customisation/custom_objects.yml) in the repository for a complete working example.
 
 ### Running with Custom Configuration
 ```bash