Merge pull request #38 from MarcSkovMadsen/fix/holoviz-configuration

Fix/holoviz configuration

Author: MarcSkovMadsen

.github/prompts/holoviz.prompt.md

@@ -0,0 +1,142 @@
+# HoloViz Development Guidelines
+
+## Overview
+
+Use the HoloViz ecosystem including Panel, Param, and hvPlot for building interactive data applications following intermediate to expert patterns.
+
+## Panel Application Development
+
+### Core Architecture Principles
+
+**Parameter-Driven Design**
+
+- Create applications as `param.Parameterized` or `pn.Viewable` classes
+- Let Parameters drive application state, not widgets directly
+- Structure code so user interactions can be tested using Parameterized classes
+- Use a source `data` parameter to drive your app - structure code so app state resets when source data changes
+
+**Widget and Display Patterns**
+
+- Create widgets from parameters: `pn.widgets.Select.from_param(state.param.parameter_name, ...)`
+- Display Parameter objects in panes: `pn.pane.HoloViews(state.param.parameter_name, ...)`
+- Prefer `pn.bind` or `@param.depends` without `watch=True` for reactive updates
+- Use `.on_click` for Button interactions over `watch=True` patterns
+- Avoid `pn.bind` or `@pn.depends` with `watch=True`, `.watch`, or `.link` methods as they make apps harder to reason about
+
+**other**
+
+- prefer using `.servable()` over `.show()` for serving applications
+- use `pn.state.served:` to check if the app is being served instead of `if __name__ == "__main__"
+
+### Code Organization
+
+**Module Structure**
+
+- Put data extractions and transformations in `data.py` - keep clean and reusable without HoloViz dependencies
+- Put plot functions in `plots.py` - keep clean and reusable without Panel code
+- Separate business logic from UI concerns
+
+**Component Selection**
+
+- Use `panel-graphic-walker` package for Tableau-like data exploration components
+- Use `panel-material-ui` components for new projects or projects already using this package
+- Continue using standard Panel components in existing projects that already use them
+
+### Testing Strategy
+
+**Testable Architecture**
+
+- Structure code so user interactions can be tested through Parameterized classes
+- Separate UI logic from business logic to enable unit testing
+- Use parameter watchers and dependencies for reactive behavior that can be tested
+
+## Best Practices
+
+### Reactive Programming
+
+- Prefer declarative reactive patterns over imperative event handling
+- Use `@param.depends` decorators to create reactive methods
+- Leverage parameter watchers for automatic state management
+
+### Performance Considerations
+
+- Use `sizing_mode="stretch_width"` for responsive layouts
+- Avoid unnecessary parameter watchers that could cause performance issues
+- Structure data flows to minimize redundant computations
+
+### Error Handling
+
+- Implement graceful handling of missing or invalid data
+- Provide meaningful feedback to users when operations fail
+- Use safe data access patterns for robust applications
+
+## Example Patterns
+
+### Parameter-Driven Widget Creation
+```python
+# Good: Widget driven by parameter
+select_widget = pn.widgets.Select.from_param(
+    self.param.model_type,
+    name="Model Type"
+)
+
+# Avoid: Manual widget management
+select_widget = pn.widgets.Select(
+    options=['Option1', 'Option2'],
+    value='Option1'
+)
+```
+
+### Reactive Display Updates
+
+```python
+# Best: Depends functions and methods
+@param.depends('model_results')
+def create_plot(self):
+    return create_performance_plot(self.model_results)
+
+plot_pane = pn.pane.Matplotlib(
+    self.create_plot
+)
+
+# Good: Bound functions and methods
+def create_plot(self):
+    return create_performance_plot(self.model_results)
+
+plot_pane = pn.pane.Matplotlib(
+    pn.bind(self.create_plot)
+)
+
+# Avoid: Manual updates with watchers
+def update_plot(self):
+    self.plot_pane.object = create_performance_plot(self.model_results)
+
+self.param.watch(self.update_plot, 'model_results')
+```
+
+### Data-Driven Architecture
+
+```python
+class DataApp(param.Parameterized):
+    data = param.DataFrame(default=pd.DataFrame())
+
+    @param.depends('data', watch=True)
+    def _reset_app_state(self):
+        """Reset all app state when source data changes."""
+        # Reset or update parameters but not widgets directly
+        ...
+
+    @param.depends('data')
+    def _get_xyz(self):
+        """Return some transformed object like a DataFrame or a Plot/ Figure."""
+        # Keep this method short by using imported method from data or plots module
+        ...
+```
+
+## Resources
+
+- [Panel Intermediate Tutorials](https://panel.holoviz.org/tutorials/intermediate/index.html)
+- [Panel Expert Tutorials](https://panel.holoviz.org/tutorials/expert/index.html)
+- [Param Documentation](https://param.holoviz.org/)
+- [Panel Material UI Components](https://panel-material-ui.holoviz.org/)
+- [Panel Graphic Walker](https://panel-graphic-walker.holoviz.org/)

.github/prompts/ruff_check.prompt.md

@@ -0,0 +1,17 @@
+When fixing ruff issues, please follow these guidelines below.
+
+Please fix one issue at a time and test the change before progressing to next issue.
+Please fix ruff issues in the 'src' and 'tests' folders only.
+Please fix easy to fix issues before more complex ones.
+For complex ones feel free to ask the user for guidance or help.
+
+## Specific Rules
+
+### D405
+
+Please ensure the docstring change is meaningful to end users and Google style.
+If the summary extends to multiple lines please reformulate instead of breaking the summary into two distinct lines.
+
+### ARG001
+
+If the argument is an unused pytest fixture please change it to a marker, e.g. to `@pytest.mark.usefixtures("name_of_fixture")`.

src/holoviz_mcp/apps/configuration_viewer.py

@@ -41,7 +41,7 @@ def __init__(self, **params):
         Loads environment config and config loader.
         """
         super().__init__(**params)
-        self._env_config = HoloVizMCPConfig.from_environment()
+        self._env_config = HoloVizMCPConfig()
         self._loader = get_config_loader()
 
     @param.depends("config_source")

src/holoviz_mcp/apps/search.py

@@ -258,6 +258,7 @@ class SearchApp(pn.viewable.Viewer):
         - Integration with HoloViz MCP docs_search tool
     """
 
+    title = param.String(default="HoloViz MCP Search Tool", doc="Title of the search app")
     config = param.ClassSelector(class_=SearchConfiguration, doc="Configuration for the search app")
 
     def __init__(self, **params):
@@ -297,7 +298,7 @@ def __panel__(self):
             github_button.js_on_click(code=js_code_to_open_holoviz_mcp)
 
             return pmui.Page(
-                title="HoloViz MCP Search Tool",
+                title=self.title,
                 site_url="./",
                 sidebar=[self._config, menu],
                 sidebar_width=400,

src/holoviz_mcp/config/loader.py

@@ -52,7 +52,7 @@ def load_config(self) -> HoloVizMCPConfig:
         if self._env_config is not None:
             env_config = self._env_config
         else:
-            env_config = HoloVizMCPConfig.from_environment()
+            env_config = HoloVizMCPConfig()
 
         # Start with default configuration dict
         config_dict = self._get_default_config()

src/holoviz_mcp/config/models.py

@@ -16,6 +16,16 @@
 from pydantic import field_validator
 
 
+def _holoviz_mcp_user_dir() -> Path:
+    """Get the default user directory for HoloViz MCP."""
+    return Path(os.environ.get("HOLOVIZ_MCP_USER_DIR", Path.home() / ".holoviz-mcp"))
+
+
+def _holoviz_mcp_default_dir() -> Path:
+    """Get the default configuration directory for HoloViz MCP."""
+    return Path(os.environ.get("HOLOVIZ_MCP_DEFAULT_DIR", Path(__file__).parent.parent / "config"))
+
+
 class FolderConfig(BaseModel):
     """Configuration for a folder within a repository."""
 
@@ -144,21 +154,9 @@ class ServerConfig(BaseModel):
     anonymized_telemetry: bool = Field(default=False, description="Enable anonymized telemetry")
     jupyter_server_proxy_url: str = Field(default="", description="Jupyter server proxy URL for Panel app integration")
     security: SecurityConfig = Field(default_factory=SecurityConfig, description="Security configuration")
-    vector_db_path: Path = Field(default_factory=lambda: Path.home() / ".holoviz-mcp" / "vector_db" / "chroma", description="Path to the Chroma vector database.")
-
-    def resolve_vector_db_path(self, user_dir: Path) -> Path:
-        """Resolve the path to the Chroma vector database.
-
-        This method ensures the vector_db_path is properly expanded.
-
-        Args:
-            user_dir: User directory (for compatibility, but not used since vector_db_path is now always set)
-
-        Returns
-        -------
-            Resolved path to the vector database
-        """
-        return Path(self.vector_db_path).expanduser()
+    vector_db_path: Path = Field(
+        default_factory=lambda: (_holoviz_mcp_user_dir() / "vector_db" / "chroma").expanduser(), description="Path to the Chroma vector database."
+    )
 
 
 class HoloVizMCPConfig(BaseModel):
@@ -170,29 +168,12 @@ class HoloVizMCPConfig(BaseModel):
     prompts: PromptConfig = Field(default_factory=PromptConfig)
 
     # Environment paths - merged from EnvironmentConfig with defaults
-    user_dir: Path = Field(default_factory=lambda: Path.home() / ".holoviz-mcp", description="User configuration directory")
-    default_dir: Path = Field(default_factory=lambda: Path(__file__).parent.parent / "config", description="Default configuration directory")
-    repos_dir: Path = Field(default_factory=lambda: Path.home() / ".holoviz-mcp" / "repos", description="Repository download directory")
+    user_dir: Path = Field(default_factory=_holoviz_mcp_user_dir, description="User configuration directory")
+    default_dir: Path = Field(default_factory=_holoviz_mcp_default_dir, description="Default configuration directory")
+    repos_dir: Path = Field(default_factory=lambda: _holoviz_mcp_user_dir() / "repos", description="Repository download directory")
 
     model_config = ConfigDict(extra="forbid", validate_assignment=True)
 
-    @classmethod
-    def from_environment(cls) -> HoloVizMCPConfig:
-        """Create configuration from environment variables with defaults."""
-        user_dir = Path(os.environ.get("HOLOVIZ_MCP_USER_DIR", Path.home() / ".holoviz-mcp"))
-        default_dir = Path(os.environ.get("HOLOVIZ_MCP_DEFAULT_DIR", Path(__file__).parent.parent / "config"))
-        repos_dir = Path(os.environ.get("HOLOVIZ_MCP_REPOS_DIR", user_dir / "repos"))
-
-        return cls(
-            server=ServerConfig(),
-            docs=DocsConfig(),
-            resources=ResourceConfig(),
-            prompts=PromptConfig(),
-            user_dir=user_dir,
-            default_dir=default_dir,
-            repos_dir=repos_dir,
-        )
-
     def config_file_path(self, location: Literal["user", "default"] = "user") -> Path:
         """Get the path to the configuration file.
 

src/holoviz_mcp/docs_mcp/data.py

@@ -204,12 +204,13 @@ def convert_path_to_url(path: Path, remove_first_part: bool = True, url_transfor
 class DocumentationIndexer:
     """Handles cloning, processing, and indexing of documentation."""
 
-    def __init__(self, data_dir: Optional[Path] = None, repos_dir: Optional[Path] = None):
+    def __init__(self, *, data_dir: Optional[Path] = None, repos_dir: Optional[Path] = None, vector_dir: Optional[Path] = None):
         """Initialize the DocumentationIndexer.
 
         Args:
             data_dir: Directory to store index data. Defaults to user config directory.
             repos_dir: Directory to store cloned repositories. Defaults to HOLOVIZ_MCP_REPOS_DIR.
+            vector_dir: Directory to store vector database. Defaults to config.vector_dir
         """
         # Use unified config for default paths
         config = get_config()
@@ -221,9 +222,8 @@ def __init__(self, data_dir: Optional[Path] = None, repos_dir: Optional[Path] =
         self.repos_dir = repos_dir or config.repos_dir
         self.repos_dir.mkdir(parents=True, exist_ok=True)
 
-        # Use config logic to resolve vector DB path
-        config = get_config()
-        vector_db_path = config.server.resolve_vector_db_path(config.user_dir)
+        # Use configurable directory for vector database path
+        vector_db_path = vector_dir or config.server.vector_db_path
         vector_db_path.parent.mkdir(parents=True, exist_ok=True)
 
         # Disable ChromaDB telemetry based on config
@@ -577,7 +577,6 @@ async def extract_docs_from_repo(self, repo_path: Path, project: str, ctx: Conte
         regular_count = len(docs) - reference_count
 
         await log_info(f"  📄 {project}: {len(docs)} total documents ({regular_count} regular, {reference_count} reference guides)", ctx)
-
         return docs
 
     async def index_documentation(self, ctx: Context | None = None):
@@ -591,7 +590,6 @@ async def index_documentation(self, ctx: Context | None = None):
         # Clone/update repositories and extract documentation
         for repo_name, repo_config in self.config.repositories.items():
             await log_info(f"Processing {repo_name}...", ctx)
-
             repo_path = await self.clone_or_update_repo(repo_name, repo_config)
             if repo_path:
                 docs = await self.extract_docs_from_repo(repo_path, repo_name, ctx)
@@ -934,34 +932,32 @@ async def _log_summary_table(self, ctx: Context | None = None):
         except Exception as e:
             await log_warning(f"Failed to generate summary table: {e}", ctx)
 
+    def run(self):
+        """Update the DocumentationIndexer."""
+        # Configure logging for the CLI
+        logging.basicConfig(level=logging.INFO, format="%(asctime)s - %(levelname)s - %(message)s", handlers=[logging.StreamHandler()])
 
-def main():
-    """Update the DocumentationIndexer."""
-    # Configure logging for the CLI
-    logging.basicConfig(level=logging.INFO, format="%(asctime)s - %(levelname)s - %(message)s", handlers=[logging.StreamHandler()])
-
-    logger.info("🚀 HoloViz MCP Documentation Indexer")
-    logger.info("=" * 50)
+        logger.info("🚀 HoloViz MCP Documentation Indexer")
+        logger.info("=" * 50)
 
-    async def run_indexer():
-        indexer = DocumentationIndexer()
-        logger.info(f"📁 Repository directory: {indexer.repos_dir}")
-        logger.info(f"💾 Vector database: {indexer.data_dir / 'chroma'}")
-        logger.info(f"🔧 Configured repositories: {len(indexer.config.repositories)}")
-        logger.info("")
+        async def run_indexer(indexer=self):
+            logger.info(f"📁 Repository directory: {indexer.repos_dir}")
+            logger.info(f"💾 Vector database: {indexer.data_dir / 'chroma'}")
+            logger.info(f"🔧 Configured repositories: {len(indexer.config.repositories)}")
+            logger.info("")
 
-        await indexer.index_documentation()
+            await indexer.index_documentation()
 
-        # Final summary
-        count = indexer.collection.count()
-        logger.info("")
-        logger.info("=" * 50)
-        logger.info("✅ Indexing completed successfully!")
-        logger.info(f"📊 Total documents in database: {count}")
-        logger.info("=" * 50)
+            # Final summary
+            count = indexer.collection.count()
+            logger.info("")
+            logger.info("=" * 50)
+            logger.info("✅ Indexing completed successfully!")
+            logger.info(f"📊 Total documents in database: {count}")
+            logger.info("=" * 50)
 
-    asyncio.run(run_indexer())
+        asyncio.run(run_indexer())
 
 
 if __name__ == "__main__":
-    main()
+    DocumentationIndexer().run()

tests/config/test_models.py

@@ -181,3 +181,11 @@ def test_config_forbids_extra_fields(self):
             # We need to construct this in a way that bypasses type checking
             config_dict = {"server": {"name": "test"}, "docs": {"repositories": {}}, "resources": {}, "prompts": {}, "extra_field": "not allowed"}
             HoloVizMCPConfig(**config_dict)
+
+    def test_with_environment_variables(self, monkeypatch):
+        """Test configuration with environment variables."""
+        monkeypatch.setenv("HOLOVIZ_MCP_USER_DIR", "/test/dir")
+        config = HoloVizMCPConfig()
+        assert config.user_dir == Path("/test/dir")
+        assert config.server.vector_db_path == Path("/test/dir") / "vector_db" / "chroma"
+        assert config.repos_dir == Path("/test/dir") / "repos"