Databases (#188)

* removed comment

* removed kinase_schema.CollectionKinaseInfo

* comment on PRKD2 and AlphaMissense

* temporary scratch for aligning sequences to DiscoverX

* implemented new class ChEMBLMolecule to query for molecule details

* added xlrd to package dependencies to process Davis dataset

* preliminary info for davis harmonization

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add check_molecules to ChEMBL; updated wrong ChEMBLMolecule argument

* add check_molecules to ChEMBL; updated wrong ChEMBLMolecule argument

* make rdkit a package dependency

* cli for querying ChEMBL for dataset preprocessing

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* moved davis and pkis2 modules to datasets

* changed error message for maybe_get_symbol_from_hgnc_search if custom_field provided

* updates to pkis2 and davis datasets modules

* removed commented out PR CIs for databases and schema

* fixed chembl search error - default empty list not None

* added adjudicate_kd_start and adjudicate_kd_end for dataset incorporation purposes

* added docstring for bool_offset

* allow for str_fasta to be used if need to hardcode for errors

* removed pytest.mark.skip as NCBI API is currently running

* added function to check if lipid kinase

* specified input_is_hgnc_symbol default in docstring

* added Pfam docstring

* UniProtRefSeqProteinGET and query_uniprotbulk_api to uniprot module; modifies nf-rnaseq package tooling

* fully working initial commit of discoverx module; construct to KD/KLIFS mapping outstanding

* added verbose flag to the KinaseInfo functions rather than logging by default

* added verbose flags

* added and commented out pip install nf-rnaseq from github; uncomment for testing if in use

* import only UniProtFASTA rather than entire uniprot module to avoid nf-rnaseq import errors; fix if want to test this functionality

* uncommented nf-rnaseq

* in progress datasets commit

* used verbose flag for caplog tests

* dict_refseq_indices working correctly

* dict_construct_sequences finalized - use this to generate harmonized representations

* generate the dataset csv files

* process now contains all code necssary to generate different aligned input sequences

* conformed to latest process module structure

* added dataset csv CLI to pyproject.toml

* added plotting functions for discoverx

* upgrades for discoverx plotting

* CLI script to generate poster dataset plots

* plot both svg and PNG formats for all

* added plot dynamic range to the plotting CLI, need to fix font size

* fixed svg in plot_dynamic_range - font still looks a little off; added docstrings and fixed comment format

* Fix test_pfam and test_ncbi to handle API 500 errors gracefully

Handle RetryError exceptions when external APIs return 500 errors by
skipping tests instead of failing. This prevents CI failures due to
unpredictable external API availability.

Changes:
- Wrap test_pfam API calls in try-except block
- Wrap test_ncbi API calls in try-except block
- Skip tests with informative messages when 500 errors occur
- Re-raise other exceptions to catch real issues

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* Refactor plotting code and fix SVG font rendering issues

This commit improves the plotting functionality by:
1. Creating a reusable save_plot() helper function to reduce code duplication
2. Fixing SVG font rendering issues by converting text to paths
3. Improving mathtext rendering for subscripts (K_d, log_10)

Changes:
- Add save_plot() function to handle saving both SVG and PNG formats
- Replace repetitive save code in all 5 plotting functions
- Change svg.fonttype from "none" to "path" for consistent rendering
- Update mathtext from \mathregular to \mathrm for proper subscript rendering
- Ensure plots render consistently in browsers, VS Code, and vector editors

Benefits:
- SVG files now render perfectly in all viewers without spacing/kerning issues
- Reduced code duplication by ~60 lines
- Easier maintenance with centralized save logic
- Consistent behavior across all plotting functions

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* absolute filepath for cwd instead of '.'

* fixed KinaseMissenseMutations.dict_replace - only do this if key in original datast

* make the checks and post_init optional in case loading from a CSV file for a cohort that requires a VPN - logger errors are now warnings; allow load_from_csv from an input str if loading from multiple dataframes (e.g., KinaseMissenseMutations ._df and ._df_filter); added pathfile_filter to KinaseMissenseMutations

* updated databases for kw_only arg study_id in Mutations

* fixed bug in dict_kinase_cbio in get_kinase_missense_mutations function - need to check if mkt_name is in dict_kinase_cbio rather than cbio_name

* changed HGNC name and mismatch error logging

* Two minor logger formatting tweaks (#186)

Linebreaks and spacing for canonical mismatch errors

* only log query errors if present

* moved classes from app to mkt.databases.app since need to use extensibly in other places (mkt_impact); simplified names for relevant app modules since no longer scripts importing locally; remove py3dmol and streamlit/bokeh related plotting functions to standalone visualization script in app; created pymol module and moved CLI script to mkt.databases; added webcolors to pyproject.toml dependencies

* removed all plotting - keep this in standalone app

* removed all plotting - keep this in standalone app

* updated imports for new app structure

* changed imports in app script

* PyMOL module and CLI

* removed self.html = self.visualize_structure() from StructureVisualizer and moved to StructureVisualizerVisualizer

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Claude <[email protected]>

Author: 3 people

missense_kinase_toolkit/app/visualizers.py

@@ -177,6 +177,10 @@ class StructureVisualizerVisualizer(StructureVisualizer):
     )
     """Dimensions for the py3Dmol viewer."""
 
+    def __post_init__(self):
+        super().__init__(self.obj_kinase, self.str_attr)
+        self.html = self.visualize_structure()
+
     def visualize_structure(self) -> str | None:
         """Visualize the structure using py3Dmol.
 

missense_kinase_toolkit/databases/mkt/databases/app/structures.py

@@ -73,7 +73,6 @@ def __post_init__(self):
         self.structure = self.convert_mmcifdict2structure()
         self.pdb_text = self.convert_structure2string()
         self.residues = self.structure.get_residues()
-        self.html = self.visualize_structure()
 
     @staticmethod
     def _parse_pdb_line(line: str) -> dict[str, Any] | None:

missense_kinase_toolkit/databases/mkt/databases/cli/generate_pymol_files.py

@@ -1,15 +1,14 @@
 #!/usr/bin/env python3
 
 import argparse
-import os
 
 from mkt.databases.app.sequences import SequenceAlignment
 from mkt.databases.app.structures import StructureVisualizer
 from mkt.databases.colors import DICT_COLORS
-from mkt.databases.pymol import generate_simple_pymol_files
-from mkt.schema import io_utils
+from mkt.databases.pymol import PyMOLGenerator
+from mkt.schema.io_utils import deserialize_kinase_dict
 
-DICT_KINASE = io_utils.deserialize_kinase_dict(str_name="DICT_KINASE")
+DICT_KINASE = deserialize_kinase_dict(str_name="DICT_KINASE")
 
 
 def parse_args():
@@ -45,30 +44,6 @@ def main():
         bool_show=False,
     )
 
+    pymol_generator = PyMOLGenerator(viz=viz)
     output_directory = f"./pymol_output/{gene}"
-    _, script_file = generate_simple_pymol_files(viz, output_directory, gene)
-
-    f"""
-    Files generated:
-      PDB: {os.path.join(output_directory, f"{gene}_structure.pdb")}
-      Script: {os.path.join(output_directory, f"{gene}_pymol_script.py")}
-
-    To run in PyMOL:
-      1. Open PyMOL
-      2. Navigate to {output_directory}
-      3. Run: run {gene}_pymol_script.py
-      4. Save PNG: set ray_trace_mode, 3; png filename.png, ray=1, dpi=300
-    """
-
-    print("\n" + "=" * 60)
-    print("MANUAL PYMOL INSTRUCTIONS:")
-    print("=" * 60)
-    print("1. Open PyMOL GUI or command line")
-    print("2. Change to the output directory:")
-    print(f"   cd {os.path.abspath(output_directory)}")
-    print("3. Run the script:")
-    print(f"   run {os.path.basename(script_file)}")
-    print("4. To save as high-res PNG:")
-    print("   set ray_trace_mode, <mode>")
-    print("   png your_filename.png, ray=1, dpi=300")
-    print("=" * 60)
+    pymol_generator.save_pymol_files(output_directory, gene)

missense_kinase_toolkit/databases/mkt/databases/pymol.py

@@ -1,14 +1,9 @@
 import os
 
-try:
-    import webcolors
+import webcolors
 
-    WEBCOLORS_AVAILABLE = True
-except ImportError:
-    WEBCOLORS_AVAILABLE = False
 
-
-class SimplePyMOLGenerator:
+class PyMOLGenerator:
     """Generate PDB file with embedded color/style info and standalone PyMOL script."""
 
     def __init__(self, structure_visualizer):
@@ -25,12 +20,7 @@ def _convert_color_to_hex(self, color: str) -> str:
         if color.startswith("#"):
             return color
 
-        # Try webcolors first
-        if WEBCOLORS_AVAILABLE:
-            try:
-                return webcolors.name_to_hex(color)
-            except ValueError:
-                pass
+        webcolors.name_to_hex(color)
 
         # Fallback color mapping
         color_map = {
@@ -270,48 +260,49 @@ def generate_pymol_script(self, pdb_path: str, output_path: str) -> str:
 
         return output_path
 
-
-def generate_simple_pymol_files(viz_obj, output_dir: str, base_name: str = None):
-    """
-    Generate PDB file and PyMOL script for manual PyMOL execution.
-
-    Parameters
-    ----------
-    viz_obj : StructureVisualizer
-        Existing StructureVisualizer object
-    output_dir : str
-        Directory to save files
-    base_name : str, optional
-        Base name for files (default: gene name)
-
-    Returns
-    -------
-    tuple
-        Paths to (pdb_file, pymol_script)
-    """
-    if base_name is None:
-        base_name = viz_obj.obj_kinase.hgnc_name
-
-    # Ensure output directory exists
-    os.makedirs(output_dir, exist_ok=True)
-
-    # Generate files
-    generator = SimplePyMOLGenerator(viz_obj)
-
-    pdb_path = os.path.join(output_dir, f"{base_name}_structure.pdb")
-    script_path = os.path.join(output_dir, f"{base_name}_pymol_script.py")
-
-    generator.generate_annotated_pdb(pdb_path)
-    generator.generate_pymol_script(pdb_path, script_path)
-
-    print("Files generated:")
-    print(f"  PDB: {pdb_path}")
-    print(f"  Script: {script_path}")
-    print("")
-    print("To run in PyMOL:")
-    print("  1. Open PyMOL")
-    print(f"  2. Navigate to {output_dir}")
-    print(f"  3. Run: run {os.path.basename(script_path)}")
-    print("  4. Save PNG: set ray_trace_mode, 3; png your_filename.png, ray=1, dpi=300")
-
-    return pdb_path, script_path
+    def save_pymol_files(self, output_dir: str):
+        """
+        Generate PDB file and PyMOL script for manual PyMOL execution.
+
+        Parameters
+        ----------
+        output_dir : str
+            Directory to save files
+
+        Returns
+        -------
+        tuple
+            Paths to (pdb_file, pymol_script)
+        """
+        os.makedirs(output_dir, exist_ok=True)
+
+        pdb_path = os.path.join(output_dir, f"{self.gene_name}_structure.pdb")
+        script_path = os.path.join(output_dir, f"{self.gene_name}_pymol_script.py")
+
+        self.generate_annotated_pdb(pdb_path)
+        self.generate_pymol_script(pdb_path, script_path)
+
+        f"""
+        Files generated:
+        PDB: {os.path.join(output_dir, f"{self.gene_name}_structure.pdb")}
+        Script: {os.path.join(output_dir, f"{self.gene_name}_pymol_script.py")}
+
+        To run in PyMOL:
+        1. Open PyMOL
+        2. Navigate to {output_dir}
+        3. Run: run {self.gene_name}_pymol_script.py
+        4. Save PNG: set ray_trace_mode, 3; png filename.png, ray=1, dpi=300
+        """
+
+        print("\n" + "=" * 60)
+        print("MANUAL PYMOL INSTRUCTIONS:")
+        print("=" * 60)
+        print("1. Open PyMOL GUI or command line")
+        print("2. Change to the output directory:")
+        print(f"   cd {os.path.abspath(output_dir)}")
+        print("3. Run the script:")
+        print(f"   run {os.path.basename(script_path)}")
+        print("4. To save as high-res PNG:")
+        print("   set ray_trace_mode, <mode>")
+        print("   png <your_filename>.png, ray=1, dpi=300")
+        print("=" * 60)