Resolve comments in PR

Author: tintinrevient

.pre-commit-config.yaml

@@ -17,6 +17,12 @@ repos:
   hooks:
     - id: validate-pyproject
 
+- repo: https://github.com/pycqa/isort
+  rev: 5.13.2
+  hooks:
+    - id: isort
+      args: [--profile=black]
+
 - repo: https://github.com/astral-sh/ruff-pre-commit
   rev: v0.11.7
   hooks:

README.md

@@ -8,8 +8,28 @@ Before you start, you need to create a `git-auth.txt` file in two folders respec
 https://username:token@github.com
 ```
 
+## Models
+
+The models are included in the [models](models/) folder, where each model occupies a subfolder as its repo.
+
+A model repo contains its README.md as a model card, which comes in two parts:
+- Metadata, which is a YAML section at the top, i.e., front matter.
+- Text descriptions, which is a Markdown file, including summary and descriptions of the model.
+
+For more information, you can reference Hugging Face's [model cards](https://huggingface.co/docs/hub/en/model-cards).
+
+## Datasets
+
+The datasets are included in the [dataset](datasets/) folder.
+
 ## Benchmark
 
+The benchmark is defined in the [benchmark](benchmark/) folder, where each dataset occupies a subfolder.
+
+In order to build the archived file for each dataset, [pg2-dataset](https://github.com/ProteinGym2/pg2-dataset) is used.
+
+You can reference [this guide](https://github.com/ProteinGym2/pg2-dataset?tab=readme-ov-file#archive-data) to build the archived dataset.
+
 ### Local environment
 
 There are two games to benchmark: supervised and zero-shot. Each game has its selected list of models and datasets defined in `dvc.yaml`.

models/esm/README.md

@@ -1,10 +1,15 @@
 ---
+# Model identifier used for referencing this model in the benchmark system
 name: "esm"
 
 hyper_params:
+    # HuggingFace model checkpoint identifier for the specific ESM-2 variant
     location: "esm2_t30_150M_UR50D"
+    # Scoring method: calculates marginal probabilities for wild-type amino acids
     scoring_strategy: "wt-marginals"
+    # Whether to disable GPU usage (false = use GPU if available)
     nogpu: false
+    # Offset index for sequence position alignment in tokenization
     offset_idx: 24
 ---
 

models/pls/README.md

@@ -1,9 +1,13 @@
 ---
+# Model identifier used for referencing this model in the benchmark system
 name: "pls"
 
 hyper_params:
+    # Number of PLS components to extract (dimensionality of the reduced space)
     n_components: 2
+    # Standard 20 amino acid single-letter codes
     aa_alphabet: ["A", "C", "D", "E", "F", "G", "H", "I", "K", "L", "M", "N", "P", "Q", "R", "S", "T", "V", "W", "Y"]
+    # Total number of amino acids in the alphabet (must match aa_alphabet length)
     aa_alphabet_length: 20
 ---
 

pyproject.toml

@@ -40,6 +40,7 @@ logging_format = "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
 
 [dependency-groups]
 dev = [
+    "isort>=6.0.1",
     "pre-commit>=4.2.0",
     "pytest>=8.4.1",
     "pytest-cov>=6.2.1",

tests/test_model_card.py

@@ -1,8 +1,10 @@
-import pytest
 from pathlib import Path
-from pg2_benchmark.model_card import ModelCard
+
+import pytest
 from pydantic import ValidationError
 
+from pg2_benchmark.model_card import ModelCard
+
 
 @pytest.fixture
 def model_card_contents() -> str:
@@ -52,4 +54,4 @@ def test_manifest_hyper_params(model_card_path: Path) -> None:
     except ValidationError as e:
         raise ValidationError("ValidationError raised") from e
     else:
-        assert model_card.hyper_params["nogpu"] is False
+        assert not model_card.hyper_params["nogpu"]