-
Notifications
You must be signed in to change notification settings - Fork 648
docs(skills): add structured DPA4 workflow support #5961
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
SchrodingersCattt
wants to merge
30
commits into
deepmodeling:master
Choose a base branch
from
SchrodingersCattt:docs/add-deepmd-dpa4-skill
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+1,254
−93
Open
Changes from 12 commits
Commits
Show all changes
30 commits
Select commit
Hold shift + click to select a range
d03cf0b
docs(skills): add DPA4 workflows
4eb0665
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 979ba46
Refine description for deepmd-finetune-dpa4 skill
SchrodingersCattt a801ee6
docs(skills): leave DPA3 skill unchanged
6e27dc2
docs(skills): clarify pt2 inference limits
dc8ec95
docs(skills): preserve selected DPA4 heads
92476d3
docs(skills): qualify DPA4 pt2 deployment
0e24272
docs(skills): pin online LAMMPS runtime
c41e68a
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] adc8616
docs(skills): qualify pt2 atomic outputs
263c340
docs(skills): make LAMMPS help noninteractive
588023c
docs(skills): clarify DPA4 checkpoint and LoRA selection
SchrodingersCattt d5e2d6c
docs(skills): streamline DPA4 finetuning guidance
SchrodingersCattt f42fdde
docs(skills): add MatMaster DPA4 workflows
weiqichen77 bbe64e0
docs(skills): address DPA4 workflow validation gaps
SchrodingersCattt a9f1279
Revert "docs(skills): add MatMaster DPA4 workflows"
d7c988a
Merge remote-tracking branch 'upstream/master' into docs/add-deepmd-d…
06bc4d9
docs(skills): gate LAMMPS runtime by capability
2541ae0
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 4ccbd91
docs(skills): define complete held-out evaluation
0807ca6
docs(skills): harden DPA4 LAMMPS deployment
fe8168c
docs(skills): fix DPA4 evaluation and export contracts
dd3fb6a
test(skills): cover DPA4 review contracts
3a871cd
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 84923b8
docs(skills): reserve held-out detail roots
2b4ac5d
Merge branch 'master' of https://github.com/deepmodeling/deepmd-kit i…
SchrodingersCattt 783ff88
docs(skills): add DPA4C training guidance
SchrodingersCattt 5437b76
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] f3328a9
docs(skills): complete DPA4C deployment guidance
cbff85a
test(skills): guard DPA4C export contracts
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,273 @@ | ||
| --- | ||
| name: deepmd-finetune-dpa4 | ||
| description: Fine-tune a DPA4 model in DeePMD-kit. Use for standard or LoRA fine-tuning from a DPA4/SeZM .pt checkpoint, validation and .pt2 export. | ||
| compatibility: Requires deepmd-kit with the PyTorch backend. DPA4/SeZM training is GPU-oriented. | ||
| license: LGPL-3.0-or-later | ||
| metadata: | ||
| author: SchrodingersCattt | ||
| version: '1.0' | ||
| repository: https://github.com/deepmodeling/deepmd-kit | ||
| --- | ||
|
|
||
| # DeePMD-kit Fine-tuning: DPA4 | ||
|
|
||
| Fine-tune a DPA4/SeZM checkpoint on downstream DeePMD data. This skill covers | ||
| single-task standard and LoRA fine-tuning. Do not infer the model family from a | ||
| `.pt` suffix or filename: DPA3 and DPA4 checkpoints use the same suffix. | ||
|
|
||
| ## Route the checkpoint | ||
|
|
||
| If the user has not already established the model family, inspect the stored | ||
| configuration: | ||
|
|
||
| ```bash | ||
| dp --pt show pretrained.pt descriptor fitting-net type-map | ||
| ``` | ||
|
|
||
| Use this skill only when the descriptor/model configuration identifies DPA4 or | ||
| SeZM. If the checkpoint is multi-task, inspect its branches before selecting a | ||
| head: | ||
|
|
||
| ```bash | ||
| dp --pt show pretrained.pt model-branch descriptor type-map | ||
| ``` | ||
|
|
||
| Do not guess a branch. Use `deepmd-finetune-dpa3` instead when the descriptor is | ||
| DPA3, and stop when the family cannot be established. | ||
|
|
||
| ## Obtain a pretrained checkpoint | ||
|
|
||
| Fine-tuning requires a DPA4/SeZM **training checkpoint** (`.pt`), not a compiled | ||
| `.pt2` deployment archive. First check the registry exposed by the installed | ||
| version: | ||
|
|
||
| ```bash | ||
| dp pretrained download -h | ||
| ``` | ||
|
|
||
| Use a built-in model only when an exact DPA4/SeZM name is listed there. Do not | ||
| guess `dp pretrained download DPA4`: some releases have no registered DPA4 | ||
| checkpoint. | ||
|
|
||
| For a workflow smoke test, a DeePMD-kit source checkout contains: | ||
|
|
||
| ```text | ||
| examples/water/dpa4/lmp/pretrained.pt | ||
| ``` | ||
|
|
||
| Use that file directly: | ||
|
|
||
| ```bash | ||
| cp examples/water/dpa4/lmp/pretrained.pt ./pretrained.pt | ||
| dp --pt show pretrained.pt descriptor fitting-net type-map | ||
| ``` | ||
|
|
||
| It is a compact O/H smoke-test model, not a general-purpose pretrained | ||
| potential. For scientific fine-tuning, obtain a checkpoint from its documented | ||
| publisher or train one with the same DeePMD-kit revision that will perform the | ||
| fine-tuning. Pin and record the model source, checksum, DeePMD-kit revision, | ||
| architecture, task branch, and `type_map`. Reject a checkpoint whose descriptor, | ||
| element set/order, or architecture does not match the intended target. Prefer a | ||
| bounded one-step compatibility run before a long job. | ||
|
|
||
| ## Before fine-tuning | ||
|
|
||
| 1. Confirm the checkpoint exists and can be inspected. | ||
| 1. Confirm training and validation systems, labels, and element type maps. | ||
| 1. Keep a held-out test set that is not used for training or model selection. | ||
| 1. Start from the exact checkpoint architecture. Introducing new element types, | ||
| changing architecture, or combining specialized spin/property/multi-task | ||
| configurations requires separate compatibility validation. | ||
| 1. Choose standard fine-tuning or LoRA. Do not assume a built-in DPA4 model name; | ||
| check `dp pretrained download -h` for the installed version. | ||
|
|
||
| ## Decide whether to use LoRA | ||
|
|
||
| Use **standard fine-tuning** by default when the target is multi-task, the domain | ||
| shift is large, all parameters should adapt, or the workflow combines untested | ||
| spin/property/denoising/ZBL changes. Use **LoRA** when the target is single-task, | ||
| parameter-efficient adaptation is desired, the downstream domain is reasonably | ||
| close to pretraining, and the exact base architecture is known. DPA4 LoRA is | ||
| supported by `dp --pt`; do not use it with the exportable training backend. | ||
|
|
||
| The pretrained checkpoint does not need to contain LoRA. LoRA is enabled by the | ||
| new fine-tuning input through a non-null `model.lora` block. Check an input with: | ||
|
|
||
| ```bash | ||
| python - input.json <<'PY' | ||
| import json | ||
| import sys | ||
|
|
||
| with open(sys.argv[1], encoding="utf-8") as stream: | ||
| config = json.load(stream) | ||
|
|
||
| model = config.get("model", {}) | ||
| branches = model.get("model_dict") | ||
| lora = model.get("lora") | ||
| print("multi_task =", isinstance(branches, dict)) | ||
| print("lora =", lora) | ||
| print("use_lora =", lora is not None) | ||
| if isinstance(branches, dict) and lora is not None: | ||
| raise SystemExit("DPA4 LoRA targets must be single-task") | ||
|
SchrodingersCattt marked this conversation as resolved.
Outdated
|
||
| PY | ||
| ``` | ||
|
|
||
| Interpretation: | ||
|
|
||
| - absent or `null` `model.lora` means standard fine-tuning; | ||
| - a `model.lora` mapping means LoRA fine-tuning; | ||
| - a multi-task target plus LoRA is unsupported. | ||
|
|
||
| To diagnose whether a `.pt` file still contains **active, unmerged** LoRA state, | ||
| inspect its saved model parameters and adapter tensors without executing pickled | ||
| code: | ||
|
|
||
| ```bash | ||
| python - pretrained.pt <<'PY' | ||
| import sys | ||
| import torch | ||
|
|
||
| raw = torch.load(sys.argv[1], map_location="cpu", weights_only=True) | ||
| state = raw["model"] if isinstance(raw, dict) and "model" in raw else raw | ||
| if not isinstance(state, dict): | ||
| raise TypeError(f"Unsupported checkpoint payload: {type(state).__name__}") | ||
|
|
||
| extra = state.get("_extra_state", {}) | ||
| params = extra.get("model_params", {}) if isinstance(extra, dict) else {} | ||
| configured = params.get("lora") if isinstance(params, dict) else None | ||
| markers = (".A_by_l", ".B_by_l", ".A_m0", ".B_m0", ".A_m.", ".B_m.", ".lora_scaling") | ||
| adapter_keys = sorted( | ||
| key for key in state if any(marker in key for marker in markers) | ||
| ) | ||
|
|
||
| print("configured_lora =", configured) | ||
| print("adapter_tensor_count =", len(adapter_keys)) | ||
| for key in adapter_keys[:20]: | ||
| print("adapter_tensor =", key) | ||
|
|
||
| if configured is not None and adapter_keys: | ||
| print("classification = active/unmerged LoRA checkpoint") | ||
| elif configured is not None or adapter_keys: | ||
| print("classification = inconsistent or transitional; inspect before use") | ||
| else: | ||
| print("classification = plain checkpoint or merged LoRA checkpoint") | ||
| PY | ||
| ``` | ||
|
|
||
| Do not infer training history from the last classification. Validation-selected | ||
| best LoRA checkpoints fold adapter deltas into ordinary DPA4 weights and remove | ||
| LoRA metadata/tensors, so they intentionally look plain. Such merged checkpoints | ||
| are suitable for evaluation, new fine-tuning, and `.pt2` export, but do not carry | ||
| the optimizer/EMA state needed to resume the same LoRA run. Periodic/final LoRA | ||
| checkpoints retain active adapters and are the resumable form. | ||
|
|
||
| ## Standard fine-tuning | ||
|
|
||
| The model section in `input.json` must match the checkpoint unless the standard | ||
| pretrained-script mechanism is deliberately used: | ||
|
|
||
| ```bash | ||
| dp --pt train input.json --finetune pretrained.pt | ||
| ``` | ||
|
|
||
| When fine-tuning a single-task target from a multi-task checkpoint and the | ||
| intent is to preserve a particular pretrained fitting head, pass the branch | ||
| selected above: | ||
|
|
||
| ```bash | ||
| dp --pt train input.json --finetune pretrained.pt --model-branch SELECTED_BRANCH | ||
| ``` | ||
|
|
||
| If `--model-branch` is omitted, the fitting net may be initialized from the | ||
| `RANDOM` branch instead. A multi-task target uses `finetune_head` in each target | ||
| branch rather than the command-line option. | ||
|
|
||
| If the architecture is unknown, `--use-pretrain-script` can inherit the stored | ||
| model configuration except for `type_map`: | ||
|
|
||
| ```bash | ||
| dp --pt train input.json --finetune pretrained.pt --use-pretrain-script | ||
| ``` | ||
|
|
||
| Inspect the resulting configuration and run a bounded initial segment before a | ||
| long training job. Do not combine model-specific additions with | ||
| `--use-pretrain-script` unless that combination has been validated. | ||
|
|
||
| ## LoRA fine-tuning | ||
|
|
||
| DPA4/SeZM supports LoRA adapters for single-task fine-tuning. Copy the exact base | ||
| architecture into `lora_ft.json`, then add: | ||
|
|
||
| ```json | ||
| { | ||
| "model": { | ||
| "type": "dpa4", | ||
| "lora": { | ||
| "rank": 16, | ||
| "alpha": 16.0 | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| Run: | ||
|
|
||
| ```bash | ||
| dp --pt train lora_ft.json --finetune pretrained.pt | ||
|
SchrodingersCattt marked this conversation as resolved.
|
||
| ``` | ||
|
|
||
| The JSON fragment above is not a complete training input. Adapt the full public | ||
| example at `../../examples/water/dpa4/lora_ft.json`, but copy the architecture | ||
| from the actual source checkpoint before adding `model.lora`. The compact | ||
| `examples/water/dpa4/lmp/pretrained.pt` does not match the larger architecture | ||
| in the maintained `lora_ft.json` and must not be paired with it unchanged. Do | ||
| not add `--use-pretrain-script` to this LoRA command unless a targeted test | ||
| confirms that the intended LoRA configuration is retained. | ||
|
|
||
| ## Monitor and validate | ||
|
|
||
| Monitor `lcurve.out` for non-finite values and train/validation divergence. | ||
| Select a checkpoint using validation data, then evaluate the selected checkpoint | ||
| on the complete held-out test systems. Report energy and force errors, plus | ||
| virial errors when those labels are part of the task. | ||
|
|
||
| ## Export and test | ||
|
|
||
| DPA4/SeZM uses the `.pt2` AOTInductor export path rather than the conventional | ||
| PyTorch `.pth` freeze path: | ||
|
|
||
| ```bash | ||
| dp --pt freeze -c ckpt/model.ckpt.pt -o finetuned_model | ||
| dp test -m finetuned_model.pt2 -s /path/to/test_system -n 30 | ||
| ``` | ||
|
|
||
| The freeze command detects DPA4/SeZM and writes `finetuned_model.pt2`. Validate | ||
| the exported archive in the target environment before deployment. | ||
|
|
||
| For a multi-task checkpoint, freeze the selected head explicitly: | ||
|
|
||
| ```bash | ||
| dp --pt freeze -c ckpt/model.ckpt.pt -o finetuned_model --head SELECTED_BRANCH | ||
| ``` | ||
|
|
||
| The resulting `.pt2` contains the selected single head; do not pass a branch | ||
| again when loading that archive. | ||
|
|
||
| ## Checklist | ||
|
|
||
| - [ ] The stored descriptor identifies DPA4/SeZM; the `.pt` suffix was not used as proof. | ||
| - [ ] The checkpoint source, checksum, DeePMD-kit revision, architecture, and type map are recorded. | ||
| - [ ] The intended branch is explicit for a multi-task checkpoint. | ||
| - [ ] Training, validation, and held-out test systems are separate. | ||
| - [ ] The input architecture is compatible with the checkpoint. | ||
| - [ ] Standard fine-tuning versus LoRA was selected from the task layout and domain shift. | ||
| - [ ] Active LoRA state versus a merged best checkpoint is interpreted correctly. | ||
| - [ ] LoRA uses a complete base configuration and is not silently overwritten. | ||
| - [ ] Training and held-out metrics are finite and reported with units. | ||
| - [ ] The selected `.pt` checkpoint was exported to and tested as `.pt2`. | ||
|
|
||
| ## References | ||
|
|
||
| - [DPA4 model and LoRA documentation](https://docs.deepmodeling.com/projects/deepmd/en/latest/model/dpa4.html) | ||
| - [Fine-tuning documentation](https://docs.deepmodeling.com/projects/deepmd/en/latest/train/finetuning.html) | ||
| - [Show model information](https://docs.deepmodeling.com/projects/deepmd/en/latest/model/show-model-info.html) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.