Skip to content

Commit 76cf1e0

Browse files
speediedanspeediedan
committed
merge recent PRs from main (#17, #18), added yaml (multi)representer for PretrainedConfig object types, update docker image to use cuda 12.6.3
1 parent cb5de7f commit 76cf1e0

File tree

9 files changed

+27
-18
lines changed

9 files changed

+27
-18
lines changed

‎.github/ISSUE_TEMPLATE/bug_report.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -39,9 +39,9 @@ python collect_env_details.py
3939
You can also fill out the list below manually.
4040
-->
4141

42-
- Fine-Tuning Scheduler Version (e.g., 2.7.0):
43-
- Lightning Version (e.g., 2.7.0):
44-
- PyTorch Version (e.g., 2.7.0):
42+
- Fine-Tuning Scheduler Version (e.g., 2.6.0):
43+
- Lightning Version (e.g., 2.6.0):
44+
- PyTorch Version (e.g., 2.6.0):
4545
- Python version (e.g., 3.12):
4646
- OS (e.g., Linux):
4747
- CUDA/cuDNN version:

‎CITATION.cff‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ date-released: 2022-02-04
66
authors:
77
- family-names: "Dale"
88
given-names: "Dan"
9-
version: 2.7.0
9+
version: 2.6.0
1010
identifiers:
1111
- description: "Fine-Tuning Scheduler (all versions)"
1212
type: doi

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -145,7 +145,7 @@ To ensure maximum stability, the latest Lightning patch release fully tested wit
145145
<details>
146146
<summary>Current build statuses for Fine-Tuning Scheduler </summary>
147147

148-
| System / (PyTorch/Python ver) | 2.3.1/3.9 | 2.7.0/3.9, 2.7.0/3.12 |
148+
| System / (PyTorch/Python ver) | 2.3.1/3.9 | 2.6.0/3.9, 2.6.0/3.12 |
149149
| :---------------------------: | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: |
150150
| Linux \[GPUs\*\*\] | - | [![Build Status](https://dev.azure.com//speediedan/finetuning-scheduler/_apis/build/status/Multi-GPU%20&%20Example%20Tests?branchName=main)](https://dev.azure.com/speediedan/finetuning-scheduler/_build/latest?definitionId=1&branchName=main) |
151151
| Linux (Ubuntu 22.04) | [![Test](https://github.com/speediedan/finetuning-scheduler/actions/workflows/ci_test-full.yml/badge.svg?branch=main&event=push)](https://github.com/speediedan/finetuning-scheduler/actions/workflows/ci_test-full.yml) | [![Test](https://github.com/speediedan/finetuning-scheduler/actions/workflows/ci_test-full.yml/badge.svg?branch=main&event=push)](https://github.com/speediedan/finetuning-scheduler/actions/workflows/ci_test-full.yml) |

‎dockers/fts-az-base/Dockerfile‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,8 +11,8 @@
1111
# limitations under the License.
1212

1313
ARG PYTHON_VERSION=3.12
14-
ARG PYTORCH_VERSION=2.7.0
15-
ARG LIGHTNING_VERSION=2.7
14+
ARG PYTORCH_VERSION=2.6.0
15+
ARG LIGHTNING_VERSION=2.6
1616
ARG CUST_BASE
1717

1818
FROM speediedan/finetuning-scheduler:base-${CUST_BASE}py${PYTHON_VERSION}-pt${PYTORCH_VERSION}-pl${LIGHTNING_VERSION}

‎dockers/release/Dockerfile‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,8 +11,8 @@
1111
# limitations under the License.
1212

1313
ARG PYTHON_VERSION=3.12
14-
ARG PYTORCH_VERSION=2.7.0
15-
ARG LIGHTNING_VERSION=2.7
14+
ARG PYTORCH_VERSION=2.6.0
15+
ARG LIGHTNING_VERSION=2.6
1616
ARG CUST_BASE
1717

1818
FROM speediedan/finetuning-scheduler:base-${CUST_BASE}py${PYTHON_VERSION}-pt${PYTORCH_VERSION}-pl${LIGHTNING_VERSION}

‎scripts/build_fts_env.sh‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -94,10 +94,10 @@ base_env_build(){
9494
torchvision_dev_ver=${torch_dev_ver}
9595
fi
9696
# temporarily remove torchvision until it supports cu128 in nightly binary
97-
pip install ${pip_install_flags} --pre torch==2.7.0.${torch_dev_ver} --index-url https://download.pytorch.org/whl/nightly/cu128
98-
#pip install ${pip_install_flags} --pre torch==2.7.0.${torch_dev_ver} torchvision==0.22.0.${torchvision_dev_ver} --index-url https://download.pytorch.org/whl/nightly/cu128
97+
pip install ${pip_install_flags} --pre torch==2.6.0.${torch_dev_ver} --index-url https://download.pytorch.org/whl/nightly/cu128
98+
#pip install ${pip_install_flags} --pre torch==2.6.0.${torch_dev_ver} torchvision==0.22.0.${torchvision_dev_ver} --index-url https://download.pytorch.org/whl/nightly/cu128
9999
elif [[ $torch_test_channel -eq 1 ]]; then
100-
pip install ${pip_install_flags} --pre torch==2.7.0 torchvision==0.22.0 --index-url https://download.pytorch.org/whl/test/cu128
100+
pip install ${pip_install_flags} --pre torch==2.6.0 torchvision==0.22.0 --index-url https://download.pytorch.org/whl/test/cu128
101101
else
102102
pip install ${pip_install_flags} torch torchvision --index-url https://download.pytorch.org/whl/cu128
103103
fi

‎src/finetuning_scheduler/fts_supporters.py‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1297,7 +1297,6 @@ class ScheduleImplMixin(ABC):
12971297
ft_schedule: Optional[Union[str, dict]]
12981298
reinit_optim_cfg: Optional[Dict]
12991299
reinit_lr_cfg: Optional[Dict]
1300-
#log_dir: Optional[Union[str, os.PathLike]]
13011300
max_depth: int
13021301
_msg_cache: Set
13031302
_fts_state: FTSState

‎src/fts_examples/profiling/extended_profiler.py‎

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@
1515
from typing import Dict, List, Optional, Union
1616

1717
import torch
18-
#from lightning.fabric.utilities.imports import _TORCH_GREATER_EQUAL_2_4
18+
from lightning.fabric.utilities.imports import _TORCH_GREATER_EQUAL_2_4
1919
from lightning.fabric.accelerators.cuda import is_cuda_available
2020
from lightning.fabric.utilities import rank_zero_info
2121
from lightning.pytorch.profilers.pytorch import _KINETO_AVAILABLE, PyTorchProfiler
@@ -62,6 +62,16 @@ def _set_activities(self, kwargs) -> List["ProfilerActivity"]:
6262
activities: List["ProfilerActivity"] = []
6363
if not _KINETO_AVAILABLE:
6464
return activities
65+
if _TORCH_GREATER_EQUAL_2_4:
66+
activities.append(ProfilerActivity.CPU)
67+
if is_cuda_available():
68+
activities.append(ProfilerActivity.CUDA)
69+
else:
70+
# `use_cpu` and `use_cuda` are deprecated in PyTorch >= 2.4
71+
if kwargs.get("use_cpu", True):
72+
activities.append(ProfilerActivity.CPU)
73+
if kwargs.get("use_cuda", is_cuda_available()):
74+
activities.append(ProfilerActivity.CUDA)
6575
activities.append(ProfilerActivity.CPU)
6676
if is_cuda_available():
6777
activities.append(ProfilerActivity.CUDA)

‎tests/test_finetuning_scheduler_callback.py‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1273,7 +1273,7 @@ def test_fts_enforce_p0(tmpdir, init_lr_key, p0_params):
12731273
}
12741274

12751275

1276-
@RunIf(skip_windows=True, skip_mac_os=True, min_python="3.12")
1276+
@RunIf(skip_windows=True, skip_mac_os=True, min_torch="2.4.0", min_python="3.12")
12771277
def test_fts_dynamo_enforce_p0(tmpdir, boring_ft_schedule):
12781278
"""Inspect the scheduled fine-tuning training path in the context of dynamo to ensure thawing schedule phase 0
12791279
is enforced."""
@@ -1445,7 +1445,7 @@ def test_fts_callback_resume(tmpdir, ckpt_set, recwarn, diff_dirpath: bool, trai
14451445
assert not unexpected, tuple(w.message.args[0] + ":" + w.filename + ":" + str(w.lineno) for w in unexpected)
14461446

14471447

1448-
@RunIf(skip_windows=True, skip_mac_os=True, min_python="3.12")
1448+
@RunIf(skip_windows=True, skip_mac_os=True, min_torch="2.4.0", min_python="3.12")
14491449
def test_fts_dynamo_resume(tmpdir, ckpt_set, boring_ft_schedule, recwarn):
14501450
"""Validate scheduled fine-tuning resumption functions as expected with a default dynamo configuration."""
14511451
resume_warns = copy(BASE_EXPECTED_WARNS) + copy(BASE_DYNAMO_EXPECTED_WARNS) \
@@ -1557,7 +1557,7 @@ def test_fts_intrafit(tmpdir, restore_best: bool):
15571557
}
15581558

15591559

1560-
@RunIf(skip_windows=True, skip_mac_os=True, min_python="3.12", standalone=True)
1560+
@RunIf(skip_windows=True, skip_mac_os=True, min_torch="2.4.0", min_python="3.12", standalone=True)
15611561
@pytest.mark.parametrize("restore_best", [True, False], ids=["default", "norestorebest"])
15621562
def test_fts_dynamo_intrafit(tmpdir, boring_ft_schedule, restore_best: bool):
15631563
"""Inspect scheduled fine-tuning state within the training process to ensure it is taking the expected path in
@@ -2844,7 +2844,7 @@ def test_fts_multi_ddp(tmpdir, boring_ft_schedule, explicit_mode):
28442844
assert finetuningscheduler_callback.curr_depth == finetuningscheduler_callback.max_depth
28452845

28462846

2847-
@RunIf(standalone=True, min_cuda_gpus=2, skip_windows=True, skip_mac_os=True, min_python="3.12")
2847+
@RunIf(standalone=True, min_cuda_gpus=2, skip_windows=True, skip_mac_os=True, min_torch="2.4.0", min_python="3.12")
28482848
def test_fts_multi_ddp_dynamo(tmpdir, boring_ft_schedule):
28492849
"""Validate :class:`~finetuning_scheduler.FinetuningScheduler` functions properly in a supported 'ddp'
28502850
distributed context with default dynamo usage."""

0 commit comments

Comments
 (0)