Skip to content
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/docs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ jobs:
- name: Install dependencies
run: |
python -m pip install uv==0.2.5
uv pip install --system -c constraints.txt -e fme[docs]
uv pip install --system -c constraints.txt -e .[docs]
- name: Build docs
run: |
cd fme/docs && make doctest html
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ jobs:
- name: Install dependencies
run: |
python -m pip install uv==0.2.5
uv pip install --system -c constraints.txt -e fme[dev]
uv pip install --system -c constraints.txt -e .[dev]
- name: Run pytest
run: |
make test
16 changes: 15 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
VERSION ?= $(shell git rev-parse --short HEAD)
IMAGE ?= fme
ENVIRONMENT_NAME ?= fme
DEPLOY_TARGET ?= pypi

build_docker_image:
docker build -f docker/Dockerfile -t $(IMAGE):$(VERSION) .
Expand All @@ -12,7 +13,20 @@ enter_docker_image: build_docker_image
create_environment:
conda create -n $(ENVIRONMENT_NAME) python=3.10 pip
conda run --no-capture-output -n $(ENVIRONMENT_NAME) python -m pip install uv==0.2.5
conda run --no-capture-output -n $(ENVIRONMENT_NAME) uv pip install -c constraints.txt -e fme[dev]
conda run --no-capture-output -n $(ENVIRONMENT_NAME) uv pip install -c constraints.txt -e .[dev]

test:
pytest --durations 20 .

# For maintainer use only
# requires fme[deploy] to be installed

build_pypi:
rm -rf dist
python -m build

deploy_pypi: build_pypi
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

User needs an API key to push, so it's safe to include in make target (i.e., can't accidentally push without access, and pushing same version fails)

twine upload --repository $(DEPLOY_TARGET) dist/*

deploy_test_pypi: DEPLOY_TARGET = testpypi
deploy_test_pypi: deploy_pypi
2 changes: 2 additions & 0 deletions fme/deploy-requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
build
twine
4 changes: 2 additions & 2 deletions fme/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@ numpy<2
wandb
tensorly
tensorly-torch
xarray
xarray>=2023.2.0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly out of curiosity.. was there a particular feature that required this version?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the original setup.py python>=3.8 the tests were failing without xarray above that version (which also forced the bump to python>=3.9). I removed since the python requirement automatically solves the xarray version correctly.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, that is helpful explanation. I'm +1 on requiring python 3.9 given EOL for 3.8.

dacite
torch
torch-harmonics@git+https://github.com/NVIDIA/torch-harmonics.git@8826246cacf6c37b600cdd63fde210815ba238fd
torch-harmonics==0.6.2
zarr
gcsfs
s3fs
Expand Down
42 changes: 0 additions & 42 deletions fme/setup.py

This file was deleted.

38 changes: 38 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
[build-system]
requires = ["setuptools", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "fme"
version = "2024.10.3"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This version number was just for testing things out on testpypi. I'm thinking we should use the 2023.12.0 version that's tagged in the repo for actual pypi, but I wanted to check before actually committing a version.

description = "Train and evaluate weather/climate model emulators"
readme = "README.md"
requires-python = ">=3.9"
license = {file = "LICENSE"}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to move the pyproject.toml to the top level in order to dynamically link the README and LICENSE files. I did try to leave it under fme, but it's not permitted to access parent directories for dynamic builds.

@mcgibbon doesn't recall if there was a requirement that setup.py and other files exist in fme and not at the top level, but did mention that it would make things more difficult for having multiple packages. Could be something worth discussing.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does fundamentally change our repo so that it is now the repo of a single python package, as opposed to being a repo containing a single python package (but which could contain more). Probably worth discussing in our weekly fme sync.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would lean to maintaining the current repo structure. Maybe we could duplicate the LICENSE within fme and make an fme-specific README within that folder? The benefit of current structure is we can have the scripts folder with data processing etc scripts that are distinct from the fme package.

authors = [
{name = "Ai2 Climate Modeling", email = "[email protected]"}
]
keywords = ["weather", "climate", "machine learning", "emulation"]
classifiers = [
"Intended Audience :: Science/Research",
"Development Status :: 4 - Beta",
"License :: OSI Approved :: Apache Software License",
"Programming Language :: Python :: 3",
"Topic :: Scientific/Engineering :: Artificial Intelligence"
]
dynamic = ["dependencies", "optional-dependencies"]
[project.urls]
Homepage = "https://github.com/ai2cm/ace"
Documentation = "https://ai2-climate-emulator.readthedocs.io/"

[tool.setuptools.dynamic]
dependencies = { file = "fme/requirements.txt" }
optional-dependencies.dev = { file = "fme/dev-requirements.txt" }
optional-dependencies.docs = { file = "fme/docs/requirements.txt" }
optional-dependencies.deploy = { file = "fme/deploy-requirements.txt" }

[tool.setuptools.packages.find]
where = ["fme"]

[tool.setuptools.exclude-package-data]
fme = ["*.tests", "*.tests.*", "tests.*", "tests"]
Loading