Skip to content

Commit 8b90425

Browse files
Add pixi install and dev instructions to README (#77)
* Add pixi install and dev instructions to README * Add CI test config updates * Add manifest sharing instructions * Remove manifest path statement in favour of note.
1 parent fb75225 commit 8b90425

1 file changed

Lines changed: 87 additions & 8 deletions

File tree

README.md

Lines changed: 87 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ This repository is maintained using [OET's soft-fork strategy](https://open-ener
2929
* `cutouts`: will store raw weather data cutouts from `atlite` (does not exist initially)
3030
* `data`: includes input data that is not produced by any `snakemake` rule
3131
* `doc`: includes all files necessary to build the `readthedocs` documentation of PyPSA-Eur
32-
* `envs`: includes all the `mamba` environment specifications to run the workflow
32+
* `envs`: includes backup `conda` environments if `pixi` installation does not work.
3333
* `logs`: will store log files (does not exist initially)
3434
* `notebooks`: includes all the `notebooks` used for ad-hoc analysis
3535
* `report`: contains all files necessary to build the report; plots and result files are generated automatically
@@ -44,25 +44,104 @@ This repository is maintained using [OET's soft-fork strategy](https://open-ener
4444

4545
Clone the repository:
4646

47-
git clone https://github.com/open-energy-transition/{{repository}}
47+
```sh
48+
git clone https://github.com/open-energy-transition/{{repository}}
49+
```
4850

49-
You need [mamba](https://mamba.readthedocs.io/en/latest/) to run the analysis. Users may also prefer to use [micromamba](https://mamba.readthedocs.io/en/latest/installation/micromamba-installation.html) or [conda](https://docs.conda.io/projects/conda/en/latest/index.html). Using `mamba`, you can create an environment from within you can run it:
51+
You need [pixi](https://pixi.sh/latest/) to run the analysis.
52+
Once installed, activate your pixi environment in a terminal session:
5053

51-
mamba env create -f environment.yaml
54+
```sh
55+
pixi shell
56+
```
5257

53-
Activate the newly created `{{project_short_name}}` environment:
58+
>[!NOTE]
59+
>`pixi` will create a distinct environment in every project directory, even if you have identical copies of a project cloned locally.
60+
>As there is a common system-level package cache, `pixi` efficiently conserves disk space in such cases.
61+
62+
>[!TIP]
63+
>If `pixi` isn't working, you can install from one of the fallback `conda` environment files found in `envs`.
64+
>For more details see [the PyPSA-Eur installation guide](https://pypsa-eur.readthedocs.io/en/latest/installation.html).
65+
66+
### Extra soft-fork dependencies
67+
68+
If you add dependencies to your project, we recommend you add them to a [new `pixi` environment](https://pixi.sh/v0.21.1/features/multi_environment/#feature-environment-set-definitions).
69+
For instance, if you need access to `plotly`, want to pin the version of gurobi you are using, and want to add a PyPI dependency:
70+
71+
```sh
72+
pixi add -f {{ project_short_name }} "gurobi<13" "plotly"
73+
pixi add -f {{ project_short_name }} --pypi pypsa-explorer
74+
```
75+
76+
This will create these entries in your `pixi.toml`
77+
78+
```toml
79+
[feature.{{ project_short_name }}.pypi-dependencies]
80+
pypsa-explorer = "*"
81+
82+
[feature.{{ project_short_name }}.dependencies]
83+
gurobi = "<13"
84+
plotly = "*"
85+
```
86+
87+
Then, you can create an environment from this feature in `pixi.toml`:
88+
89+
```toml
90+
[environments]
91+
...
92+
{{ project_short_name }} = [{{ project_short_name }}]
93+
```
94+
95+
These dependencies will be combined with the core PyPSA-Eur dependencies and can be accessed by calling:
96+
97+
```sh
98+
pixi shell -e {{ project_short_name }}
99+
```
100+
101+
#### Updating CI tests
102+
103+
To run CI tests using your environment you should add the `test` feature to it and create test tasks for your environment, e.g.:
104+
105+
```toml
106+
[feature.{{ project_short_name }}.tasks]
107+
{{ project_short_name }}-test = """
108+
snakemake --configfile config/config.{{ project_short_name }}.default.yaml -n &&
109+
"""
110+
[environments]
111+
...
112+
{{ project_short_name }} = ["test", {{ project_short_name }}]
113+
```
114+
115+
And then update `.github/workflows/test.yaml` to run that test:
116+
117+
```yaml
118+
- name: Run project-specific snakemake test workflows
119+
run: |
120+
pixi run {{ project_short_name }}-test
121+
```
122+
123+
If you also add your own unit tests, update the unit test runner to use your environment as well:
124+
125+
```yaml
126+
- name: Run unit tests
127+
run: |
128+
pixi run -e {{ project_short_name }} unit-tests
129+
```
54130
55-
mamba activate {{project_short_name}}
56131
57132
## 2. Run the analysis
58133
59-
snakemake -call
134+
```sh
135+
snakemake -call
136+
```
60137

61138
This will run all analysis steps to reproduce results and build the report.
62139

63140
To generate a PDF of the dependency graph of all steps `resources/dag.pdf` run:
64141

65-
snakemake -c1 dag
142+
```sh
143+
snakemake -c1 dag
144+
```
66145

67146
<sup>*</sup> Open Energy Transition (g)GmbH, Königsallee 52, 95448 Bayreuth, Germany
68147

0 commit comments

Comments
 (0)