Skip to content

Conversation

@mherrmann3
Copy link
Contributor

@mherrmann3 mherrmann3 commented Oct 18, 2024

I went through the docs and executed the tutorials; I really enjoyed it. The only critical fix to make models run on my system is commit 'Run models whose absolute path contains a whitespace'. The rest are purely suggestions. @pabloitu Feel free to take what you need 😏

Further comments that require more elaboration follow below 🙂


Note: comments starting with 'PR:' are already addressed in this PR; for the other ones I did not yet provide code suggestions either because they would be too difficult/complex for me or because you have to decide by yourself how you'd address it)

  • I installed floatCSEP both via the latest version from github and the stable release from conde-forge; both install fine.

    • but strange: the latest version from github is at v0.1.5 (when installed with pip), the one from conda-forge at v0.2.0. Not sure why, even though both end at commit df2dbe0. I expected the opposite or at least both on v0.2.0.
      btw: floatcsep/__init__.py is at __version__ = "0.1.4"
  • installation instructions:

    • PR (commit): conda env create -n csep_env expects an environment.yml. So this can only be executed inside the floatCSEP folder; what you meant (in both cases) is conda create -n csep_env (without env). To keep using the one with env at least for section Latest Version > 1. Using conda, you only have to reorder your commands, that is, create env after cd floatcsep; this has a bonus: all dependencies are installed by conda, not pip. I did that in my PR and provided further suggestions. I guess this solves the issue of Toño.
    • the README section "Installing floatCSEP" is not in line with the docs (for instance, no need to install pycsep manually, since it's already specified as dependency); I'd only mention the conda-forge install option and link to the docs for alternative ways.
  • markdown reports:

    • PR (commit): I'd output relative instead of absolute links to figures; in this way the whole results folder becomes "portable" and self-contained, that is, can be moved somewhere else or to other devices without breaking links.
    • to consume reports without a markdown-compatible editor, also generate a .html file, which can be viewed with a browser; either write out directly or convert from markdown using a separate package like markdown or pygments. However, those won't be consumed by GitHub.
    • to avoid what Francesco observed when viewing the markdown report in a wide window, specify a reasonable width= parameter for the 'Input catalog' figures (I like 800). (I didn't touch the code because I didn't know how to do it for this figure table.)
    • PR (commit): the test results should get their own ##-heading, otherwise they hierarchically appear under "Authoritative Data".
  • plots/figures:

    • the 'catalog' figure: it has super-mini axis labels and excessive whitespace, which creates large margins in the report.
    • the 'Sequential *' plots: for better reference, the relative year in the x-axis label (0, 1, ..) might be converted into a corresponding date axis (2010, 2011, ...).
  • Case D:

    • now that the used forecasts become more interesting, perhaps mention that they are plotted in 2020-01-01_2021-01-01/figures (together with the test results). Also, it might we worthwhile to include them in the report (e.g., a section between input data and results: ## Forecast(s)).
  • Case E:

    • Setting use_db: False is indeed muuuch slower (~10x).
      • Q1: Why not setting use_db: True as the default in init of TimeIndependentModel?
      • Q2: The HDF5Serializer will eventually be replaced by dbcsep, correct?
    • Post-process & Plot command sections: the forecast plots are not in {timewindow}/forecasts but (only) in 2010-01-01_2022-01-01/figures (i.e., only the last date and together with the test results). Again, it might we worthwhile to include them in the report.
    • Plot command section: actually, the provided colormap was already used in the config.xml. I think you wanted to specify a different one, e.g., 'viridis'?
    • Q: I spotted a case_e.py file in the folder; does it have a purpose?
  • Case F:

    • test results in the report only refer to the last date
      • Q: being T-D, perhaps create a report for each date? Otherwise, mention this aspect.
    • likewise and as before, the forecast is only plotted for the last date (2016-11-20_2016-11-21/figures). This made sense for time-independent models, but not so much for a T-D forecast. Also, it might we worthwhile to mention those outputs again in the doc of case F (and perhaps include them in the report).
  • Case G:

    • I believe using build: venv is incompatible with running floatCSEP in a conda env; this warning is printed and it doesn't find pymock. In the following, I use build: conda (using mamba, which it nicely detects) and assume that build: venv won't work inside an active conda env; not sure if it works on your system.
      • you may simply omit build: venv to use conda by default (since most users will work in a conda env per install instructions); if you do, don't forget to adapt the 'Important' box;
      • perhaps mention that this conda environment must be removed again manually if not needed anymore (and how: conda remove -n pymock_<hash> --all);
      • in the 'Important' box, mention that build: venv won't work when already working in a conda environment (if it is truly so).
    • PR (commit): if an experiment (or pymock) has an absolute path that contains a whitespace, nothing will run; it is important to put the args_file path into quotation marks (see commit)
      • btw: with build: venv, additionally also the activate_script path must be set in quotation marks (see commit)
    • Regarding custom plots, it would be nice to automatically include them into the report in a separate section below the test results (## Custom plots).
    • same as before:
      • mention that the test results in the report only refer to the last date (and perhaps create a report for each date?)
      • the forecast is only plotted for the last date (2012-06-13_2012-06-20/figures) + it might we worthwhile to mention those outputs again in the doc of case G (and perhaps include them in the report).
  • Case H:

    • Like Toño & Francesco, I also have problems installing the etas model. conda seems to error on installing the pynverse package. Perhaps the newer v0.1.4.6 fixes it?
  • misc:

    • PR (commit) I allowed myself to make minor wording and typographic edits in tutorials E,F,G,H (+ provide & link some references).
    • I found floatcsep, floatCSEP, and floatCSEP in the docs; I'd homogenize it for "brand recognition" 😄
      • same for the logo: different in the README and docs

@mherrmann3 mherrmann3 force-pushed the mh-feedback branch 2 times, most recently from cf44e8f to 43f370b Compare October 19, 2024 09:28
(instead of absolute ones)
... in this way the whole `results` folder becomes "portable" and self-contained, that is, it can be moved somewhere else or to other devices without breaking links.
…ting relative paths into the report.

tests: adapted tests to this PR changes
@pabloitu
Copy link
Collaborator

pabloitu commented Nov 11, 2025

Excellent super thorough PR @mherrmann3, sorry I didn't addressed earlier. I included all of your changes, and updated some of them to the recent release. I answer some questions/comments and create a new issues from them:

* I installed floatCSEP both via the latest version from github and the stable release from conde-forge; both install fine.
  
  * but strange: the latest version from github is at v0.1.5 (when installed with pip), the one from conda-forge at v0.2.0. Not sure why, even though both end at commit [df2dbe0](https://github.com/cseptesting/floatcsep/commit/df2dbe0). I expected the opposite or at least both on v0.2.0.
    btw: `floatcsep/__init__.py` is at `__version__ = "0.1.4"`

I implemented automatic versioning, so this should be fixed

* markdown reports:
  
  * **PR** ([commit](https://github.com/cseptesting/floatcsep/pull/45/commits/6ceff662)): I'd output _relative_ instead of absolute links to figures; in this way the whole `results` folder becomes "portable" and self-contained, that is, can be moved somewhere else or to other devices without breaking links.

I kept the registry logic to work with absolute paths, but modified the report generation to inject only relative paths to the report.

  * to consume reports without a markdown-compatible editor, also generate a `.html` file, which can be viewed with a browser; either write out directly or convert from markdown using a separate package like `markdown` or [`pygments`](https://pygments.org/docs/quickstart/). However, those won't be consumed by GitHub.
  * to avoid what Francesco observed when viewing the markdown report in a wide window, specify a reasonable `width=` parameter for the _'Input catalog'_ figures (I like 800). (I didn't touch the code because I didn't know how to do it for [this figure table](https://github.com/cseptesting/floatcsep/blob/df2dbe00/floatcsep/postprocess/reporting.py#L59).)

I will implement this changes in the following version bump. Thank you for the ideas!

* plots/figures:
  
  * the 'catalog' figure: it has super-mini axis labels and excessive whitespace, which creates large margins in the report.
  * the 'Sequential *' plots: for better reference, the relative year in the x-axis label (0, 1, ..) might be converted into a corresponding date axis (2010, 2011, ...).

This should be fixed with the pycsep refactoring we will push in v0.8.0

* Case D:
  
  * now that the used forecasts become more interesting, perhaps mention that they are plotted in `2020-01-01_2021-01-01/figures` (together with the test results). Also, it might we worthwhile to include them in the report (e.g., a section between input data and results: `## Forecast(s)`).

I think the default should be plotting forecasts in the report. Need to think though how it will be managed when a lot of forecasts were created. In any case, these should be available in full within the dataapp

* Case E:
  
  * Setting `use_db: False` is indeed muuuch slower (~10x).
    
    * **Q1**: Why not setting `use_db: True` as the default in **init** of `TimeIndependentModel`?
    * **Q2**: The `HDF5Serializer` will eventually be replaced by [`dbcsep`](https://github.com/cseptesting/dbcsep), correct?

I removed the hdf5 usage alltogether, because eventually all of the data should be contained in an experiment hdf5

  * _Post-process_ & _Plot command_ sections: the forecast plots are _not_ in `{timewindow}/forecasts` but (only) in `2010-01-01_2022-01-01/figures` (i.e., only the last date and together with the test results). Again, it might we worthwhile to include them in the report.

I need to check this

  * _Plot command_ section: actually, the provided colormap was already used in the _config.xml_. I think you wanted to specify a different one, e.g., `'viridis'`?
  * **Q**: I spotted a `case_e.py` file in the folder; does it have a purpose?

I actually don't know. I left it as example for setting up an experiment, since it is always good to have access to the python API. However, I guess that deserves its own tutorial.

* Case F:
  
  * test results in the report only refer to the last date
    
    * **Q**: being T-D, perhaps create a report for each date? Otherwise, mention this aspect.
  * likewise and as before, the forecast is only plotted for the last date (`2016-11-20_2016-11-21/figures`). This made sense for time-independent models, but not so much for a T-D forecast. Also, it might we worthwhile to mention those outputs again in the doc of case F (and perhaps include them in the report).

Good questions, but haven't reached a decision yet. With too many timewindows, the report could become unmanageable. Although the report is a good output product anyways, regardless if a dataapp is also created.

* Case G:
  
  * I believe using `build: venv` is incompatible with running floatCSEP in a conda env; [this warning](https://github.com/cseptesting/floatcsep/blob/df2dbe00/floatcsep/infrastructure/environments.py#L351) is printed and it doesn't find `pymock`. In the following, I use `build: conda` (using mamba, which it nicely detects) and assume that `build: venv` won't work inside an active conda env; not sure if it works on your system.    
    * you may simply omit `build: venv` to use conda by default (since most users will work in a conda env per install instructions); if you do, don't forget to adapt the _'Important'_ box;

It does. I raise a warning now if different environment managers are used... Still needs to digest it, but perhaps it doesn;'t make sense to support venv (not the most reproducible anyways).

  * Regarding custom plots, it would be nice to automatically include them into the report in a separate section below the test results (`## Custom plots`).

Right, will work on this in the next report release.

* Case H:
  
  * Like Toño & Francesco, I also have problems installing the etas model. conda seems to error on installing the [pynverse](https://pypi.org/project/pynverse/) package. Perhaps the newer v0.1.4.6 fixes it?

Case H was revamped, as well as the ETAS and Pymock repositories (now the run function should be only "pymock", the "args_file" will be part of the experiment-model interface.

  * I found `floatcsep`, `floatCSEP`, and **floatCSEP** in the docs; I'd homogenize it for "brand recognition" 😄
    
    * same for the logo: different in the README and docs

Needs to be addressed also.

Thank you!

Closes #41

@pabloitu pabloitu closed this Nov 11, 2025
@pabloitu pabloitu reopened this Nov 11, 2025
@pabloitu pabloitu merged commit 3a93e6f into cseptesting:main Nov 11, 2025
7 checks passed
@pabloitu pabloitu mentioned this pull request Nov 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants