Track and write measurement time for area

[gerritholl]

Track measurement times such that a representative measurement time can be estimated and written for an area.

To include valid time in the filename:

• Add reader_kwargs: {"track_time": True} to Scene.__init__, and
• add resample_coords=True to Scene.resample, and
• add dynamic_fields={"valid_time"} to Scene.save_datasets, and
• include {valid_time} in the filename pattern passing to Scene.save_datasets.

When storing as ninjogeotiff, time coordinates will be averaged and stored in the ninjo_ValidTimeID tag.

An example is included in the PR under doc/source/examples/valid_time.rst.

The PR also does some refactoring in Scene._resampled_scene, on top of #3178, which should be merged first.

Next steps, either within this PR or within one or more later PRs:

• Add other readers
• Add other writers
• Add a way to include the valid time in the filename

import os
from satpy import Scene
from glob import glob
from satpy.utils import debug_on; debug_on()
seviri_files = glob("/media/nas/x21308/scratch/SEVIRI/202103300900/H-000*")
sc = Scene(filenames={"seviri_l1b_hrit": seviri_files}, reader_kwargs={"track_time": True})
sc.load(["IR_108"])
ls = sc.resample("nqeuro3km", resample_coords=True, resampler="gradient_search")
ls.save_dataset(
        "IR_108",
        "{platform_name}-{sensor}-{name}-{area.area_id}-{start_time:%Y%m%d%H%M}-{valid_time:%Y%m%d%H%M%S}-geotiff.tif",
        writer="ninjogeotiff",
        ChannelID="IR 10.8",
        DataType="GORN",
        PhysicValue="Temperature",
        PhysicUnit="K",
        SatelliteNameID="METEOSAT Europe-Atlantic",
        fill_value=0,
        dynamic_fields={"valid_time"})

• Closes Write "valid time" to filename or headers #3161
• Tests added
• Fully documented
• Add your name to AUTHORS.md if not there already

[codecov]

Codecov Report

❌ Patch coverage is 94.41624% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 96.33%. Comparing base (3c5f8ba) to head (da329f4).
⚠️ Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
satpy/scene.py	83.82%	11 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3168      +/-   ##
==========================================
- Coverage   96.34%   96.33%   -0.01%     
==========================================
  Files         463      465       +2     
  Lines       58974    59113     +139     
==========================================
+ Hits        56817    56949     +132     
- Misses       2157     2164       +7     

Flag	Coverage Δ
behaviourtests	3.61% <9.64%> (+0.02%)	⬆️
unittests	96.42% <94.41%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[coveralls]

Pull Request Test Coverage Report for Build 16744116940

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 190 of 201 (94.53%) changed or added relevant lines in 13 files are covered.
• No unchanged relevant lines lost coverage.
• Overall first build on feature-valid-time at 96.38%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
satpy/scene.py	57	68	83.82%

Totals
Change from base Build 16697789205:	96.4%
Covered Lines:	56147
Relevant Lines:	58256

---

💛 - Coveralls

[gerritholl]

To track "valid time" through resampling for geostationary satellite data, I'm trying to think of a good way to approach this. My current thinking is that if the user creates the scene with reader_kwargs={"track_time": True}, the file handler adds per-pixel time information to the variable attributes. By putting it inside ancillary variables, dataset_walker and co ensure it gets sliced and resampled along with the parent dataset, so that at the end, an approximate measurement time can be estimated for the resampled scene. But normally ancillary variables are actual variables available in the data file, they remain text attributes until after segments have been concatenated, and then the file handler is called again for every segment, to replace the text label by actual data in ancillary variables. Therefore, I'm not sure if ancillary variables is the right place to put it, or if time should be stored there at all.

[gerritholl]

Maybe it fits better in a coordinate variable, but not sure if it's valid to have those be not-unique.

[gerritholl]

Coordinates can probably be non-unique, but are currently dropped in resampling. In case of nearest neighbour resampling, this is due to

https://github.com/pytroll/pyresample/blob/61628530636b596150f3b0f45d0a9e78a2d80f2c/pyresample/kd_tree.py#L1094-L1095

[gerritholl]

Addressing the CodeScene complaint about Scene._resampled_datasets is too difficult for me. I've looked at it for too long and tried different things, but I do not manage to improve this, sorry.

https://github.com/gerritholl/satpy/blob/329dab5489adf9dfa0b170be3cd16acc52f1674d/satpy/scene.py#L864-L907

I am not able to resolve this as part of this PR.

[gerritholl]

I find it hard to make sense of the CodeScene complaints. Yes, I increased save_image from 90 to 94 LOC, but 57 lines are documentation and the rest is just 13 logical LOCs. And suppressing the complaint does not work ("Error occurred while suppressing! Please try again").

[pnuu]

Some comments and change requests inline.

[djhoese]

Docs still failing:

satpy/readers/seviri_l1b_hrit.py:docstring of satpy.readers.seviri_l1b_hrit.HRITMSGFileHandler:27: WARNING: unknown document: 'valid_time' [ref.doc]
/home/docs/checkouts/readthedocs.org/user_builds/satpy/conda/3168/lib/python3.11/site-packages/satpy/writers/ninjogeotiff.py:docstring of satpy.writers.ninjogeotiff.NinJoGeoTIFFWriter.save_image:74: WARNING: unknown document: 'mean_time' [ref.doc]
/home/docs/checkouts/readthedocs.org/user_builds/satpy/checkouts/3168/doc/source/writing.rst:20: WARNING: unknown document: 'mean_time' [ref.doc]

[gerritholl]

Returned to draft because it fails with reduce_data=False. It also seems to fail with the EffectiveSolarPathlengthCorrector in combination with generate=False…