Dynamic grid carbon intensity

[davidkopp]

Resolves #1330

Documentation: green-coding-solutions/documentation#126

Open questions:

• Is the new way how carbon metrics are calculated ok? (after the normal phase stats were calculated)
• Is it ok to store the carbon intensity as integer and with the unit gram in the database? ElectricityMaps returns also only integer values, however, Carbon-Aware-Computing.com returns also float values (e.g. 270.9). With the current implementation 270.9 would be stored as 270 in the database. Should I leave it, add round() or store it as mg in the database (and show it as mg in the frontend)?
• Is it ok that I also store the static carbon intensity from the config as a measurement_metric with multiple values (at least one value per phase) in the database?
• With the current approach it is ensured that there are always at least one data point per phase and in addition one data point for the measurement start and end point. So at least 7 data points always exist, even if there is only one constant value. Is that ok? The reasoning was, that with this approach it aligns with the general phase stats approach and I always can use the same calculation approach, independent of the number of data points given from the API provider.
• what should be displayed as the "scope" of the new metric in the frontend? system, global, container, etc. don't make sense here I think. Maybe grid or zone for the dynamic value, but for static? The metric name for the dynamic carbon intensity is currently grid_carbon_intensity_dynamic, therefore "dynamic" is displayed as the scope.
• the new metric should be displayed as key metric, correct? which one should be replaced with it?
• EDIT: Is it ok that the grid carbon intensity metrics are only stored in database if phase stats are calculated (dev-no-phase-stats is disabled)?

Screenshots

The following two screenshots were taken from the synthetic test case test_phase_stats_dynamic_grid_carbon_intensity.

[github-actions]

Eco CI Output [RUN-ID: 18030396455]:

🌳 CO2 Data:
City: San Jose, Lat: 37.3388, Lon: -121.8916
IP: 68.220.62.244
CO₂ from energy is: 1.593508540 g
CO₂ from manufacturing (embodied carbon) is: 0.384631545 g
Carbon Intensity for this location: 299 gCO₂eq/kWh
SCI: 1.978140 gCO₂eq / pipeline run emitted

---

Total cost of whole PR so far:

Label	🖥 avg. CPU utilization [%]	🔋 Total Energy [Joules]	🔌 avg. Power [Watts]	Duration [Seconds]
Measurement #1	23.8396	5329.46	3.95	1348.10
Total Run	23.84	5329.46	3.95	1348.10
Additional overhead from Eco CI	N/A	15.33	4.20	3.65

[ArneTR]

Before I review this PR, can you please slim this a little.

I personally feel that the comments in the functions add little to no value and make it very hard for me to read the code. This was ok in dependency_resolver as this is a stand alone component. In GMT I think it should not be there.

Example:

    def get_carbon_intensity_history(self, location: str, start_time: str, end_time: str) -> List[Dict[str, Any]]:
        """
        Fetch carbon intensity history from Elephant service.

        Args:
            location: Location code (e.g., "DE", "ES-IB-MA")
            start_time: Start time in ISO 8601 format (e.g., "2025-09-22T10:50:00Z")
            end_time: End time in ISO 8601 format (e.g., "2025-09-22T10:55:00Z")

        Returns:
            List of carbon intensity data points:
            [{"location": "DE", "time": "2025-09-22T10:00:00Z", "carbon_intensity": 185.0}, ...]

        Raises:
            Exception: On any service error, network issue, or invalid response
        """
        url = f"{self.base_url}/carbon-intensity/history"
        params = {
            'location': location,
            'startTime': start_time,
            'endTime': end_time,
            'interpolate': 'true' # we also want to get data points that are adjacent to the requested time range, to be ensure we always get at least one data point
        }

--- to be slimmed down to

    def get_carbon_intensity_history(self, location: str, start_time: str, end_time: str) -> List[Dict[str, Any]]:
        url = f"{self.base_url}/carbon-intensity/history"
        params = {
            'location': location, #Location code (e.g., "DE", "ES-IB-MA")
            'startTime': start_time, #ISO 8601 format (e.g., "2025-09-22T10:50:00Z")
            'endTime': end_time, #ISO 8601 format (e.g., "2025-09-22T10:55:00Z")
            'interpolate': 'true' # we also want to get data points that are adjacent to the requested time range, to be ensure we always get at least one data point
        }

In this format it is very clear to me still what the function does and what kind of format the arguments have without having this big docs block.

2. Can you also please remove not helpful comments:

Example:

# Bulk insert measurement values
    _bulk_insert_measurement_values(measurement_metric_id, value_timestamp_pairs)

=> The function is already well named. No need for the comment to re-iterate.

3. To answer your questions beforehand

• Is the new way how carbon metrics are calculated ok? (after the normal phase stats were calculated)

  • Have you thought about making it a proper metrics provider? Personally I think it might still get merged in like this but will soon be refactored into that form. So I wanted to explore if you faced any issues there. What I could see are two metric providers, one static and one dynamic, which are mutually excluside. They produce the same value though, a "grid_carbon_intensity". This will then later be used in phase_stats for furhther processing. All API calls needed can happen in the provider.py of the metrics reporter. It has however no actual binary that runs and would thus be similar to the SDIA reporter for instance.

• Is it ok to store the carbon intensity as integer and with the unit gram in the database? ElectricityMaps returns also only integer values, however, Carbon-Aware-Computing.com returns also float values (e.g. 270.9). With the current implementation 270.9 would be stored as 270 in the database. Should I leave it, add round() or store it as mg in the database (and show it as mg in the frontend)?

  • Yes. A round would be nice I guess though.

• Is it ok that I also store the static carbon intensity from the config as a measurement_metric with multiple values (at least one value per phase) in the database?

  • If that makes things easier: Yes. Have not looked at the PR in detail though to give more feedback here.

• With the current approach it is ensured that there are always at least one data point per phase and in addition one data point for the measurement start and end point. So at least 7 data points always exist, even if there is only one constant value. Is that ok? The reasoning was, that with this approach it aligns with the general phase stats approach and I always can use the same calculation approach, independent of the number of data points given from the API provider.

  • Makes sense. Would have probably implemented it the same way.

• what should be displayed as the "scope" of the new metric in the frontend? system, global, container, etc. don't make sense here I think. Maybe grid or zone for the dynamic value, but for static? The metric name for the dynamic carbon intensity is currently grid_carbon_intensity_dynamic, therefore "dynamic" is displayed as the scope.

  • How about 'grid_carbon_intensity_location'. Naming is typically done at the moment as "origin" -> "metric" -> "source" -> "scope" . Scope would be then the location it is derived for. One could argue to separate between "grid_carbon_intensity_api_location" and "grid_carbon_intensity_static_location" to include the source. In the future we could then also add "grid_carbon_intensity_api_market" to have market based values or something. How do you feel about this?

• the new metric should be displayed as key metric, correct? which one should be replaced with it?

• The idea came up when we were talking about moving network to it's own row. Still waiting on a wording idea or design idea from you on this that you feel comfortable with in terms of displaying network carbon emissions :) For this PR please make it only display in the details table. Lets do the key metric thing separate.

• Is it ok that the grid carbon intensity metrics are only stored in database if phase stats are calculated (dev-no-phase-stats is disabled)?

  • This is an edge case I would say as phase stats should only be missing in DEV and depends on the implementation we choose. If it will become a metrics provider this will resolve on it's own. It not just leave it as it is atm.

[davidkopp]

I removed the docstrings and some comments.

Have you thought about making it a proper metrics provider? Personally I think it might still get merged in like this but will soon be refactored into that form. So I wanted to explore if you faced any issues there. What I could see are two metric providers, one static and one dynamic, which are mutually excluside. They produce the same value though, a "grid_carbon_intensity". This will then later be used in phase_stats for furhther processing. All API calls needed can happen in the provider.py of the metrics reporter. It has however no actual binary that runs and would thus be similar to the SDIA reporter for instance.

No, I have not thought about making it a proper metrics provider. It seems that I have interpreted the following design guideline incorrectly:

Grid Intensity is not a classic metric provider. Nor is it really a phase stat. What we is that the CGI shall be collected at the end of the run. This can be done inthe provider.py, because we need the data available in the phase_stats.py as proper metric series
— #1330 (comment)

If you want to have one metric provider that produces a metric called "grid_carbon_intensity" then you might have issues with the display name in the frontend (see frontend/js/helpers/config.js), at least if you want to differentiate between static and dynamic what is currently the case.

A round would be nice I guess though.

Done.

How about 'grid_carbon_intensity_location'. Naming is typically done at the moment as "origin" -> "metric" -> "source" -> "scope" . Scope would be then the location it is derived for. One could argue to separate between "grid_carbon_intensity_api_location" and "grid_carbon_intensity_static_location" to include the source. In the future we could then also add "grid_carbon_intensity_api_market" to have market based values or something. How do you feel about this?

I changed the metric names to grid_carbon_intensity_config_location and grid_carbon_intensity_api_location. I went with config_location instead of static_location as config is a noun like api.

[davidkopp]

The test test_network_providers failed again in the pipeline. 10887248 Bytes (~ 10 MB) were transferred instead of the expected 5-6 MB. The test uses the static carbon value given from the config, so there are no additional network calls introduced by this PR. So I assume the failing of the test is independent of the changes in this PR.