One-Step Pipeline

One-Step Pipeline

Now we have an understanding of the basics of IF and IMPs, lets walks you through a simple IMP that converts energy consumption into carbon emissions using the built‑in Coefficient plugin.

Learning Outcomes:

Demonstrate the process of creating a component, finding some observations and turning them into outputs using plugins
Get you comfortable with the process of expressing a calculation in the form of an IMP, running it using Impact Framework and receiving an output file.

Overview

Imagine you have an application running in the cloud. You want to quantify how much carbon you are emitting by running it.

Boilerplate

First off let's create the boilterplate empty imp file which we will continue to work with for the rest of the module.

There are two top-level sections to the IMP, the context and the tree.

context is everything above the node called tree. This is where you provide all the configuration and metadata for this specific IMP file. This includes loading and configuring the plugins you want to run, this is also where you give the IMP a title and description, like so:

name: energy-to-carbon
description: minimal demo
tags:
initialize:
  plugins:

tree is everything under the tree key. This is where you organize individual components that represent your system. Each component is just a simple block of YAML with a fixed set of keys. For example, you might have blocks of YAML representing each of several servers, or maybe different parts of your stack such as front-end, back-end, database, storage. What you define as a component is left to you and how you want to structure your IMP file, in our example we are going to model a single server in a data center so we will call our component server-1 however you can call your components whatever makes the most sense to you.

tree:
  children:
    server-1:    # <1>
      defaults:  # <2>
      pipeline:  # <3>
      inputs:    # <4>

<1> This is the name of your component, in our case we will just call this server-1. In a real IMP you might name each component more specifically, such as azure-storage-blob-d23re23d1e7, etc
<2> defaults are a convenient place to put any values that are common for all inputs in the inputs array.
<3> pipeline this is where you define which plugins should execute for this component. you have to provide the instance names for the specific plugins you want to execute from the plugin:initialize block. This is where you define the order you want the plugins to execute in. In longer pipelines, the outputs from one plugin will often be inputs to another, so the order that plugins execute is very important.
<4> inputs is an array of observations. This is the data that the model plugins operate on. Every element in inputs has a timestamp and a duration.

So all together our starting IMP file looks like so:

name: energy-to-carbon
description: minimal demo
tags:
initialize:
  plugins:
tree:
  children:
    server-1:
      defaults:
      pipeline:
      inputs:

Create a file called single-server.yml in your workspace and copy and paste the above YAML into it.

IDOMI

The standard development process for a manifest involves these 5 steps. They don't have to be linear, sometimes they are circular or even recursive. But they are a useful mental model and process for how to develop a manifest file. We will be using it extensively throughout this course.

Impacts: What impact(s) do you want to compute?
Dependancies: Start building up an impact dependancy tree.
Observations: What observation(s) can you gather that could be induced to an impact via a methodology?
Methodology: What methodology will you select for inducing the observation(s), into the impact(s)?
Implementation: Implementation of the methodology as a pipeline of plugins.

The general process starts with IDO and ends with MI.

In the example below, the component we want to measure is that of a server.

Impacts

The impact we want to measure is carbon.

Dependancies & Observations

We start to build up a impact dependancy tree, this is a useful tool which can help us figure out what data we need and what methodologies we should be applying.

The root of our tree is carbon like so

☑️ carbon

Note

The ☑️ is a useful indicator to signal that we still have to implement this.

In this simple example we are lucky enough to be using a cloud provider which already measures and exposes the energy consumption of our server in their monitoring environment. So the observation we can gather is energy. Let's add this to our impact dependancy tree like so:

☑️ carbon

🌿 energy

Note

An impact or an impact dependancy is something we need to compute, an observation is something we will use in a computation. Since energy is an observation it also means it's a leaf in our impact dependancy tree. I like to visually denote an observation with 🌿 to distinguich between an impact/dependancy.

We will need a methodology that induces energy into carbon, there is a well known methodology using the carbon intensity of electricity. Let's add carbon-intensity as another observtion we will need to gather in our impact depandancy tree.

☑️ carbon

🌿 energy
🌿 carbon-intensity

The rest of this module describes how we implement the above process.

Methodology & Implementation

`energy`

You look at your cloud provider’s monitor dashboard and see that it returns energy consumption in kWh for each hour and it currently shows 0.05 kWh.

We need to add an observation of energy in our IMP file. We do that by adding an element to the inputs property in the server-1 component like so:

tree:
  children:
    server-1:
      defaults:
      pipeline:
      inputs:
        - timestamp: 2023-08-06T00:00 # <1>
          duration: 3600 # <2>
          energy: 0.05 # <3>

<1>. We add the timestamp of the observation.
<2> The duration of the observation in seconds, here it's 5 minutes.
<3> The energy in kWh

Note

There are no standard names of properties or units of properties in IMP. Since each observation property is going to be input into a model plugin, it's important to read the documentation of the model plugin to determine what name of property and unit is expected.

`carbon-intensity`

We now have added energy to our IMP file, but how do we convert that into carbon?

This is actually pretty straightforward, because estimates of the carbon emitted per unit of energy used, known as the “carbon intensity”, are widely available. Grid operators sometimes publish these values, often exposing APIs, and there are also third party providers of this data, such as WattTime and Electricity Maps.

If you have energy consumption in kWh and carbon intensity in g CO2e/kWh, their product is carbon emissions in g CO2e.

Let’s say you are in the United Kingdom - you can look up the value for carbon intensity for a specific moment in time from the National Grid API, but in this minimal example you’ll use the annual average, around 163 gCO2e/kWh.

Let's add this to the observations of our server, like so:

server-1:
  defaults:
  pipeline:
  inputs:
    - timestamp: 2023-08-06T00:00
      duration: 3600
      energy: 0.05
      # highlight-next-line
      carbon-intensity: 163

Multiplying the energy consumption and carbon intensity together yields 8.15 g CO2e:

0.05 (KWh) * 163 (gCO2e/kWh) = 8.15 gCO2e

So in order to convert the enegy in carbon we want a model that let's us multiply the carbon intensity of electricity by the energy, luckily there is a built in plugin called Multiply that let's you do this.

TODO - FIX BELOW MESS

[!NOTE] Finding model plugins You can check the standard library of “builtins” and find that there is a prebuilt “Coefficient” plugin that you can use. The builtins are convenient because they are bundled with IF - there’s no need to install any third-party code. There are several places you can find model plugins. If you are using a simple coefficient like above, one of the built in models will work out just fine. But perhaps the model is more complex, in that case you can search the Model Explorer for published model plugins that perform more complex activities. Or you can even build your own if no existing plugin works for you. Impact Framework is felxibel and extensible for your use case.

To use the Multiply plugin we first need to add it to the context, specifically in the plugins: initialize block like so:

carbon-from-energy:           # <1>
  method: Multiply            # <2>
  path: "builtin"             # <3>
  config:                     # <4>
    input-parameters:         # <5>  
      - energy  
      - carbon-intensity
    output-parameter: carbon # <6>

<1> This is the name of this model, a model is an instance of a plugin.
<2> method is the name of the function you are calling from the model plugin. Here, it is Multiply.
<3> path is the location where the plugin is installed in your node_modules folder. For builtins, the path is always just builtin. You can always check the plugin documentation to find the right values for method and path.
<4> config is where you configure a model plugin. Each model plugin has different configuration, refer to the model plugin documentation for details.
<5> input-parameters is an array of input parameters which you want to multiply together.
<7> output-parameter Is the name of paramter you want the result of the multiplication output to.

Note

In more complicated IMPs you might have many instances of the same plugin (e.g. you might use the Multiply plugin many times). Each instance we call a mode, you can call models whatever you like, but we recommend using very descriptive names, using hyphen separated words.

Adding this to your boilerplate IMP, you get:

name: energy-to-carbon
description: minimal demo
tags:
initialize:
  plugins:
    carbon-from-energy:          
      method: Multiply           
      path: "builtin"            
      config:                    
        input-parameters:        
          - energy  
          - carbon-intensity
        output-parameter: carbon 
tree:
  children:
    server-1:
      defaults:
      pipeline:
      inputs:
        - timestamp: 2023-08-06T00:00
          duration: 3600
          energy: 0.05
          carbon-intensity: 163

`carbon`

The last step is to add the carbon-from-energy model plugin we just configured to the pipeline of the server-1 component.

There are different types of pipelines you can configure, for the majority of simple use cases you will want to add your model to the compute pipeline like so:

pipeline:
  compute:
    - carbon-from-energy

Adding that to the IMP file yields this:

name: energy-to-carbon
description: minimal demo
tags:
initialize:
  plugins:
    carbon-from-energy:          
      method: Multiply           
      path: "builtin"            
      config:                    
        input-parameters:        
          - energy  
          - carbon-intensity
        output-parameter: carbon 
tree:
  children:
    server-1:
      defaults:
      # highlight-start
      pipeline:
        compute:
          - carbon-from-energy
      # highlight-end          
      inputs:
        - timestamp: 2023-08-06T00:00
          duration: 3600
          energy: 0.05
          carbon-intensity: 163

Run

Let's review what we've done and where we are at.

We've added a server component called server-1.
We observed that for a 5 min period of time the energy consumed by server-1 was 0.05 kWh.
We added an observation to server-1 by adding an element to the inputs array with an energy parameter of 0.05.
We configured a Coefficient model plugin called carbon-from-energy which uses a hardcoded value for carbon intensity of electricity to convert energy into carbon.
We configured the model plugin to run on the server-1 component, by adding it to the pipeline of the component.

Now it's time to actually compute the carbon, make sure you have saved your file as imp.yml and run the command below:

if-run -m imp.yml

You will see some logs appear in your console, and then the output IMP file will be printed there too. Two will notice that two blocks of YAML have been added to your original IMP data: execution and outputs, like so:

# start
name: energy-to-carbon
description: minimal demo
tags: null
initialize:
  plugins:
    carbon-from-energy:
      path: builtin
      method: Multiply
      config:
        input-parameters:
          - energy
          - carbon-intensity
        output-parameter: carbon
execution:  # <1>
  command: >-
    /Users/jawache/.nvm/versions/node/v23.11.0/bin/node
    /Users/jawache/.nvm/versions/node/v23.11.0/bin/if-run -m ./src/direct.yml
  environment:
    if-version: 1.0.1
    os: macOS
    os-version: 14.7.5
    node-version: 23.11.0
    date-time: 2025-04-14T19:28:00.818Z (UTC)
    dependencies:
  status: success
tree:
  children:
    server-1:
      defaults: null
      pipeline:
        compute:
          - carbon-from-energy
      inputs:
        - timestamp: 2023-08-06T00:00
          duration: 3600
          energy: 0.05
          carbon-intensity: 163
      outputs:                          # <2>
        - timestamp: 2023-08-06T00:00
          duration: 3600
          energy: 0.05
          carbon-intensity: 163
          carbon: 8.15                  # <3>

# end

<1>: execution captures information about the runtime environment for your IMP execution.
<2>: outputs includes all your input data with the result of your plugin execution appended to it.
<3>: You will see in outputs the value 8.15 associated with the key carbon

Important

Notice the output file contains all the same information that was in your manifest. All the plugin declarations, configuration, components, the layout of the tree - it's all there, identically to the original manifest. This is important because it allows others to re-execute your output file. The manifest data is your "working out" - having it here alongside your output data allows others to audit your process as well as your numbers.

Crucially, there are several new sections that have been added to the manifest. These new sections are:

`execution`

The execution section captures the details of the runtime environment that just executed your manifest. This includes details about your operating system, Nodejs and Imapct Framework versions, the date and time the manifest was run, and the packages that are installed in your environment. There is also a status field that can either be success or fail . If the status is fail , then the error message is captured and reported here so you can see why your run failed to execute.

`outputs`

For each component, the results from the execution of each plugin is added to an outputs section. In this example, there is only one plugin that returns a single value, so outputs is identical to inputs but with one additional field - carbon .

inputs:
  - timestamp: 2023-08-06T00:00
      duration: 3600
      energy: 0.05
outputs:
  - timestamp: 2023-08-06T00:00
      duration: 3600
      energy: 0.05
      carbon: 8.15

Review

Congratulations, you just computed the carbon value for your application for a very simple use case. You can do a lot more with IMPs and IF but this simple use case taught you:

The context and the tree.
How to configure a component and add observations
How to congigure a model plugin.
How to add model plugin to the pipeline for a component.
How to execute an IMP file and parse the output.

Tip

You just computed carbon from energy using the Multiply plugin. However, this is not the only way to compute the product of two values in Impact framework.

You could also have chosen to use the Coefficient plugin

There is also the WattTime plugin avaialble from the plugin explorer which pulls live Carbon Intensity data by calling the WattTime API so you don't have to hardcode values in the IMP files.

Test your learning

Try swapping out the Coefficient plugin for Multiply to give the same result.
Try varying the values for energy and carbon intensity coefficient and observe the changing output
Try adding more servers, as new components, and observing how each component's pipeline is executed independently.
Try passing your output file to Impact framework. Notice that Impact Framework can run your output file as if it were a fresh manifest!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

3.direct-measurement.md

3.direct-measurement.md

One-Step Pipeline

Overview

Boilerplate

IDOMI

Impacts

Dependancies & Observations

Methodology & Implementation

`energy`

`carbon-intensity`

`carbon`

Run

`execution`

`outputs`

Review

Test your learning

Files

3.direct-measurement.md

Latest commit

History

3.direct-measurement.md

File metadata and controls

One-Step Pipeline

Overview

Boilerplate

IDOMI

Impacts

Dependancies & Observations

Methodology & Implementation

energy

carbon-intensity

carbon

Run

execution

outputs

Review

Test your learning

`energy`

`carbon-intensity`

`carbon`

`execution`

`outputs`