Update amalgamator for municipal datasets #588
Conversation
| combination_method: | ||
| weighted_average: | ||
| - sum: | ||
| - present_number_of_apartments_before_1945 | ||
| - present_number_of_apartments_1945_1964 | ||
| - present_number_of_apartments_1965_1984 | ||
| - present_number_of_apartments_1985_2004 | ||
| - present_number_of_apartments_2005_present | ||
| - present_number_of_detached_houses_before_1945 | ||
| - present_number_of_detached_houses_1945_1964 | ||
| - present_number_of_detached_houses_1965_1984 | ||
| - present_number_of_detached_houses_1985_2004 | ||
| - present_number_of_detached_houses_2005_present | ||
| - present_number_of_semi_detached_houses_before_1945 | ||
| - present_number_of_semi_detached_houses_1945_1964 | ||
| - present_number_of_semi_detached_houses_1965_1984 | ||
| - present_number_of_semi_detached_houses_1985_2004 | ||
| - present_number_of_semi_detached_houses_2005_present | ||
| - present_number_of_terraced_houses_before_1945 | ||
| - present_number_of_terraced_houses_1945_1964 | ||
| - present_number_of_terraced_houses_1965_1984 | ||
| - present_number_of_terraced_houses_1985_2004 | ||
| - present_number_of_terraced_houses_2005_present |
There was a problem hiding this comment.
Would a comment concerning the combination method be useful here: explaining how we use the total number of residences?
| @@ -39,7 +39,7 @@ | |||
|
|
|||
| it 'rounds values to 8 decimals and replaces nil with 0.0' do | |||
There was a problem hiding this comment.
If we have changed the method from round to floor, shouldn't we update the comment as well?
This I don't really get. Does this mean that there will be something like a 0.000000001 share attributed to the flexible share? |
Sure. I think in the future we should reconsider the 60 GW interconnector capacity per municipality value, but that's something for future improvement. It is also not really relevant for EU countries now, since we do not use the amalgamator for those. @louispt1 would this be relevant for the "subtracting" amalgamator - so the one for UK, GB and NI? |
Yes you are right! Same example with floor rounding on 2 decimals: Share_1: 0.42 |
|
This PR is a requirement for solving this issue |
Do I need to look at this PR @kndehaan/ @kaskranenburgQ? If so I will add it to the dev sprint |
|
Postpone to update of regional datasets to 2023. |
louispt1
force-pushed
the
update-amalgamator
branch
2 times, most recently
from
6691c44 to
324e8c8
Compare
There was a problem hiding this comment.
I'm not sure about the modelling changes in this PR, but I've just rebased it and updated the round item values to use a 'smart floor' functionality.
Can you test if this works @kaskranenburgQ? Hopefully the comments are clear enough
louispt1
force-pushed
the
update-amalgamator
branch
from
324e8c8 to
4375a54
Compare
louispt1
force-pushed
the
update-amalgamator
branch
from
b7cc0fa to
b85ec4b
Compare
louispt1
force-pushed
the
update-amalgamator
branch
from
b85ec4b to
32cbdfa
Compare
While updating the RES-regions to 2019 we encountered the following issues :
The Eemsdelta dataset had incorrect share values for useful demand per housing and period.
The first issue was fixed by updating the household shares in the Eemsdelta municipality and migrating this change on etlocal. --> This will be fixed in the data-update of the RES-regions
The second issue was fixed by altering the variable on which the combination method was called. For the combination method weighted_average to function correctly for shares, we need to specify one variable for the whole share group. We changed it to the sum of al households in the dataset for the households_heat share_in_useful_demand group.
The third issue was caused due to the rounding of the keys to 8 digits. We changed the round method to a floor method. With the floor method, when the total sum of the share group will become lower than 1.0, instead of higher. The difference will be attributed to the flexible share in the share group. This way, rounding to 8 digits will no longer lead to errors.
This PR fixes the last 2 issues and in addition I made an alteration on how the interconnector capacity is calculated, for the European datasets it makes sense to use the sum method, since we use the actual interconnector data for this capacity.
For Municipal datasets this value is set to 60 GW per municipality, to ensure the connection to the rest of the Dutch grid.
If we use the sum method for local datasets, the RES regions would end up with 100+ GW connections, which have a huge influence on the cost of the scenario.