Update amalgamator for municipal datasets #588
Conversation
| combination_method: | ||
| weighted_average: | ||
| - sum: | ||
| - present_number_of_apartments_before_1945 | ||
| - present_number_of_apartments_1945_1964 | ||
| - present_number_of_apartments_1965_1984 | ||
| - present_number_of_apartments_1985_2004 | ||
| - present_number_of_apartments_2005_present | ||
| - present_number_of_detached_houses_before_1945 | ||
| - present_number_of_detached_houses_1945_1964 | ||
| - present_number_of_detached_houses_1965_1984 | ||
| - present_number_of_detached_houses_1985_2004 | ||
| - present_number_of_detached_houses_2005_present | ||
| - present_number_of_semi_detached_houses_before_1945 | ||
| - present_number_of_semi_detached_houses_1945_1964 | ||
| - present_number_of_semi_detached_houses_1965_1984 | ||
| - present_number_of_semi_detached_houses_1985_2004 | ||
| - present_number_of_semi_detached_houses_2005_present | ||
| - present_number_of_terraced_houses_before_1945 | ||
| - present_number_of_terraced_houses_1945_1964 | ||
| - present_number_of_terraced_houses_1965_1984 | ||
| - present_number_of_terraced_houses_1985_2004 | ||
| - present_number_of_terraced_houses_2005_present |
There was a problem hiding this comment.
Would a comment concerning the combination method be useful here: explaining how we use the total number of residences?
| @@ -47,7 +47,7 @@ def adjust_flexible_shares(dataset_c_values) | |||
|
|
|||
| # Rounds values to 8 decimals and replaces nil with 0.0 | |||
There was a problem hiding this comment.
If we have changed the method from round to floor, shouldn't we update the comment as well?
| @@ -39,7 +39,7 @@ | |||
|
|
|||
| it 'rounds values to 8 decimals and replaces nil with 0.0' do | |||
There was a problem hiding this comment.
If we have changed the method from round to floor, shouldn't we update the comment as well?
This I don't really get. Does this mean that there will be something like a 0.000000001 share attributed to the flexible share? |
Sure. I think in the future we should reconsider the 60 GW interconnector capacity per municipality value, but that's something for future improvement. It is also not really relevant for EU countries now, since we do not use the amalgamator for those. @louispt1 would this be relevant for the "subtracting" amalgamator - so the one for UK, GB and NI? |
Yes you are right! Same example with floor rounding on 2 decimals: Share_1: 0.42 |
While updating the RES-regions to 2019 we encountered the following issues :
The Eemsdelta dataset had incorrect share values for useful demand per housing and period.
The first issue was fixed by updating the household shares in the Eemsdelta municipality and migrating this change on etlocal. --> This will be fixed in the data-update of the RES-regions
The second issue was fixed by altering the variable on which the combination method was called. For the combination method weighted_average to function correctly for shares, we need to specify one variable for the whole share group. We changed it to the sum of al households in the dataset for the households_heat share_in_useful_demand group.
The third issue was caused due to the rounding of the keys to 8 digits. We changed the round method to a floor method. With the floor method, when the total sum of the share group will become lower than 1.0, instead of higher. The difference will be attributed to the flexible share in the share group. This way, rounding to 8 digits will no longer lead to errors.
This PR fixes the last 2 issues and in addition I made an alteration on how the interconnector capacity is calculated, for the European datasets it makes sense to use the sum method, since we use the actual interconnector data for this capacity.
For Municipal datasets this value is set to 60 GW per municipality, to ensure the connection to the rest of the Dutch grid.
If we use the sum method for local datasets, the RES regions would end up with 100+ GW connections, which have a huge influence on the cost of the scenario.