[Data Issue]: Possible double counting of storage related emissions in consumption view about electricitymaps-contrib HOT 18 OPEN

I'll have a look next week :)

from electricitymaps-contrib.

@pierresegonne when you have the time could you take a look at this issue? @leon-hard raises some valid points but I think you would be in a better position to answer this than I am.

from electricitymaps-contrib.

I haven't received any feedback yet, so I attempted some reverse engineering to better understand the available data concerning electricity production and consumption.

I noticed that when toggling between the consumption and production views in the "Total Electricity Consumption by Source" and "Total Electricity Production by Source" sections, the specific production values remain unchanged. However, the aggregate figure displayed after each production type changes.

For example, the total electricity production and consumption in Germany for 2023 were as follows:

Total production: 446 TWh
Total consumption: 498 TWh

This suggests a net import balance of 52 TWh.

However, the official reported import balance is only 9.2 TWh, while reports to ENTSO-E TP sum up to 11.7 TWh.

The load reported to ENTSO-E TP for 2023 was 458.3 TWh.

When I add the production value to the ENTSO-E reported import balance, it aligns nearly exactly with the load reported by ENTSO-E TP.

This leads me to question where the 498 TWh figure for total consumption comes from. Could you provide some insight into how the consumption estimate is calculated? Understanding this would help clarify the situation.

This observation seems to be similar for many countries, however I did not crunch the numbers for all of them.

from electricitymaps-contrib.

Just to address your last reply (I'm not the best person to talk to about the first part) but the value you used for consumption don't actually represent the consumption, it is the available energy. Available energy is just the production + imports without the exports accounted for at all.

We did test accounting for both imports and exports to get the actual consumption but then there was a lot of confusion with how hydro could produce 300% of the consumption in SE-SE3 for example since most of the produced energy there is exported.

from electricitymaps-contrib.

Thank you very much for your reply!

The import for Germany in 2023 according to the physical flows at ENTSO-E TP was 63.6 TWh (52.0 TWh exports).

This would lead to 446 TWh + 63.6 TWh = 509.6 TWh (there stays a difference of 11,6 TWh to the reported 498 TWh)

The consumption of pumped hydro was 14,1 TWh in 2023 in Germany. So this is in the range of pumped hydro consumption, but not really adding up, so I guess there are other factors also considered in the calculation.

from electricitymaps-contrib.

Thank you very much for your reply!

The import for Germany in 2023 according to the physical flows at ENTSO-E TP was 63.6 TWh (52.0 TWh imports).

This would lead to 446 TWh + 63.6 TWh = 509.6 TWh (there stays a difference of 11,6 TWh to the reported 498 TWh)

The consumption of pumped hydro was 14,1 TWh in 2023 in Germany. So this is in the range of pumped hydro consumption, but not really adding up, so I guess there are other factors also considered in the calculation.

It's also possible that some data was updated after we consumed it, we continuously refetch the last 3 months but if they did a bulk update that could have been missed. It is suspicious there is such a large difference though but I'll look into it.

from electricitymaps-contrib.

I did a brief check for Switzerland for the aggregated values for 2023 for production values (chosen bcs. it has less production types to consider):

It seems like a renewable share factor for unknown of 0.8 is used, at least I need it to replicate the results.

Hydro storage is not used for the production as well as not used for the total values for the calcualtion of carbon intensity and the shares (which I agree should be done to avoid double counting).

I was a bit confused, because the tooltip of the emissions shows the overall emission sum excluding pumped hydro emissions, on the other hand, the tooltip for the production shows the sum including pumped hydro production. I would suggest to do it similar for both graphs. 68.3 TWh is including pumped hydro, but this is not the value used for the calculations.

It would be great to be able to understand the calculations used for the consumption view like here for the production view.

from electricitymaps-contrib.

You can cross check with the ENTSO-E statistical fact-sheet:

https://www.entsoe.eu/data/power-stats/

It gives these key values:

Total net generation: 448.2 TWh (currently at EM: 446 TWh)
Consumption of hydro storage: 14.1 TWh

It also gives values for sum of imports and sum of exports on page 13. But you have to be careful here, as this is the sum of im-/exports with netted/balanced values (if there is more than one interconnector between two countries, you can have im- and export at the same time). For flow tracing, I would argue that you have to consider the physical flow as reported/measured.

Here are the values for import and export:

Total import: 63.6 TWh (with netted/balanced values: 51.9 TWh)
Total export: 52.0 TWh (with netted/balanced values: 43.3 TWh)

Difference for both is of course matching and 11.6 TWh.

And if I do the calculation of the 446 TWh + netted/balanced import of 51.9 TWh, I end up at 497.9 TWh wich could be equal to the total available energy of 498 TWh (considering that 446 is rounded).

But I have two issues with this:

1. The 446 TWh is including pumped hydro production (at least it matches almost the 448.2 TWh reported by ENTSO-E, which includes pumped hydro). If this is true, with my reasoning from the initial post, I have to assume that double counting is appearing. It could still be possible that it is later substracted for the calculation of the derived values, however I can not really check this.

2. I would argue that you need to use the total import and export values for flow tracing and not the netted/balanced values. Just think for example of the long border between Austria and Germany with mutliple interconnectors. You might have import near to lake Constance and export near to Passau, if you net out im- and export, you pretend nothing has happened. But in reality, these two interconnectors are very far apart and it is hard to reason to neglect this effect. But this can defintely be discussed and the choice of either should be documented (or is already and I am not aware of it ;-) )

from electricitymaps-contrib.

Hi,
I think it is hard to learn from the aggregated values about the corrections happening in each hour to get a balanced system, so it would be great to learn how this is currently implemented. Just to clarify the different questions now arising in this thread:

1. To apply flow tracing (which is used to derive the origin of electricity imports for consumption-based accounting), you need a balanced pattern at each node. EM is using cross-border physical flows, so the balance reads:
   generation + import + storage charging = load + export + storage discharge
   ENTSO-E gives values for all the terms in this equation, but unfortunately they don't add up (and the difference can be quite significant). So to get a balanced pattern, you need to add some correction to the values in the equation. Basically there are two perspective on that, I'd say:
   a) You don't know which of the values are actually correct, so you try to correct some or all of them just a bit to get a balanced pattern. This is what we do for CO2Map.de - we have a balancing algorithm (inspired by the paper from de Chalendar) which adds small corrections to all terms to get a balanced pattern, with the corrections determined through an optimization problem (which tries to correct as little as possible, with some tweaks which cause the process, because we look at Germany, to correct predominantly at other places).
   b) You assume that one of the reported values is just wrong , the others are right, and you calculate the supposedly wrong value yourself from the others (using the balancing equation above). This could be the load, for instance, which you then would calculate from all other values (physical flows, storage, generation) - an approach taken, to my understanding, by co2-monitor in Germany.
   In any case, your hourly values feeding into the flow tracing algorithm (and accordingly used for the final results) depend on the balancing corrections (if you do not re-scale them somehow afterwards). If you only do corrections for consumption-based emission accounting (where you need flow tracing), but not for generation-based accounting, then you get different totals for each approach (if you do not somehow rescale one of them at the end).
   Would be good to know how exactly that is currently handled for Electricitymaps.

2. Storage accounting:
   If you assume your load, generation and storage values are representing load, generation and storage operation, you can account for storage as in this example:
   First time step: Generation of 2 kWh of electricity from a source with 1000gCO/kWh, 50% of generation goes into storage (1 kWh), 50% of generation goes into load (1 kWh)
   Second time step: Storage is discharged (1 kWh) to cover the load (1 kWh), no generation from the generator above.
   In this case you could calculate the consumption-based intensity as follows (you know that all generation/storage discharge goes into load/storage charging at the same node, for the actual system you have to figure the origin out through flow tracing):
   First time step: (generation*emissionfactor)/(storage charging + load) = 1000gCO2/kWh
   Second time step: (storage discharge * emissionfactor_storage)/load) = 1000gCO2/kWh
   (Here you know the emission factor of storage discharge, which for the actual system is quite hard and currently is only estimated).
   This works well for the emission intensities, but for the total emissions this causes some inconsistencies. You could include the storage charging or discharging for calculating total emissions, but then you are double counting. Or could exclude storage charging and discharging, then the total emissions from generation and attributed to the load do not coincide in time (in the example above, in the first time step we have 2kgCO2 from generation and 1 kgCO2 attributed to load, in the second time step no emissions from generation and 1 kgCO2 attributed to load).

3. Net imports/exports or imports and exports for the flow tracing algorithm:
   To actually account for the effect with flows in different directions on different parts of the border, you would need a higher spatial resolution in your model. The difference between using net imports/exports or both imports and exports for the flow tracing method is discussed here a bit ("aggregated coupling" vs. "direct coupling").

from electricitymaps-contrib.

Thank you very much for detailed description and background!

I also didn't see directly that the netted/balanced im-/exports for one border are equal to the two main ideas behind how to do flow tracing, i.e. in my own simplified words: mixing the smoothie in a big bin where everything goes in (direct coupling) vs. supplying domestic demand by domestic production and only mix im-/export in a separate, smaller bin (aggregated coupling).

One simple way to test which one is used, just check if the values change for a region with net export between consumption and production view, you can easily observe this effect in CO2map and EM.

If I understand it correctly, EM is currently using a hybrid version, i.e. direct coupling but aggregating the flows between one border. It is a bit hard to understand the reasoning of mixing both together.

from electricitymaps-contrib.

Hey, I've finally managed to find some time to read through this thread.

I applaude the level of scrutinity, it's really important that we nail down the communication around that topic.

I'll start with storage, and follow up with more general questions around the flowtracing methodology in a separate answer.

For storage, please find what happens with the example you had in mind:

The key to understanding why there is no double counting from storage is that we compute consumption as:

consumption = production + imports + discharge - exports - storage

That means that the carbon content of the electricity stored is "sunk", to be later reinjected when there is discharge.

The source of error then comes the mismatch between the content of the electricity that was sunk and the static assumed content exposed by the discharge emission factor

from electricitymaps-contrib.

Regarding now the flowtracing methodology itself:

• We do not use the load values from ENTSO-E. What we're doing is akin to solution 2. mentioned by @mirkoschaefer, we recompute the consumption from the other variables - that introduces the assumption that we can trust those.

• Are we using net import or not? Find here a simple example where we run through our flowtracing methodology in a simplified manner https://docs.google.com/presentation/d/1KIHVRgaWzgxaTZwiAcCfRB_swi99GTM2wlazE4t3SPU/edit?usp=sharing It does not show what happens if we have two connections between two zones with different flow directions though. We create the matrix with aggregated imports. That aggregation happens at the parser level, for ENTSO-E we query the Cross-Border Physical Flow (12.1G), which has data for both directions, and we then combine for each hour.

• Regarding the difference observed at the yearly level, I am not sure about the reason for such a large difference. I unfortunately don't have any more time today to look into it, but it would be great if someone could follow-up on that

from electricitymaps-contrib.

Thank you so much for your response @pierresegonne!

After reviewing the slides, I actually ended up with a few more questions 😊. I noticed a potential issue on the last slide with the solution provided. When I run through the calculations myself, I get a different solution vector:

x1 ≈ 0.488
x2 ≈ 0.250
x3 ≈ 0.659

I'm not sure if I'm misunderstanding the methodology, or if there might be an error. For example, regarding country B's data in the graphic, it seems to indicate that country B is exporting (though it's a bit unclear since E_B-A = -10), which would suggest that the wind share should remain at 25%. However, even if I adjust the E_B-A direction, my results still differ. Could you help clarify this?

Thanks a lot!

from electricitymaps-contrib.

Firstly, I would like to express my appreciation for the detailed graphic and explanation regarding emissions accounting you provided. It's an excellent resource, and I find myself in agreement with the methodology for calculating consumption-based emission intensity.

However, upon a detailed analysis of the data from the EM interface at a specific point in time (today at 12:00 in Germany), I've noticed a discrepancy that doesn't align with what is depicted in the graphic.

I conducted a thorough examination and here are my findings:

• The consumption-based emission intensity, as depicted in your graphic, aligns with my understanding and the results of my calculations.

• There appears to be an inconsistency in how the EM calculates "consumption." It seems to define consumption as the sum of total production plus the import saldo, which also includes pumped hydro consumption. This methodology leads to a potential issue of double counting.

I've attached a screenshot here that illustrates this point. The tooltip indicates that the total production-based emissions are 13.2 kt, whereas the consumption-based emissions are reported as 13.3 kt.

However, if we adjust the consumption definition to be production plus import saldo minus pumped hydro consumption, the corrected total consumption-based emissions, according to my calculations, should be approximately 11.6 kt. This adjustment addresses the issue of double counting that is currently present.

Could you possibly review this and share your thoughts on whether the EM interface might need to adjust its calculation methodology to avoid this discrepancy?

Just to sum it up:

Currently reported total consumption based emissions: 13.3 kt
Accroding to my calculations: 11.6 kt (taking into account that hydro pumped storage consumption should not be used to calculate consumption based emissions)

Thank you very much in advance!

from electricitymaps-contrib.

FYI the app backend might be doing some simplifications to the calculations after the flowtracing for performance reasons which could cause small differences. I'm not sure it's the cause here but I know it's simplifying some calculations by making some assumptions about data availability. I'll add an internal issue to double check it, personally I won't have time to do it until after the 2nd of June do to uni exams but I'll add it to my todo list just in case.

from electricitymaps-contrib.

Hi all,
also thanks from my side for the detailed conversation. I currently have no time to check the calculations, but will try to take a look at it next week. For now I wanted to add two comments to points from the discussion:

• Consumption: From the discussion here I learn that it should be made clear what "load" or "consumption" refers to, either the value calculated from the balance (as done by EM, co2-monitor, Agorameter) or the (possibly adjusted) "actual total load" (as done by co2map and (?) energy-charts). The differences can be several GW and also depend on your corrections of generation data (as done by Agorameter and co2-monitor, for instance).
• Flow tracing: I think the cross-border physical flows from ENTSO-E are only in one direction, but if they would be in both directions (or, for any reason, you would use the so-called scheduled commercial flows published after the intraday market) you could include flows in both directions instead of the net flow in the direct coupling flow tracing algorithm. This would lead to cycles on the border though, making the intuition behind it a bit challenging (which is usually "tracking from source to sink"). There are occasionally some minor cycles in the flow pattern already now for the net flows, because the published values represent some aggregation, but that is no problem for the matrix formulation also used by EM (there is still some intuition behind it based on converging partial flows).

from electricitymaps-contrib.

Thank you very much for your comments @mirkoschaefer!

I think we have to be careful with accessing data from ENTSO-E. The cross-border physical flows are reported by the TSOs to ENTSO-E. They are reporting for their corresponding control area. Many countries only consist of a single control area, so there might be not a big difference. However, in Germany, there are four different control areas.

Each TSO is reporting its own physical flows. ENTSO-E is then aggregating these values. I would suggest to use directly the TSO data without prior aggregation.

Here, just an example which you can check at ENTSO-E TP for 2nd of May, 00:00 - 00:15 CEST, and the border DE-AT:

1. CTA|DE(Amprion) -> CTA|AT: 295 MW
2. CTA|AT -> CTA|DE(TransnetBW): 56 MW
3. CTA|DE(TenneT GER) > CTA|AT: 126 MW

Total export for DE: 421 MW
Total import for DE: 56 MW
Saldo for DE: 365 MW

This corresponds to the value reported for the country border (including some rounding errors):
Germany (DE) -> Austria (AT): 367 MW

Flow tracing should fundamentally be regarded as a modeling tool rather than a direct representation of physical processes. It is essential to understand that while flow tracing attempts to answer questions about the origins of energy consumption - essentially tracing the "colors" of the electrons utilized - it does not physically track the electrons themselves. Therefore, the results produced by flow tracing models should be interpreted within the context of their methodological framework rather than as exact depictions of electrical flow. This distinction is crucial when analyzing or drawing conclusions from flow tracing outcomes, as the models provide approximations based on theoretical constructs rather than empirical observations.

However, I would expect a flow tracing algorithm to at least be based on the flows where actual data is available. And as I had shown above, these number can deviate considerable (63.6 TWh vs 51.9 TWh, +22.5%).

Regarding the question on double counting, I also attached my excel sheet in order to be able to understand my calculations. My observation is that the calculation for the total consumption based emissions is using emission factor * consumption (being production + import saldo).

Recalculate EM.xlsx

from electricitymaps-contrib.

Hey sorry for the delay in replying, I'm completely taken by internal tasks and I've still not been able to look at your answers. We have a big delivery coming up end of June so most of the people that could jump in to answer are also busy with that. I'll come back to that as soon as I have capacity

from electricitymaps-contrib.