[Data Issue]: bmreports closedown about electricitymaps-contrib HOT 11 CLOSED

If historical pumping data is still to be corrected (#6419), am I right to assume #6441 needs to be merged before the closedown happens?

from electricitymaps-contrib.

Everything has been deployed and seems to be functioning as expected so I'll close this issue. Thanks for reminding us about it!

from electricitymaps-contrib.

I saw this yesterday and forgot to close the issue, thank you for fixing it!

from electricitymaps-contrib.

I noticed that GB still has 100% estimated for all days since the beginning of this month, did anything go wrong?

We don't think it was anything with the parser, we also added som additional internal validation rules that seems to have invalidated a lot of GB datapoint when it shouldn't have. That has since been fixed but I don't think anyone has run a manual flowtracing job to fix it directly as we are letting our automated processes do it.

We will be double checking if they don't fix it though.

FYI @florianscheidl.

from electricitymaps-contrib.

@q-- we have a fix ready in the pipeline.
The estimation comes from our quality validation layer expecting coal production. Since the current production parser reports nulls instead of zeros, we have been filtering all production parser events with missing coal values and estimating them instead. After fixing, the data will be reprocessed and estimated values should be replaced with measured ones.

from electricitymaps-contrib.

Thanks for the reminder!

@mathilde-daugy the ELEXON parser seems to still be using this. I suppose that is something we want to prioritise fixing? Do you have time to do so otherwise I'll try and get it done next week.

from electricitymaps-contrib.

last day :)

from electricitymaps-contrib.

The fixes have been merged but unfortunately I do not think they have been deployed yet. I'll monitor the situation and if need be we might have to deploy on the weekend, otherwise a deployment will happen on Monday.

from electricitymaps-contrib.

Ok, got it. Hopefully you can have it up and running soon! (for now i can just use https://grid.iamkate.com/ for UK data which uses the insights solution)

from electricitymaps-contrib.

Ok, got it. Hopefully you can have it up and running soon! (for now i can just use https://grid.iamkate.com/ for UK data which uses the insights solution)

Even if the data source goes down we should be estimating the production in the meantime, not optimal I know but once live data is available again it will be corrected.

from electricitymaps-contrib.

I noticed that GB still has 100% estimated for all days since the beginning of this month, did anything go wrong?

from electricitymaps-contrib.