Comments (4)
JrtPec
commented on July 17, 2024
Great news! I'll put it on my to do list.
from entsoe-py.
JrtPec
commented on July 17, 2024
So, I downloaded the code lists, and also the general CIM package.
I tried using the xmlschema library.
I was able to validate a downloaded generation xml-file using the corresponding xsd-file, like so:
import xmlschema
schema = xmlschema.XMLSchema('entsoe/codelists/CIM_2020-10-19/iec62325-451-6-generationload_v3_0.xsd')
schema.is_valid('generation_doc.xml') # True
schema.to_dict('generation_doc.xml')
{'@xmlns': 'urn:iec62325.351:tc57wg16:451-6:generationloaddocument:3:0',
'mRID': '974d8ef382cf479a96768c4c728736ca',
'revisionNumber': '1',
'type': 'A71',
'process.processType': 'A01',
'sender_MarketParticipant.mRID': {'@codingScheme': 'A01',
'$': '10X1001A1001A450'},
'sender_MarketParticipant.marketRole.type': 'A32',
'receiver_MarketParticipant.mRID': {'@codingScheme': 'A01',
'$': '10X1001A1001A450'},
'receiver_MarketParticipant.marketRole.type': 'A33',
'createdDateTime': '2020-10-21T10:36:59Z',
'time_Period.timeInterval': {'start': '2017-09-27T22:00Z',
'end': '2017-09-28T22:00Z'},
'TimeSeries': [{'mRID': '1',
'businessType': 'A01',
'objectAggregation': 'A01',
'inBiddingZone_Domain.mRID': {'@codingScheme': 'A01',
'$': '10YAT-APG------L'},
'quantity_Measure_Unit.name': 'MAW',
'curveType': 'A01',
'Period': [{'timeInterval': {'start': '2017-09-27T22:00Z',
'end': '2017-09-28T22:00Z'},
'resolution': 'PT60M',
'Point': [{'position': 1, 'quantity': Decimal('5107')},
{'position': 2, 'quantity': Decimal('5052')},
{'position': 3, 'quantity': Decimal('5079')},
{'position': 4, 'quantity': Decimal('4948')},
{'position': 5, 'quantity': Decimal('4946')},
{'position': 6, 'quantity': Decimal('5106')},
{'position': 7, 'quantity': Decimal('5644')},
{'position': 8, 'quantity': Decimal('7208')},
{'position': 9, 'quantity': Decimal('7430')},
{'position': 10, 'quantity': Decimal('7398')},
{'position': 11, 'quantity': Decimal('7002')},
{'position': 12, 'quantity': Decimal('6402')},
{'position': 13, 'quantity': Decimal('5810')},
{'position': 14, 'quantity': Decimal('5802')},
{'position': 15, 'quantity': Decimal('5764')},
{'position': 16, 'quantity': Decimal('5709')},
{'position': 17, 'quantity': Decimal('6080')},
{'position': 18, 'quantity': Decimal('7241')},
{'position': 19, 'quantity': Decimal('7966')},
{'position': 20, 'quantity': Decimal('8122')},
{'position': 21, 'quantity': Decimal('7724')},
{'position': 22, 'quantity': Decimal('5541')},
{'position': 23, 'quantity': Decimal('5408')},
{'position': 24, 'quantity': Decimal('5145')}]}]},
{'mRID': '2',
'businessType': 'A01',
'objectAggregation': 'A01',
'outBiddingZone_Domain.mRID': {'@codingScheme': 'A01',
'$': '10YAT-APG------L'},
'quantity_Measure_Unit.name': 'MAW',
'curveType': 'A01',
'Period': [{'timeInterval': {'start': '2017-09-27T22:00Z',
'end': '2017-09-28T22:00Z'},
'resolution': 'PT60M',
'Point': [{'position': 1, 'quantity': Decimal('0')},
{'position': 2, 'quantity': Decimal('0')},
{'position': 3, 'quantity': Decimal('0')},
{'position': 4, 'quantity': Decimal('0')},
{'position': 5, 'quantity': Decimal('0')},
{'position': 6, 'quantity': Decimal('0')},
{'position': 7, 'quantity': Decimal('0')},
{'position': 8, 'quantity': Decimal('0')},
{'position': 9, 'quantity': Decimal('0')},
{'position': 10, 'quantity': Decimal('0')},
{'position': 11, 'quantity': Decimal('0')},
{'position': 12, 'quantity': Decimal('0')},
{'position': 13, 'quantity': Decimal('0')},
{'position': 14, 'quantity': Decimal('0')},
{'position': 15, 'quantity': Decimal('0')},
{'position': 16, 'quantity': Decimal('0')},
{'position': 17, 'quantity': Decimal('0')},
{'position': 18, 'quantity': Decimal('0')},
{'position': 19, 'quantity': Decimal('0')},
{'position': 20, 'quantity': Decimal('0')},
{'position': 21, 'quantity': Decimal('0')},
{'position': 22, 'quantity': Decimal('0')},
{'position': 23, 'quantity': Decimal('0')},
{'position': 24, 'quantity': Decimal('0')}]}]}]}So that's already a nice result, that could save us some effort when parsing files.
However, what I'd like is a way to convert the values such as 'businessType': 'A01' into its corresponding string value. I found these things in etso-code-lists.xsd.
However, I lack the experience with XML files to glue all this together... Any XML experts, or people with enough time on their hands to figure this out?
from entsoe-py.
JrtPec
commented on July 17, 2024
The alternative approach would be to write a simple parser that grabs all those mappings from the code-lists file; and makes them accessible to the code similar to how it's done right now.
That would also reduce the maintenance effort whenever mappings change, we'd just have to switch out the file.
from entsoe-py.
ba1dr
commented on July 17, 2024
That's tricky. I did not find a simlple solution (apart from writing a parser manully), but some dirty solution would be:
- use gererateDS to create a source file with enums:
generateDS -o codelists_dummy_file.py urn-entsoe-eu-wgedi-codelists.xsd
class StandardAssetTypeList(Enum):
"""ET0031
The identification of the type of asset."""
A_01='A01' # Tieline A high voltage line used for cross border energy interconnections.
A_02='A02' # Line A specific electric line within a country.
A_03='A03' # Resource Object A resource that can either produce or consume energy.
A_04='A04' # Generation A resource that can produce energy.
A_05='A05' # Load A resource that can consume energy.
...
- then use regex to convert comments to descriptions
Actually I prefer to use that generated py file as parser for xml files:
from ds_b15x import parseString as ds_parseString
doc = ds_parseString(xml_content, silence=True)
print(doc.createdDateTime)
from entsoe-py.
Related Issues (20)
- Cannot download prices anymore since last week "error 404" HOT 9
- Georgia is missing when use query_load HOT 2
- Entsoe import error using old version of bs4 HOT 1
- certificate verify failed: unable to get local issuer certificate (_ssl.c:1125)'))) HOT 2
- Issue with EntsoePandasClient --> query_day_ahead_prices HOT 2
- `query_generation_forecast` returns an extra column for Lithuania HOT 2
- Issue with EntsoePandasClient: query_installed_generation_capacity HOT 1
- query_unavailability_transmission broken in commit 9f4364c HOT 2
- query_day_ahead_prices resulting in NoMatchingDataError HOT 2
- query_generation & query_load resulting in NoMatchingDataError for UK HOT 1
- Connection Error: 'Connection aborted.' HOT 1
- `NoMatchingDataError` for imbalance price query for Germany depending on query start HOT 5
- Missing neighbours in NEIGHBOURS HOT 1
- issue with 403 Client Error HOT 5
- No function to query the Aggregated Unavailability of Consumption Units and Unavailability of Offshore Grid HOT 2
- Typo in HEADERS_UNAVAIL_TRANSM HOT 1
- Use datetime module HOT 1
- Missing Day-Ahead NTC for DE_LU-NO_2 and DE_LU-SE_4 HOT 1
- InvalidBusinessParameterError Encountered without Further Explanation HOT 1
- NoMatchinDataError HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from entsoe-py.