juniper / jxmlease Goto Github PK
View Code? Open in Web Editor NEWA Python module for converting XML to intelligent Python data structures, and converting Python data structures to XML.
License: Other
A Python module for converting XML to intelligent Python data structures, and converting Python data structures to XML.
License: Other
The claim was made that jxmlease does not have the same problem as xmltodict on martinblech/xmltodict#182. But that does not seem to be true. ๐ญ
>>> import jxmlease
>>> s = '''
... <foo>
... <a>one</a>
... <b>two</b>
... <a>three</a>
... </foo>
... '''
>>> print(jxmlease.parse(s).emit_xml())
<?xml version="1.0" encoding="utf-8"?>
<foo>
<a>one</a>
<a>three</a>
<b>two</b>
</foo>
Hello all, thank you for this wonderful library. I have been using it for the past 3 weeks now and it's a blessing.
When working with namespaces in creating an xml it inserts the namespaces to both the opening and closing of the namespaced xml tags (details provided below). Is there a way to remove the namespace from the closing of the namespaced xml tag?
{
"soapenv:Envelope xmlns:'http://schemas.xmlsoap.org/soap/envelope/' xmlns:loc='http://www.csapi.org/schema/osg/ussd/notification_manager/v1_0/local'": {
"soapenv:Body": {
"loc:startUSSDNotification": [
{
"loc:reference": {
"correlator": "09876",
"endpoint": "http://172.16.130.193:8080/ussd"
},
"loc:ussdServiceActivationNumber": "9475"
}
]
},
"soapenv:Header": {
"tns:RequestSOAPHeader xmlns:tns='http://www.huawei.com.cn/schema/common/v2_1'":{
"tns:serviceId": "35000001000029",
"tns:spId": "015363",
"tns:spPassword":"278b129b3a1db993b615825f0b04dfb2",
"tns:timeStamp": "20221019121812"
}
}
}
}
which comes out to:
<?xml version="1.0" encoding="utf-8"?>
<soapenv:Envelope xmlns:'http://schemas.xmlsoap.org/soap/envelope/' xmlns:loc='http://www.csapi.org/schema/osg/ussd/notification_manager/v1_0/local'>
<soapenv:Header>
<tns:RequestSOAPHeader xmlns:tns='http://www.huawei.com.cn/schema/common/v2_1'>
<tns:spId>015363</tns:spId>
<tns:spPassword>278b129b3a1db993b615825f0b04dfb2</tns:spPassword>
<tns:serviceId>35000001000029</tns:serviceId>
<tns:timeStamp>20221019121812</tns:timeStamp>
</tns:RequestSOAPHeader xmlns:tns='http://www.huawei.com.cn/schema/common/v2_1'>
</soapenv:Header>
<soapenv:Body>
<loc:startUSSDNotification>
<loc:reference>
<endpoint>http://172.16.130.193:8080/ussd</endpoint>
<correlator>09876</correlator>
</loc:reference>
<loc:ussdServiceActivationNumber>9475</loc:ussdServiceActivationNumber>
</loc:startUSSDNotification>
</soapenv:Body>
</soapenv:Envelope xmlns:'http://schemas.xmlsoap.org/soap/envelope/' xmlns:loc='http://www.csapi.org/schema/osg/ussd/notification_manager/v1_0/local'>
when it should have been as so:
<?xml version="1.0" encoding="utf-8"?>
<soapenv:Envelope xmlns:'http://schemas.xmlsoap.org/soap/envelope/' xmlns:loc='http://www.csapi.org/schema/osg/ussd/notification_manager/v1_0/local'>
<soapenv:Header>
<tns:RequestSOAPHeader xmlns:tns='http://www.huawei.com.cn/schema/common/v2_1'>
<tns:spId>015363</tns:spId>
<tns:spPassword>278b129b3a1db993b615825f0b04dfb2</tns:spPassword>
<tns:serviceId>35000001000029</tns:serviceId>
<tns:timeStamp>20221019121812</tns:timeStamp>
</tns:RequestSOAPHeader>
</soapenv:Header>
<soapenv:Body>
<loc:startUSSDNotification>
<loc:reference>
<endpoint>http://172.16.130.193:8080/ussd</endpoint>
<correlator>09876</correlator>
</loc:reference>
<loc:ussdServiceActivationNumber>9475</loc:ussdServiceActivationNumber>
</loc:startUSSDNotification>
</soapenv:Body>
</soapenv:Envelope>
how do I remove the namespaces from the closing tags?
with the utmost gratitude.
Hello,
When I attempt to parse a WSDL file, I get traceback below, however, running the file through an XML Validator (http://www.w3schools.com/xml/xml_validator.asp) says there are no errors with the structure/content.
Here is the traceback:
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/home/jwolosonovich/.virtualenvs/ztrip_fare_estimator/lib/python3.4/site-packages/jxmlease/xmlparser.py", line 326, in parse
return Parser(**kwargs)(xml_input)
File "/home/jwolosonovich/.virtualenvs/ztrip_fare_estimator/lib/python3.4/site-packages/jxmlease/xmlparser.py", line 303, in __call__
self._parser.Parse(xml_input, True)
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 1, column 0
The beginning of the file looks like this:
<?xml version="1.0" encoding="utf-8"?>
<wsdl:definitions xmlns:tm="http://microsoft.com/wsdl/mime/textMatching/" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:mime="http://schemas.xmlsoap.org/wsdl/mime/" xmlns:tns="http://schemas.mtdata.com.au/dispatch/2.20.0/booking" xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:s="http://www.w3.org/2001/XMLSchema" xmlns:soap12="http://schemas.xmlsoap.org/wsdl/soap12/" xmlns:http="http://schemas.xmlsoap.org/wsdl/http/" targetNamespace="http://schemas.mtdata.com.au/dispatch/2.20.0/booking" xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/">
Currently, we can produce XML from a native Python data structure by doing this:
XMLDictNode({'a': {'b': 'foo'}}).emit_xml()
Wouldn't it be better to provide a standalone method (like json.dumps()
) to do the conversion in one shot? For example:
emit_xml({'a': {'b': 'foo'}}
Implementation details:
XMLNodeBase
data structure.
XMLDictNode
or XMLListNode
format.)emit_xml()
method of the resulting data structure.Given
<data1 xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" > <items> <item name="item1" time="now1"></item> <item name="item2" time="now2"></item> </items>
The code
tree_text = open('test.xml', 'r').read() parse = jxmlease.Parser() eparse = jxmlease.EtreeParser() adict = parse(tree_text) adict.prettyprint() print(f"['data1']['items']['item'][0]: {adict['data1']['items']['item'][0].get_xml_attrs()}") print(f"['data1']['items']['item'][1]: {adict['data1']['items']['item'][1].get_xml_attrs()}")
Renders:
{'data1': {'a': {'b': 'c'},
'fred': {'ethel': 'ethel_data'},
'items': {'item': ['', '']}}}
['data1']['items']['item'][0]: OrderedDict([('name', 'item2'), ('time', 'now2')])
['data1']['items']['item'][1]: OrderedDict([('name', 'item2'), ('time', 'now2')])
That is the last nodes attributes are the only ones provided.
When we parse XML, we should provide a way to iterate over all nodes at a given level, in the order received.
Currently, you can iterate over all nodes at a given level this way:
for node in root['a'].values():
for subnode in node.list():
pass
But, this has two problems:
Ideally, we should be able to do something like:
for node in root['a'].nodes():
pass
Expected implementation:
XMLDictNode
, update the node list.
__setitem__
, __delitem__
, and clear
methodsXMLListNode
, update the parent's ordered list.
__setitem__
, __delitem__
, __delslice__
, and pop
methods; however, this will probably actually require switching the
XMLListNodeto be a subclass of
MutableSequence`.Some quirks to be considered:
dict
method shouldn't modify order.Hello,
I'm trying to build a variable to pass to the REST API. What I've found is that when the variable is is passed to XMLDictNode and emit_xml is called, things get re-ordered and this causes the rest api to reject the request. If I create the config var by parsing an XML string, the reordering does not happen. I even used the output from print to configure the variable, but the reordering still happens when the variable is set directly. I've included a small test program that exhibits this behaviour. Any idea why this happens or how to avoid it?
#!/usr/bin/env python
"""test
"""
import sys
import email
import getpass
import pprint
import requests
import jxmlease
# Create a jxmlease parser with desired defaults.
parser = jxmlease.Parser()
def main():
"""The main loop.
"""
# parsing XML works
root = jxmlease.parse("""<?xml version="1.0" encoding="utf-8"?>
<configuration>
<logical-systems>
<name>R30</name>
<firewall>
<policer>
<name>30M</name>
<if-exceeding>
<bandwidth-limit>30m</bandwidth-limit>
<burst-size-limit>30M</burst-size-limit>
</if-exceeding>
<then>
<discard/>
</then>
</policer>
</firewall>
</logical-systems>
</configuration>""")
print('----root----')
print root
print('---- ----')
config_node = jxmlease.XMLDictNode(root)
print('---config_node---')
config_node.prettyprint()
print('---config_node.xml---')
print config_node.emit_xml()
print('**** ****')
# creating confg directly results in re-ordered xml output
config = {u'configuration': {u'logical-systems': {u'firewall': {u'policer': {u'if-exceeding': {u'bandwidth-limit': u'30m',
u'burst-size-limit': u'30M'},
u'name': u'30M',
u'then': {u'discard': u''}}},
u'name': u'R30'}}}
config_node = jxmlease.XMLDictNode(config)
print('---config_node---')
config_node.prettyprint()
print('---config_node.xml---')
print config_node.emit_xml()
print('---- ----')
return config_node
if __name__ == "__main__":
sys.exit(main())
$ ./py-cfg.py
----root----
{u'configuration': {u'logical-systems': {u'firewall': {u'policer': {u'if-exceeding': {u'bandwidth-limit': u'30m',
u'burst-size-limit': u'30M'},
u'name': u'30M',
u'then': {u'discard': u''}}},
u'name': u'R30'}}}
---- ----
---config_node---
{u'configuration': {u'logical-systems': {u'firewall': {u'policer': {u'if-exceeding': {u'bandwidth-limit': u'30m',
u'burst-size-limit': u'30M'},
u'name': u'30M',
u'then': {u'discard': u''}}},
u'name': u'R30'}}}
---config_node.xml---
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<logical-systems>
<name>R30</name>
<firewall>
<policer>
<name>30M</name>
<if-exceeding>
<bandwidth-limit>30m</bandwidth-limit>
<burst-size-limit>30M</burst-size-limit>
</if-exceeding>
<then>
<discard></discard>
</then>
</policer>
</firewall>
</logical-systems>
</configuration>
**** ****
---config_node---
{u'configuration': {u'logical-systems': {u'firewall': {u'policer': {u'if-exceeding': {u'bandwidth-limit': u'30m',
u'burst-size-limit': u'30M'},
u'name': u'30M',
u'then': {u'discard': u''}}},
u'name': u'R30'}}}
---config_node.xml---
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<logical-systems>
<firewall>
<policer>
<then>
<discard></discard>
</then>
<if-exceeding>
<bandwidth-limit>30m</bandwidth-limit>
<burst-size-limit>30M</burst-size-limit>
</if-exceeding>
<name>30M</name>
</policer>
</firewall>
<name>R30</name>
</logical-systems>
</configuration>
---- ----
{u'configuration': {u'logical-systems': {u'firewall': {u'policer': {u'if-exceeding': {u'bandwidth-limit': u'30m',
u'burst-size-limit': u'30M'},
u'name': u'30M',
u'then': {u'discard': u''}}},
u'name': u'R30'}}}
When parsing a corrupt XML document, the parser is returning a corrupt data structure and not raising an exception, as expected.
For example:
>>> import jxmlease
>>> root = jxmlease.parse("""\
... <a>
... <b>
... <c>foo</c>
... </b>
... """)
>>> root.prettyprint()
{u'b': {u'c': u'foo'}}
Would be good to have a new method to natively create Python JSON values (dict):
Example:
>>> data = jxmlease.parse('<a>123</a>')
>>> json = data.emit_json()
>>> print json
{u'a': u'123'}
>>> print json['a']
123
While the current implementation of prettyprint
is almost doing the same, the intend would not really be printing but using it with other Python libraries like deepdiff
.
Example:
>>> import deepdiff
>>> d1 = jxmlease.parse('<a>123</a>')
>>> d2 = jxmlease.parse('<a>456</a>')
>>> deepdiff.DeepDiff(d1, d2)
{'values_changed': {"root['a']": {'new_value': XMLCDATANode(xml_attrs=OrderedDict(), value=u'456'), 'old_value': XMLCDATANode(xml_attrs=OrderedDict(), value=u'123')}}}
>>>
>>> deepdiff.DeepDiff(d1.emit_json(), d2.emit_json())
{'values_changed': {"root['a']": {'new_value': u'456', 'old_value': u'123'}}}
In consequence the output of deepdiff would be very much more streamlined.
Hello,
I'm unable to get the routing instance status (inactive) in JSON
This is the code:
_filter_xml="""<configuration>
<routing-instances>
<instance>
<name>""" + _ri + """</name>
</instance>
</routing-instances>
</configuration>"""
response = dev.rpc.get_config(filter_xml=_filter_xml.replace("\n", ""))
rpc_xml = etree.tostring(response, pretty_print=True, encoding='unicode')
print(rpc_xml)
xmlparser = jxmlease.Parser(xml_attribs=True)
json_result = xmlparser(rpc_xml)
print json.dumps(json_result, indent=2)
PRINT RPC
<configuration changed-seconds="1538582209" changed-localtime="2018-10-03 17:56:49 CEST">
<routing-instances>
<instance inactive="inactive">
<name>CC1TEST</name>
<description>Fake RI for CC1 test automation</description>
<instance-type>virtual-router</instance-type>
</instance>
</routing-instances>
</configuration>
JSON
{
"configuration": {
"routing-instances": {
"instance": {
"name": "CC1TEST",
"description": "Fake RI for CC1 test automation",
"instance-type": "virtual-router"
}
}
}
}
Riccardo
I would like to have the ability in jxmlease to control how XML nodes w/o children are rendered. At the moment, this is done as empty string:
>>> x2 = jxmlease.parse("<a/>")
>>> x2
XMLDictNode(xml_attrs=OrderedDict(), value=OrderedDict([(u'a', XMLCDATANode(xml_attrs=OrderedDict(), value=u''))]))
>>> x2.prettyprint()
{u'a': u''}
So actually it means, that even has no children nodes, it is associated with an empty string. From an implementation point of view, I would like have the possibility to control how this is rendered. Technically it could be an empty list, empty dict or NoneType as well. Preferred implementation from my point of view would actually be the NoneType:
>>> x2 = jxmlease.parse("<a/>")
>>> x2.prettyprint()
{u'a': None}
Environment:
recreating issue:
>>> import jxmlease
>>>
>>> xml = """<api version="2">
<currentTime>2017-10-07 17:01:10</currentTime>
<result>
<rowset name="itemList" key="recordID" columns="recordID,typeID,quantity,rawQuantity,singleton,included">
<row recordID="2340848521" typeID="2319" quantity="2165" singleton="0" included="1" />
<row recordID="2340848522" typeID="2399" quantity="2103" singleton="0" included="1" />
<row recordID="2340848523" typeID="9832" quantity="4565" singleton="0" included="1" />
<row recordID="2340848524" typeID="2328" quantity="1500" singleton="0" included="1" />
<row recordID="2340848525" typeID="2398" quantity="15055" singleton="0" included="1" />
<row recordID="2340848526" typeID="3689" quantity="2525" singleton="0" included="1" />
<row recordID="2340848527" typeID="3693" quantity="2005" singleton="0" included="1" />
<row recordID="2340848528" typeID="9836" quantity="3605" singleton="0" included="1" />
<row recordID="2340848529" typeID="3645" quantity="22480" singleton="0" included="1" />
<row recordID="2340848530" typeID="2400" quantity="25822" singleton="0" included="1" />
<row recordID="2340848531" typeID="3691" quantity="2060" singleton="0" included="1" />
</rowset>
</result>
<cachedUntil>2027-10-05 17:01:10</cachedUntil>
</api>"""
>>>
>>> parsed_xml = jxmlease.parse(xml)
>>> for _row in parsed_xml.find_nodes_with_tag('row'):
_row
XMLCDATANode(xml_attrs=OrderedDict([('recordID', '2340848531'), ('typeID', '3691'), ('quantity', '2060'), ('singleton', '0'), ('included', '1')]), value='')
XMLCDATANode(xml_attrs=OrderedDict([('recordID', '2340848522'), ('typeID', '2399'), ('quantity', '2103'), ('singleton', '0'), ('included', '1')]), value='')
XMLCDATANode(xml_attrs=OrderedDict([('recordID', '2340848523'), ('typeID', '9832'), ('quantity', '4565'), ('singleton', '0'), ('included', '1')]), value='')
XMLCDATANode(xml_attrs=OrderedDict([('recordID', '2340848524'), ('typeID', '2328'), ('quantity', '1500'), ('singleton', '0'), ('included', '1')]), value='')
XMLCDATANode(xml_attrs=OrderedDict([('recordID', '2340848525'), ('typeID', '2398'), ('quantity', '15055'), ('singleton', '0'), ('included', '1')]), value='')
XMLCDATANode(xml_attrs=OrderedDict([('recordID', '2340848526'), ('typeID', '3689'), ('quantity', '2525'), ('singleton', '0'), ('included', '1')]), value='')
XMLCDATANode(xml_attrs=OrderedDict([('recordID', '2340848527'), ('typeID', '3693'), ('quantity', '2005'), ('singleton', '0'), ('included', '1')]), value='')
XMLCDATANode(xml_attrs=OrderedDict([('recordID', '2340848528'), ('typeID', '9836'), ('quantity', '3605'), ('singleton', '0'), ('included', '1')]), value='')
XMLCDATANode(xml_attrs=OrderedDict([('recordID', '2340848529'), ('typeID', '3645'), ('quantity', '22480'), ('singleton', '0'), ('included', '1')]), value='')
XMLCDATANode(xml_attrs=OrderedDict([('recordID', '2340848530'), ('typeID', '2400'), ('quantity', '25822'), ('singleton', '0'), ('included', '1')]), value='')
XMLCDATANode(xml_attrs=OrderedDict([('recordID', '2340848531'), ('typeID', '3691'), ('quantity', '2060'), ('singleton', '0'), ('included', '1')]), value='')
The prettyprint() function recursively moves over a jxmlease data structure and produces a new intermediate structure of native dicts and lists which is then feed into the standard pprint function (potentially using a small backup version instead). I wonder, would it not be better to split this in two parts, the first that generates the data structure and the second doing the prettyprinting. The reason I'm asking for this is that I think that intermediate data structure could come in handy, especially if it can be made to return OrderedDict, in converting to other data formats. Thoughts?
Hi All,
Some XML prolog attributes seem to be lost during parsing.
Issue:
# prolog of original xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
# prolog of xml after jxmlease parsing
<?xml version="1.0" encoding="utf-8"?>
Environment:
Reproduce:
#! /usr/bin/env python3
from jxmlease import Parser
parser = Parser()
root = parser('<?xml version="1.0" encoding="UTF-8" standalone="yes"?>')
print(root.emit_xml())
What is the syntax for defining attributes for the rendered xml from a dictionary?
Is this possible?
Something like the below does not work.
{
'Monkey': {
'xml_attrs': {'key': 'value'},
'Name': 'Lando Griffin'
}
}
Perhaps a convention where the attributes are prefixed by the @
symbol.
{
'Monkey': {
'@key': 'value',
'Name': 'Lando Griffin'
}
}
To render:
<Monkey key="value">
<Name>Lando Griffin</Name>
</Monkey>
Part of XML RPC:
<family>
<inet>
<filter>
<input inactive="inactive"> --> not parsed in OrderedDict
<filter-name>re-protection</filter-name>
</input>
</filter>
<address>
XMLCDATANode(xml_attrs=OrderedDict()
The OrderedDict is empty
pprint(root['node']['result']['rpc-reply']['data']['configuration']['interfaces']['interface']['unit'][
1]['family']['inet']['filter'].get_xml_attrs())
output: OrderedDict()
With expat 2.4.6 we're seeing the following tests fail in jxmlease 1.0.3:
======================================================================
ERROR: test_namespace_collapse (tests.test.XMLToObjTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/build/jxmlease-1.0.3/tests/test.py", line 524, in test_namespace_collapse
rv = self.parse(xml, process_namespaces=True, namespaces=namespaces)
File "/build/jxmlease-1.0.3/jxmlease/xmlparser.py", line 326, in parse
return Parser(**kwargs)(xml_input)
File "/build/jxmlease-1.0.3/jxmlease/xmlparser.py", line 303, in __call__
self._parser.Parse(xml_input, True)
xml.parsers.expat.ExpatError: out of memory: line 1, column 0
======================================================================
ERROR: test_namespace_support (tests.test.XMLToObjTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/build/jxmlease-1.0.3/tests/test.py", line 491, in test_namespace_support
rv = self.parse(xml, process_namespaces=True)
File "/build/jxmlease-1.0.3/jxmlease/xmlparser.py", line 326, in parse
return Parser(**kwargs)(xml_input)
File "/build/jxmlease-1.0.3/jxmlease/xmlparser.py", line 303, in __call__
self._parser.Parse(xml_input, True)
xml.parsers.expat.ExpatError: out of memory: line 1, column 0
----------------------------------------------------------------------
Ran 98 tests in 5.740s
FAILED (errors=2, skipped=8)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.