data1D.load fails to detect header rows about refnx HOT 7 CLOSED

Please provide an example. On some package versions it's been a ValueError, which is why that Exception was used. I was trying to make a clever reader that would autodetect how many header rows were used. But it might make sense to have a keyword.

from refnx.

example data file is in the original comment. I'm using refnx version 0.1.1.dev0+e71da24 on Windows 10.

Error log:

ParseError Traceback (most recent call last)
~\Anaconda3\envs\BleedingRefnx\lib\site-packages\refnx-0.1.1.dev0+e71da24-py3.6-win-amd64.egg\refnx\dataset\reflectdataset.py in load(self, f)
117 tree = ET.ElementTree()
--> 118 tree.parse(f)
119

~\Anaconda3\envs\BleedingRefnx\lib\xml\etree\ElementTree.py in parse(self, source, parser)
596 # it with chunks.
--> 597 self._root = parser._parse_whole(source)
598 return self._root

ParseError: syntax error: line 1, column 0

During handling of the above exception, another exception occurred:

ValueError Traceback (most recent call last)
in ()
1 name = 'W335 dry'
----> 2 data = RD('PPR/c_PLP0033833.dat')

~\Anaconda3\envs\BleedingRefnx\lib\site-packages\refnx-0.1.1.dev0+e71da24-py3.6-win-amd64.egg\refnx\dataset\reflectdataset.py in init(self, data, **kwds)
64 All arrays must have the same shape.
65 """
---> 66 super(ReflectDataset, self).init(data=data, **kwds)
67 self.datafilenumber = list()
68 self.sld_profile = None

~\Anaconda3\envs\BleedingRefnx\lib\site-packages\refnx-0.1.1.dev0+e71da24-py3.6-win-amd64.egg\refnx\dataset\data1d.py in init(self, data, mask, **kwds)
77 # if it's a file then open and load the file.
78 if hasattr(data, 'read') or type(data) is str:
---> 79 self.load(data)
80 elif data is not None:
81 self._x = np.array(data[0], dtype=float)

~\Anaconda3\envs\BleedingRefnx\lib\site-packages\refnx-0.1.1.dev0+e71da24-py3.6-win-amd64.egg\refnx\dataset\reflectdataset.py in load(self, f)
133 self.data = (qvals, rvals, drvals, dqvals)
134 except ET.ParseError:
--> 135 super(ReflectDataset, self).load(fname)

~\Anaconda3\envs\BleedingRefnx\lib\site-packages\refnx-0.1.1.dev0+e71da24-py3.6-win-amd64.egg\refnx\dataset\data1d.py in load(self, f)
397 continue
398
--> 399 self.data = np.loadtxt(f, unpack=True, skiprows=header_lines)
400
401 if hasattr(f, 'read'):

~\Anaconda3\envs\BleedingRefnx\lib\site-packages\numpy\lib\npyio.py in loadtxt(fname, dtype, comments, delimiter, converters, skiprows, usecols, unpack, ndmin)
1022
1023 # Convert each value according to its column and store
-> 1024 items = [conv(val) for (conv, val) in zip(converters, vals)]
1025 # Then pack it according to the dtype's nesting
1026 items = pack_items(items, packing)

~\Anaconda3\envs\BleedingRefnx\lib\site-packages\numpy\lib\npyio.py in (.0)
1022
1023 # Convert each value according to its column and store
-> 1024 items = [conv(val) for (conv, val) in zip(converters, vals)]
1025 # Then pack it according to the dtype's nesting
1026 items = pack_items(items, packing)

~\Anaconda3\envs\BleedingRefnx\lib\site-packages\numpy\lib\npyio.py in floatconv(x)
723 if b'0x' in x:
724 return float.fromhex(asstr(x))
--> 725 return float(x)
726
727 typ = dtype.type

ValueError: could not convert string to float: b'Q'

from refnx.

ok, I didn't see the file in the comment as I was reading on my phone. I'll look into it.

from refnx.

I try that file on my computer, and it works without issue. The bit of code that determines how many header rows there could be could probably be improved, but that will require a regex wizard.
ReflectDataset is passing the load up to the base class to read, because it's not an XML file. It's that baseclass load that's the issue. I'm not sure why you see it and I don't.
What version of numpy are you using?

from refnx.

numpy 1.14.1.

Could be some sort of Mac/PC file handling issue?

from refnx.

Maybe end of line vs. carriage return bug related to how the different OSs are parsing the file?

from refnx.

Hopefully closed by #237. If it's not, then please reopen.

from refnx.