iris-edu / irisfetch-matlab Goto Github PK

View Code? Open in Web Editor NEW

10.0 10.0 8.0 110 KB

MATLAB 100.00%

irisfetch-matlab's People

Contributors

Stargazers

Watchers

Forkers

sevenysw falconkbo ogweno joelsimon arinsiskai jiangwb wanyingw akashkharita

irisfetch-matlab's Issues

no data incorrectly returned with federated requests that use 'includePZ' flag

Federated requests go to multiple datacenters, which may not host a version of the sacpz web service. When irisFetch.Traces method invokes the 'includePZ' option, this can result in a "no data" error despite the datacenter having the timeseries and associated metadata available.

This behavior is not exclusive to federated requests and will occur if the sacpz web service is generally unreachable.

An example:

tr = irisFetch.Traces('II,GR','BFO,XBFO','*','LHZ','2019-08-17 00:00:00','2019-08-18 00:00:00','includePZ','federated','verbose')
Fetching data from BGR (http://eida.bgr.de)
[BGR] : DATACENTER > http://eida.bgr.de
[BGR] : DATASELECTSERVICE > http://eida.bgr.de/fdsnws/dataselect/1/
[BGR] : STATIONSERVICE > http://eida.bgr.de/fdsnws/station/1/
query : net=II,GR&sta=BFO,XBFO&loc=*&cha=LHZ&start=2019-08-17T00:00:00.000&end=2019-08-18T00:00:00.000
traces = tracedata.fetchTraces("GR", "BFO", "--", "LHZ", "2019-08-17 00:00:00.000", "2019-08-18 00:00:00.000", 'M', 1)
2019/09/09 10:33:20 INFO: edu.iris.dmc.extensions.fetch.TraceData fetchTraces: StationService url set to: http://eida.bgr.de/fdsnws/station/1/
2019/09/09 10:33:21 FINER: edu.iris.dmc.extensions.fetch.TraceData usrMessage: Found  1 applicable metadata
2019/09/09 10:33:21 INFO: edu.iris.dmc.extensions.fetch.TraceData fetchTraces: WaveService url set to: http://eida.bgr.de/fdsnws/dataselect/1/
2019/09/09 10:33:21 FINER: edu.iris.dmc.extensions.fetch.TraceData usrMessage: Requesting: GR BFO -- LHZ 2019-08-17T00:00:00.000 2019-08-18T00:00:00.000
2019/09/09 10:33:21 FINER: edu.iris.dmc.extensions.fetch.TraceData usrMessage: Fetching waveform data
2019/09/09 10:33:22 INFO: edu.iris.dmc.extensions.fetch.TraceData fetchTraces: SacpzService url set to: http://service.iris.edu/irisws/sacpz/1/
2019/09/09 10:33:22 FINER: edu.iris.dmc.extensions.fetch.TraceData usrMessage: Matching waveform to metadata: 
	GR,BFO,,LHZ,48.33007,8.329563,589.0,0.0,0.0,-90.0,null,2.5169897E9,1.0,m/s,1.0,2011-10-19T00:00:00

An [MATLAB:Java:GenericException] exception occurred in irisFetch.getTheTraces() but was caught
 full text follows:
message:
Java exception occurred: 
java.io.IOException
	at edu.iris.dmc.service.SacpzService.fetch(SacpzService.java:463)
	at edu.iris.dmc.extensions.fetch.TraceData.fetchTraces(TraceData.java:464)
	at edu.iris.dmc.extensions.fetch.TraceData.fetchTraces(TraceData.java:202)
	at edu.iris.dmc.extensions.fetch.TraceData.fetchTraces(TraceData.java:143)
	at edu.iris.dmc.extensions.fetch.TraceData.fetchTraces(TraceData.java:107)

Received 0 channels in 1.4 seconds

irisFetch.Traces() returns 204 error when requesting data that should be available.

I am getting errors when making irisFetch.Traces() requests, using version '2.0.12', cloned from Git.
My startDate and endDate values are set programmatically, and are generated with type "datetime".
I am making the request as follows:

data = irisFetch.Traces('IM', 'H01W1', '--', 'EDH', chunk_start, chunk_end);

Here, "chunk_start" and "chunk_end" are 1x1 "datetime" arrays. This results in the following error:

edu.iris.dmc.service.NoDataFoundException: (204) No data found for: quality=M IM H01W1 -- EDH 2010-01-01T00:00:00.000 2010-01-02T00:00:00.000 at edu.iris.dmc.service.WaveformService.fetch(WaveformService.java:116) at edu.iris.dmc.service.WaveformService.fetch(WaveformService.java:239) at edu.iris.dmc.service.WaveformService.fetch(WaveformService.java:292) at edu.iris.dmc.extensions.fetch.TraceData.fetchTraces(TraceData.java:394) at edu.iris.dmc.extensions.fetch.TraceData.fetchTraces(TraceData.java:192) at edu.iris.dmc.extensions.fetch.TraceData.fetchTraces(TraceData.java:137) at edu.iris.dmc.extensions.fetch.TraceData.fetchTraces(TraceData.java:102)

According to this page, the data for the dates I am requesting are available.

I have tried specifying other values for "quality" but this does not prevent the error.
I gave tried converting the 'datetime' arrays to strings, resulting in the following request:
data = irisFetch.Traces('IM', 'H01W1', '--', 'EDH', '2010-01-01 00:00:00', '2010-01-02 00:00:00');

This results in an identical error.
Any assistance would be most appreciated.

Thanks,
Ben

problems with specifying other FDSN event services

A couple problems when trying to specify non-IRIS fdsnws-event service URLs

The IRIS-WS library wants to append the below parameters to every request. In the v 1.1 version of the FDSN spec, support for these parameters is considered optional. This is especially problematic for IRIS PH5 services.
- includearrivals=false
- includeallmagnitudes=false
- includeallorigins=false
The following error occurs when trying to send requests to the USGS event web service (https://earthquake.usgs.gov/fdsnws/event/1/). Potentially related to https vs http ?

Oct 15, 2019 10:10:10 PM com.sun.xml.internal.bind.v2.util.XmlFactory createTransformerFactory
SEVERE: null
javax.xml.transform.TransformerConfigurationException: Unsupported feature: http://javax.xml.XMLConstants/feature/secure-processing
at com.icl.saxon.TransformerFactoryImpl.setFeature(TransformerFactoryImpl.java:373)
at com.sun.xml.internal.bind.v2.util.XmlFactory.createTransformerFactory(XmlFactory.java:169)
at com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl.createTransformerHandler(JAXBContextImpl.java:712)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.DomLoader$State.(DomLoader.java:60)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.DomLoader.startElement(DomLoader.java:103)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.ProxyLoader.startElement(ProxyLoader.java:45)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallingContext._startElement(UnmarshallingContext.java:559)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallingContext.startElement(UnmarshallingContext.java:538)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.InterningXmlVisitor.startElement(InterningXmlVisitor.java:60)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.StAXEventConnector.handleStartElement(StAXEventConnector.java:246)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.StAXEventConnector.bridge(StAXEventConnector.java:115)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:445)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:415)
at edu.iris.dmc.service.EventService.fetch(EventService.java:182)
at edu.iris.dmc.service.EventService.fetch(EventService.java:113)

Error using irisFetch.Events (line 983)
Java exception occurred:
java.lang.IllegalStateException: javax.xml.transform.TransformerConfigurationException:
Unsupported feature: http://javax.xml.XMLConstants/feature/secure-processing
at
com.sun.xml.internal.bind.v2.util.XmlFactory.createTransformerFactory(XmlFactory.java:173)
at
com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl.createTransformerHandler(JAXBContextImpl.java:712)
at
com.sun.xml.internal.bind.v2.runtime.unmarshaller.DomLoader$State.(DomLoader.java:60)
at
com.sun.xml.internal.bind.v2.runtime.unmarshaller.DomLoader.startElement(DomLoader.java:103)
at
com.sun.xml.internal.bind.v2.runtime.unmarshaller.ProxyLoader.startElement(ProxyLoader.java:45)
at
com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallingContext._startElement(UnmarshallingContext.java:559)
at
com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallingContext.startElement(UnmarshallingContext.java:538)
at
com.sun.xml.internal.bind.v2.runtime.unmarshaller.InterningXmlVisitor.startElement(InterningXmlVisitor.java:60)
at
com.sun.xml.internal.bind.v2.runtime.unmarshaller.StAXEventConnector.handleStartElement(StAXEventConnector.java:246)
at
com.sun.xml.internal.bind.v2.runtime.unmarshaller.StAXEventConnector.bridge(StAXEventConnector.java:115)
at
com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:445)
at
com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:415)
at edu.iris.dmc.service.EventService.fetch(EventService.java:182)
at edu.iris.dmc.service.EventService.fetch(EventService.java:113)
Caused by: javax.xml.transform.TransformerConfigurationException: Unsupported feature:
http://javax.xml.XMLConstants/feature/secure-processing
at com.icl.saxon.TransformerFactoryImpl.setFeature(TransformerFactoryImpl.java:373)
at
com.sun.xml.internal.bind.v2.util.XmlFactory.createTransformerFactory(XmlFactory.java:169)
... 13 more

memory problems with SAC2Trace

Some memory issues related to keeping a file identifier "open" during the conversion of SAC data to Trace structure. Can be fixed by including a fclose(fid) type of call. See below for more details.

From Zach Eilon (March 7 email):

Hi both, just a quick note for a bug that I found in the irisFetch package.

There is an fclose(fid)missing in the irisFetch.read_sac_in function at line 2420. If you look closely at the case that hn(77)~=6, you will note that files are opened but never closed. This led to memory issues and MATLAB crashing when I used the irisFetch.SAC2Trace tool on multiple files.

Hope you can correct that for future versions so that others avoid the problem.

Best,
Zach

irisFetch doesn't recognize https for specifying a new base url

Specifying a new base url as described here (https://ds.iris.edu/ds/nodes/dmc/software/downloads/irisFetch.m/2-0-10/manual/#retrieving-data-from-global-fdsn-data-centers-and-the-iris-federator) doesn't work if your webservices URL has a https address. Line 434 should be changed to something like
if length(param)>7 && (strcmpi(param(1:7),'http://') || strcmpi(param(1:7),'https:/'))
to get it to work.

standardize input parameters for alternate data centers

The way to specify web services hosted by other FDSN datacenters is different for various irisFetch methods. For instance, using the Stations method, it's required to specify parameter baseurl and then list out the full fdsnws-station service endpoint of the other data center but when using the Traces method, the baseurl parameter is omitted and you just list the data center host name without the full endpoint.

These examples show the inconsistency:

st=irisFetch.Stations('CHANNEL','BK','ARC','*','BHZ','BASEURL','http://service.ncedc.org/fdsnws/station/1/')

tr=irisFetch.Traces('BK','ARC','*','BHE','2000-01-01 00:00:00','2000-01-01 01:00:00','http://service.ncedc.org')

edited: the BASEURL parameter is also required for Events, Channels, and Networks methods

suppress uncommented lines

When using the Traces method and setting options for using alternate datacenter endpoints for the station and dataselect services, the opts structure will be uncommented. This should be a "silent" operation.

tr = irisFetch.Traces('FR','*','*','BHZ','2008-09-01 00:00:00','2009-01-01 00:00:00','DATASELECTURL:http://ws.resif.fr/fdsnws/dataselect/1/','STATIONURL:http://ws.resif.fr/fdsnws/station/1/','verbose')

opts = 

  struct with fields:

           getsacpz: 0
          verbosity: 0
          authorize: 0
            quality: 'M'
           username: ''
            userpwd: ''
            newbase: ''
      dataselectURL: 'http://ws.resif.fr/fdsnws/dataselect/1/'
         stationURL: ''
       useFederator: 0
     writeDirectory: ''
    convertToMatlab: 1
            saveSAC: 0


opts = 

  struct with fields:

           getsacpz: 0
          verbosity: 0
          authorize: 0
            quality: 'M'
           username: ''
            userpwd: ''
            newbase: ''
      dataselectURL: 'http://ws.resif.fr/fdsnws/dataselect/1/'
         stationURL: 'http://ws.resif.fr/fdsnws/station/1/'
       useFederator: 0
     writeDirectory: ''
    convertToMatlab: 1
            saveSAC: 0

    'spz:'    [0]    'vb:'    [1]    'auth:'    [0]    'qual:'    'M'    'un&pw:'    ''    ''

Using dataselect service at: http://ws.resif.fr/fdsnws/dataselect/1/
Using station service at: http://ws.resif.fr/fdsnws/station/1/

crash on empty Azimuth in StationXML

There are many channels that do not have an Azimuth field defined in the StationXML metadata. This field is not required by the stationxml-validator and is commonly missing for non-seismic (e.g. BDF) channel codes. This causes problems for irisFetch because the library treats it as a required field. There is likely a similar issue with Dip.

Example: IM.I52H1,I51H4.--.BDF
Channel-level metadata for station I52H1 DOES NOT include an Azimuth or Dip field, but station I51H4 DOES.
https://service.iris.edu/fdsnws/station/1/query?net=IM&sta=I52H1,I51H4&cha=BDF&level=channel&starttime=2022-01-15T00:00:00

<Station code="I51H4" startDate="2008-11-10T00:00:00.0000" restrictedStatus="open" iris:alternateNetworkCodes="_INT-NON_FDSN,_REALTIME,.UNRESTRICTED">
    <Latitude>32.349571</Latitude>
    <Longitude>-64.711037</Longitude>
    <Elevation>12.6</Elevation>
    <Site>
        <Name>Bermuda Infrasound Array, Site H4, United Kingdom</Name>
    </Site>
    <TotalNumberChannels>1</TotalNumberChannels>
    <SelectedNumberChannels>1</SelectedNumberChannels>
    <Channel code="BDF" locationCode="" startDate="2008-11-10T00:00:00.0000" restrictedStatus="open">
        <Latitude>32.349571</Latitude>
        <Longitude>-64.711037</Longitude>
        <Elevation>12.6</Elevation>
        <Depth>0</Depth>
        <Azimuth>0</Azimuth>
        <Dip>0</Dip>
        <Type>CONTINUOUS</Type>
        <Type>GEOPHYSICAL</Type>
        <SampleRate>2E01</SampleRate>
        <ClockDrift>0E00</ClockDrift>
        ...

<Station code="I52H1" startDate="2002-12-18T00:00:00.0000" restrictedStatus="open" iris:alternateNetworkCodes="_INT-NON_FDSN,_REALTIME,.UNRESTRICTED">
    <Latitude>-7.37779</Latitude>
    <Longitude>72.484169</Longitude>
    <Elevation>2.3</Elevation>
    <Site>
        <Name>Diego Garcia infrasound array site H1, United Kingdom</Name>
    </Site>
    <CreationDate>2002-12-18T00:00:00.0000</CreationDate>
    <TotalNumberChannels>6</TotalNumberChannels>
    <SelectedNumberChannels>1</SelectedNumberChannels>
    <Channel code="BDF" locationCode="" startDate="2021-02-27T00:00:00.0000" restrictedStatus="open">
        <Latitude>-7.37779</Latitude>
        <Longitude>72.484169</Longitude>
        <Elevation>2.3</Elevation>
        <Depth>0</Depth>
        <Type>CONTINUOUS</Type>
        <Type>GEOPHYSICAL</Type>
        <SampleRate>2E01</SampleRate>
        ...

In Matlab, this creates a NullPointerException:

traces = tracedata.fetchTraces("IM", "I52H1", "*", "BDF", "2021-01-15 00:00:00.000", "2021-01-15 00:00:00.000", 'M', 0) 2022/01/31 14:50:49 INFO: edu.iris.dmc.extensions.fetch.TraceData fetchTraces: StationService url set to: http://service.iris.edu/fdsnws/station/1/ An [MATLAB:Java:GenericException] exception occurred in irisFetch.getTheTraces() but was caught full text follows: message: Java exception occurred: java.lang.NullPointerException at edu.iris.dmc.extensions.entities.Metadata.parseMetadata(Metadata.java:239) at edu.iris.dmc.extensions.fetch.TraceData.fetchTraces(TraceData.java:325) at edu.iris.dmc.extensions.fetch.TraceData.fetchTraces(TraceData.java:192) at edu.iris.dmc.extensions.fetch.TraceData.fetchTraces(TraceData.java:137) at edu.iris.dmc.extensions.fetch.TraceData.fetchTraces(TraceData.java:102)

There may still be issues with the underlying Matlab code in dealing with an empty/null Azimuth (and Dip), but those problems are difficult to address without a successful request made through the library.

irisFetch does not support V7 SAC files

I posted this issue on SAC Help and was told to make the issue known here.

In the words of one commenter: "Briefly, the code cannot handle V7 SAC files properly; if the version number is not 6, the code assumes it is just a byte-swapped SAC file. It does not recognize that V7 headers exist."

http://ds.iris.edu/message-center/thread/8336/#m-11223

Please help with this issue! What other options to I have in the meantime? Using sac2mat?

Compatibility with PH5 services

Currently, irisFetch is not compatible with the suite of PH5 web services. The underlying dependency on the IRIS-WS Java library, by default, adds some parameters to the event and dataselect service calls that are not recognized/applicable to the ph5ws-* services but are compliant with the FDSN spec.

As an example:

tr = irisFetch.Traces('Z1','1','*','DPZ','2018-07-01 00:00:00','2018-07-01 00:00:10','DATASELECTURL:http://service.iris.edu/ph5ws/dataselect/1/','STATIONURL:http://service.iris.edu/ph5ws/station/1/','verbose')

    'spz:'    [0]    'vb:'    [1]    'auth:'    [0]    'qual:'    'M'    'un&pw:'    ''    ''

Using dataselect service at: http://service.iris.edu/ph5ws/dataselect/1/
Using station service at: http://service.iris.edu/ph5ws/station/1/
traces = tracedata.fetchTraces("Z1", "1", "*", "DPZ", "2018-07-01 00:00:00.000", "2018-07-01 00:00:10.000", 'M', 0)
2022/02/23 19:34:29 INFO: edu.iris.dmc.extensions.fetch.TraceData fetchTraces: StationService url set to: http://service.iris.edu/ph5ws/station/1/
2022/02/23 19:34:29 FINER: edu.iris.dmc.extensions.fetch.TraceData usrMessage: Found  1 applicable metadata
2022/02/23 19:34:29 INFO: edu.iris.dmc.extensions.fetch.TraceData fetchTraces: WaveService url set to: http://service.iris.edu/ph5ws/dataselect/1/
2022/02/23 19:34:29 FINER: edu.iris.dmc.extensions.fetch.TraceData usrMessage: Requesting: Z1 1 -- DPZ 2018-07-01T00:00:00.000 2018-07-01T00:00:10.000
2022/02/23 19:34:29 FINER: edu.iris.dmc.extensions.fetch.TraceData usrMessage: Fetching waveform data
edu.iris.dmc.criteria.CriteriaException: Bad request parameter: Error 400: Unrecognized argument - quality -- Exception while reading stderr, excep: Stream closed

More Details:
handler exited, code: 3  reason: Bad Request

Request:
http://service.iris.edu/ph5ws/dataselect/1/query

Request Submitted:
2022/02/24 03:34:31 UTC

Service version:
Service: ph5ws-dataselect  version: 1.0.0


	at edu.iris.dmc.service.WaveformService.fetch(WaveformService.java:120)
	at edu.iris.dmc.service.WaveformService.fetch(WaveformService.java:239)
	at edu.iris.dmc.service.WaveformService.fetch(WaveformService.java:292)
	at edu.iris.dmc.extensions.fetch.TraceData.fetchTraces(TraceData.java:394)
	at edu.iris.dmc.extensions.fetch.TraceData.fetchTraces(TraceData.java:192)
	at edu.iris.dmc.extensions.fetch.TraceData.fetchTraces(TraceData.java:137)
	at edu.iris.dmc.extensions.fetch.TraceData.fetchTraces(TraceData.java:102)
An [MATLAB:Java:GenericException] exception occurred in irisFetch.getTheTraces() but was caught
 full text follows:
message:
Java exception occurred: 
java.io.IOException: edu.iris.dmc.criteria.CriteriaException: Bad request parameter: Error 400: Unrecognized argument - quality -- Exception while reading stderr, excep: Stream closed

More Details:
handler exited, code: 3  reason: Bad Request

The full list of parameters that need to be validated, the variable type, and the corresponding web service is below

ph5ws-event:

includearrivals (boolean)
includeallmagnitudes (boolean)
includeallorigins (boolean)

ph5ws-dataselect:

quality (single-character string)

Requesting null instrument response

There are some channels that have timeseries data but do not include instrument response information. An example is the hydrophone data associated with the Ocean Observatories network (net=OO). When these "null" response channels are requested while using the includePZ option, the Traces method will crash and not return any timeseries data. This is due to how the IRIS-WS library interacts with the irisws-sacpz service to get response information.

The result is that the exception below will crash and not return any data at all. Ideally, what ought to happen is for the library to skip past that channel's empty response information and continue to serve the timeseries data as well as the response information, when applicable.

trace = irisFetch.Traces('OO','AXCC1','*','HHZ,HDH',736696,736696+1/24/60,'includePZ','verbose');

An [MATLAB:Java:GenericException] exception occurred in irisFetch.getTheTraces() but was caught full text follows:
message:
Java exception occurred: 
edu.iris.dmc.service.DataFormatException: Unable to parse data: Constant not found where expected
    at edu.iris.dmc.service.SacpzService.fetch(SacpzService.java:432)
    at edu.iris.dmc.extensions.fetch.TraceData.fetchTraces(TraceData.java:465)
    at edu.iris.dmc.extensions.fetch.TraceData.fetchTraces(TraceData.java:203)
    at edu.iris.dmc.extensions.fetch.TraceData.fetchTraces(TraceData.java:144)
    at edu.iris.dmc.extensions.fetch.TraceData.fetchTraces(TraceData.java:108)

'inlcudePZ' no longer working

Adding the 'includePZ' flag now causes a java error and is unable to retrieve traces or the poles and zeros.

As an example
mytrace=irisFetch.Traces('IU','ANMO','00','BHZ','2010-02-27 06:30:00','2010-02-27 07:30:00','includePZ')
Produces the error:

java.text.ParseException: Unparseable date: ""
at java.text.DateFormat.parse(DateFormat.java:357)
at edu.iris.dmc.service.SacpzService.fetch(SacpzService.java:195)
at edu.iris.dmc.extensions.fetch.TraceData.fetchTraces(TraceData.java:464)
at edu.iris.dmc.extensions.fetch.TraceData.fetchTraces(TraceData.java:202)
at edu.iris.dmc.extensions.fetch.TraceData.fetchTraces(TraceData.java:143)
at edu.iris.dmc.extensions.fetch.TraceData.fetchTraces(TraceData.java:107)

and mytrace is an empty 1x0 structure. Removing 'includePZ' from the function call works as expected with no errors.

Any help would be much appreciated!

Edit: Issue has been reproduced with IRIS-WS-2.0.18.jar, IRIS-WS-2.0.17.jar, IRIS-WS-2.0.15.jar, and IRIS-WS-2.0.6.jar.
Matlab versions 2014a and 2017a

TransformerConfigurationException during processing of HTTP response with `Set-Cookie: HttpOnly;Secure;SameSite=Lax` header

Dear all,

The configuration of the KNMI servers providing seismic data has been recently altered with the introduction of an additional HTTP response header:

Set-Cookie: HttpOnly;Secure;SameSite=Lax

It seems that this causes the irisFetch library to fail - please see our case below:

Matlab statement:

javaaddpath('/home/user/tools/IRIS-WS-2.20.1.jar')
s = irisFetch.Stations('station','NL,NR','*','*','*Z','BASEURL','http://rdsa.knmi.nl/fdsnws/station/1/'); % for KNMI

Exception:

{Transfer-Encoding=[chunked], Keep-Alive=[timeout=5, max=100], null=[HTTP/1.1 200 OK], Access-Control-Expose-Headers=[WWW-Authenticate], Server=[SeisComP-FDSNWS/1.2.4], Access-Control-Allow-Origin=[*], Connection=[Keep-Alive], Set-Cookie=[HttpOnly;Secure;SameSite=Lax], Access-Control-Allow-Headers=[Authorization], Date=[Tue, 23 Jan 2024 13:47:19 GMT], Content-Type=[application/xml]}
http://rdsa.knmi.nl/fdsnws/station/1/query?net=NL,NR&sta=*&cha=*Z&loc=*&level=station&format=xml:null
Jan 23, 2024 2:47:20 PM com.sun.xml.internal.bind.v2.util.XmlFactory createTransformerFactory
SEVERE: null
javax.xml.transform.TransformerConfigurationException: Unsupported feature: http://javax.xml.xmlconstants/feature/secure-processing
    at com.icl.saxon.TransformerFactoryImpl.setFeature(TransformerFactoryImpl.java:373)
    at com.sun.xml.internal.bind.v2.util.XmlFactory.createTransformerFactory(XmlFactory.java:169)
    at com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl.createTransformerHandler(JAXBContextImpl.java:712)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.DomLoader$State.<init>(DomLoader.java:60)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.DomLoader.startElement(DomLoader.java:103)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.ProxyLoader.startElement(ProxyLoader.java:45)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallingContext._startElement(UnmarshallingContext.java:559)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallingContext.startElement(UnmarshallingContext.java:538)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.InterningXmlVisitor.startElement(InterningXmlVisitor.java:60)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.StAXEventConnector.handleStartElement(StAXEventConnector.java:246)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.StAXEventConnector.bridge(StAXEventConnector.java:115)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:445)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:415)
    at edu.iris.dmc.service.station.parser.StationXMLParser.parse(StationXMLParser.java:151)
    at edu.iris.dmc.service.StationService.fetch(StationService.java:314)
    at edu.iris.dmc.service.StationService.fetch(StationService.java:262)
    at edu.iris.dmc.service.StationService.fetch(StationService.java:272)

Error using irisFetch/Networks/fetchTheStations (line 809)
Java exception occurred:
java.lang.IllegalStateException: javax.xml.transform.TransformerConfigurationException: Unsupported feature:
http://javax.xml.xmlconstants/feature/secure-processing
    at com.sun.xml.internal.bind.v2.util.XmlFactory.createTransformerFactory(XmlFactory.java:173)
    at com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl.createTransformerHandler(JAXBContextImpl.java:712)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.DomLoader$State.<init>(DomLoader.java:60)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.DomLoader.startElement(DomLoader.java:103)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.ProxyLoader.startElement(ProxyLoader.java:45)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallingContext._startElement(UnmarshallingContext.java:559)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallingContext.startElement(UnmarshallingContext.java:538)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.InterningXmlVisitor.startElement(InterningXmlVisitor.java:60)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.StAXEventConnector.handleStartElement(StAXEventConnector.java:246)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.StAXEventConnector.bridge(StAXEventConnector.java:115)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:445)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:415)
    at edu.iris.dmc.service.station.parser.StationXMLParser.parse(StationXMLParser.java:151)
    at edu.iris.dmc.service.StationService.fetch(StationService.java:314)
    at edu.iris.dmc.service.StationService.fetch(StationService.java:262)
    at edu.iris.dmc.service.StationService.fetch(StationService.java:272)
Caused by: javax.xml.transform.TransformerConfigurationException: Unsupported feature:
http://javax.xml.xmlconstants/feature/secure-processing
    at com.icl.saxon.TransformerFactoryImpl.setFeature(TransformerFactoryImpl.java:373)
    at com.sun.xml.internal.bind.v2.util.XmlFactory.createTransformerFactory(XmlFactory.java:169)
    ... 15 more

Error in irisFetch.Networks (line 713)
         fetchTheStations();

Error in irisFetch.Stations (line 639)
         [stationStructure, urlParams] = irisFetch.Networks(detailLevel,varargin{:});

Error in make_KNMIloc (line 12)
s = irisFetch.Stations('station','NL,NR','*','*','*Z','BASEURL','http://rdsa.knmi.nl/fdsnws/station/1/'); % for KNMI

We have tested it with FDSNWSs provided by other DCs responding without Set-Cookie: HttpOnly;Secure;SameSite=Lax header and those seem to work fine.

We would be grateful if you could look into this issue. Deleting this header is not permitted according to our security policy, so either a solution or a recommendation on how to address it would be greatly appreciated.

Regards,
Jarek

Java representation of StationXML values instead of numeric

When using the metadata search methods (.Networks,.Stations,.Channels), the returned values for the SensitivityFrequency and SensitivityValue fields are returned as Java class representations of a number, instead of an actual number.

>> mydata = irisFetch.Networks('channel','*','*','*','BHZ','minlatitude',40,'maxlatitude',45,'minlongitude',100,'maxlongitude',150,'starttime','2010-01-01 00:00:00')

>> mydata(1).Stations(1).Channels

ans = 

  struct with fields:

                   ChannelCode: 'BHZ'
                  LocationCode: ''
                   Description: ''
                          Type: '[CONTINUOUS, GEOPHYSICAL]'
                      Response: [1×1 struct]
          SensitivityFrequency: [1×1 java.lang.Double]
    SensitivityUnitDescription: 'Velocity in Meters per Second'
              SensitivityUnits: 'M/S'
              SensitivityValue: [1×1 java.lang.Double]
                     Equipment: []
               SampleRateRatio: []
                 StorageFormat: ''
                    ClockDrift: 0.05
                    DataLogger: []
                        Sensor: [1×1 struct]
                  PreAmplifier: []
             ExternalReference: []
                      Latitude: 41.994
                     Longitude: 128.08
                     Elevation: 2648
                         Depth: 0
                       Azimuth: 0
                           Dip: -90
                    SampleRate: 50
              CalibrationUnits: [1×1 struct]
                     StartDate: '2013-08-06 00:00:00.000'
                       EndDate: '2013-10-11 23:59:59.000'
              DataAvailability: []
              RestrictedStatus: 'OPEN'
                 AlternateCode: ''
                HistoricalCode: ''

irisFetch.Resp method needs more documentation

The current, incode help for the irisFetch.Resp method is pretty thin. In particular it does not include the calling convention (it is displayed in the GUI popup for the method, but we shouldn't rely on solely).

Adding an example would be good too, e.g.:
resp = irisFetch.Resp('IU','COLA','00','LHZ','2000-01-01 00:00:00','2020-01-01 00:00:00');

Current help:

>> help irisFetch.Resp
  retrieve the Resp information into a character string.
  net, sta, loc, and cha are all required.
  channels and locations may be wildcarded using either ? or *
  starttime and endtime options may be ignored by using [] instead of a time.

Channel-level metadata search breaks for some response types

Using any of the metadata retrieval methods (ie., irisFetch.Networks, irisFetch.Channels) for some instruments that have a polynomial response type while specifying the @channel@ level will result in an error. The issue is that the case-switch statement does not account for the class being sent by the IRIS-WS library.

Example:

>> polyres = irisFetch.Channels('channel','BK','BDM','*','LDS','starttime','2005-01-01 00:00:00')
Warning: edu.iris.dmc.fdsn.station.model.Coefficient was not found 
> In irisFetch/parse (line 2134)
In irisFetch/parseAnArray (line 2204)
In irisFetch/parse (line 1690)
In irisFetch/parse (line 1984)
In irisFetch/parse (line 2026)
In irisFetch/parseAnArray (line 2204)
In irisFetch/parse (line 2087)
In irisFetch/parseAnArray (line 2204)
In irisFetch/parse (line 2100)
In irisFetch/parse (line 1632)
In irisFetch/parser_for_IRIS_WS_2_0_0 (line 1589)
In irisFetch.Networks (line 714)
In irisFetch.Channels (line 598) 
Output argument "s" (and possibly others) not assigned a value in the execution with
"irisFetch/parse" function.

edu.iris.dmc.fdsn.station.model.Coefficient -- what are the methods that are used by this class and can they be added to the case-switch statement (starting line ~1625) ?

remove data quality

With M-quality data being the DMC standard now, it is time to deprecate the functionality of requesting different data types (D, R, etc) with the Traces method.

issues with Java unmarshalling at channel level

Some calls to the irisFetch.Traces method as well as channel- and response-level calls using irisFetch.Stations method are causing the following errors:

>> mysta = irisFetch.Stations('channel','GM','NMP31','*','HH?')
Sep 07, 2022 3:58:40 PM com.sun.xml.internal.bind.v2.util.XmlFactory createTransformerFactory
SEVERE: null
javax.xml.transform.TransformerConfigurationException: Unsupported feature: http://javax.xml.XMLConstants/feature/secure-processing
	at com.icl.saxon.TransformerFactoryImpl.setFeature(TransformerFactoryImpl.java:373)
	at com.sun.xml.internal.bind.v2.util.XmlFactory.createTransformerFactory(XmlFactory.java:169)
	at com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl.createTransformerHandler(JAXBContextImpl.java:712)
	at com.sun.xml.internal.bind.v2.runtime.unmarshaller.DomLoader$State.<init>(DomLoader.java:60)
	at com.sun.xml.internal.bind.v2.runtime.unmarshaller.DomLoader.startElement(DomLoader.java:103)
	at com.sun.xml.internal.bind.v2.runtime.unmarshaller.ProxyLoader.startElement(ProxyLoader.java:45)
	at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallingContext._startElement(UnmarshallingContext.java:559)
	at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallingContext.startElement(UnmarshallingContext.java:538)
	at com.sun.xml.internal.bind.v2.runtime.unmarshaller.InterningXmlVisitor.startElement(InterningXmlVisitor.java:60)
	at com.sun.xml.internal.bind.v2.runtime.unmarshaller.StAXEventConnector.handleStartElement(StAXEventConnector.java:246)
	at com.sun.xml.internal.bind.v2.runtime.unmarshaller.StAXEventConnector.bridge(StAXEventConnector.java:115)
	at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:445)
	at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:415)
	at edu.iris.dmc.service.station.parser.StationXMLParser.parse(StationXMLParser.java:252)
	at edu.iris.dmc.service.StationService.fetch(StationService.java:407)
	at edu.iris.dmc.service.StationService.fetch(StationService.java:316)
	at edu.iris.dmc.service.StationService.fetch(StationService.java:338)

Error using irisFetch/Networks/fetchTheStations (line 809)
Java exception occurred:
java.lang.IllegalStateException: javax.xml.transform.TransformerConfigurationException: Unsupported
feature: http://javax.xml.XMLConstants/feature/secure-processing
	at com.sun.xml.internal.bind.v2.util.XmlFactory.createTransformerFactory(XmlFactory.java:173)
	at
    com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl.createTransformerHandler(JAXBContextImpl.java:712)
	at com.sun.xml.internal.bind.v2.runtime.unmarshaller.DomLoader$State.<init>(DomLoader.java:60)
	at com.sun.xml.internal.bind.v2.runtime.unmarshaller.DomLoader.startElement(DomLoader.java:103)
	at com.sun.xml.internal.bind.v2.runtime.unmarshaller.ProxyLoader.startElement(ProxyLoader.java:45)
	at
    com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallingContext._startElement(UnmarshallingContext.java:559)
    	at
    com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallingContext.startElement(UnmarshallingContext.java:538)
    	at
    com.sun.xml.internal.bind.v2.runtime.unmarshaller.InterningXmlVisitor.startElement(InterningXmlVisitor.java:60)
    	at
    com.sun.xml.internal.bind.v2.runtime.unmarshaller.StAXEventConnector.handleStartElement(StAXEventConnector.java:246)
    	at
    com.sun.xml.internal.bind.v2.runtime.unmarshaller.StAXEventConnector.bridge(StAXEventConnector.java:115)
    	at
    com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:445)
    	at
    com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:415)
	at edu.iris.dmc.service.station.parser.StationXMLParser.parse(StationXMLParser.java:252)
	at edu.iris.dmc.service.StationService.fetch(StationService.java:407)
	at edu.iris.dmc.service.StationService.fetch(StationService.java:316)
	at edu.iris.dmc.service.StationService.fetch(StationService.java:338)
Caused by: javax.xml.transform.TransformerConfigurationException: Unsupported feature:
http://javax.xml.XMLConstants/feature/secure-processing
	at com.icl.saxon.TransformerFactoryImpl.setFeature(TransformerFactoryImpl.java:373)
	at com.sun.xml.internal.bind.v2.util.XmlFactory.createTransformerFactory(XmlFactory.java:169)
	... 15 more

Error in irisFetch.Networks (line 713)
         fetchTheStations();

Error in irisFetch.Stations (line 639)
         [stationStructure, urlParams] = irisFetch.Networks(detailLevel,varargin{:});

This error does not appear to be a global issue that affects all requests, as these requests appear to work fine:

>> mysta = irisFetch.Stations('station','OO','AXAS1','*','EHN','startTime','2022-01-01 00:00:00','2022-01-02 00:00:00')
>> mysta = irisFetch.Stations('channel','OO','AXAS1','*','EHN','startTime','2022-01-01 00:00:00','2022-01-02 00:00:00')
>> tr = irisFetch.Traces('OO','AXAS1','*','EHN','2020-01-01 00:00:00','2020-01-02 00:00:00','verbose')

Matlab version: 2021b
IRIS-WS version: 2.0.19