>>> from HinetPy imprt Client
>>> from datetime import datetime
>>> client = Client("username", "password")
>>> startdate = datetime(2010, 1, 1)
>>> client.get_focalmechanism(startdate, 5)
'focal_20100101_5.txt'

"Argument error" in the downloaded file.

https://joss.theoj.org/

Description of the solution you'd like

May I ask how you know the instrumental response has been removed in sac files and the converted sac files are velocity/acceleration? Do you mind giving any references in the documentation (https://seisman.github.io/HinetPy/tutorial/conversion.html)?

I failed to find descriptions about the output data format on the win32tools documentation (https://hinetwww11.bosai.go.jp/auth/manual/?LANG=en)

Hi Dongdong,
Great work for HinetPy.
Some error occurred when I request data continuously over a period of time, the source codes are attached.
Look forwarding to you help.

Traceback (most recent call last):
  File "/usr/local/pyenv/versions/anaconda3-4.3.1/lib/python3.6/site-packages/requests/packages/urllib3/connection.py", line 138, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/usr/local/pyenv/versions/anaconda3-4.3.1/lib/python3.6/site-packages/requests/packages/urllib3/util/connection.py", line 98, in create_connection
    raise err
  File "/usr/local/pyenv/versions/anaconda3-4.3.1/lib/python3.6/site-packages/requests/packages/urllib3/util/connection.py", line 88, in create_connection
    sock.connect(sa)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/pyenv/versions/anaconda3-4.3.1/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py", line 594, in urlopen
    chunked=chunked)
  File "/usr/local/pyenv/versions/anaconda3-4.3.1/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py", line 350, in _make_request
    self._validate_conn(conn)
  File "/usr/local/pyenv/versions/anaconda3-4.3.1/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py", line 835, in _validate_conn
    conn.connect()
  File "/usr/local/pyenv/versions/anaconda3-4.3.1/lib/python3.6/site-packages/requests/packages/urllib3/connection.py", line 281, in connect
    conn = self._new_conn()
  File "/usr/local/pyenv/versions/anaconda3-4.3.1/lib/python3.6/site-packages/requests/packages/urllib3/connection.py", line 143, in _new_conn
    (self.host, self.timeout))
requests.packages.urllib3.exceptions.ConnectTimeoutError: (<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f0318697be0>, 'Connection to hinetwww11.bosai.go.jp timed out. (connect timeout=60)')

#!/bin/env python
import datetime
import os
import glob
from HinetPy import Client
from HinetPy import win32

timeb = datetime.datetime(2016,11,8,23,30,0)
timee = datetime.datetime(2017,1,1,0,0,0)
span = 30
nowtime = timeb
clnt = Client('xxxxxx', 'xxxxxxx')

while nowtime < timee:
    dirname = nowtime.strftime("%Y%m%d%H%M")
    print(dirname)
    if os.path.isdir(dirname) and glob.glob(os.path.join(dirname, "*.SAC")):
        pass
    else:
        data, ctable = clnt.get_waveform('0101', dirname, span, outdir=dirname)
        win32.extract_sac(data, ctable, outdir=dirname)
        win32.extract_pz(ctable, outdir=dirname)
    nowtime += datetime.timedelta(minutes=span)

I have downloaded continuous waveform data of two Hi-net stations using HinetPy. But the problem is when I open the header of any sac file in SAC, there is no network name in it and this in turn is creating problem in station pair making in MSNoise. Furthermore, as I am a newbie in this field, I am also struggling with the response files of the stations. I have PZ response files but I want the response files to be in Station XML. I am unable to understand how to convert these PZ files to Station XML. I am hoping to get some help from here.

It seems that win2sac has already removed the sensitivity part of instrumental response. If that's true, the CONSTANT in PZ should be 1 or close to 1.

Using HinetPy==0.8.0
Python 3.10.6
Linux

the following code works but crashes whenever including
select_stations or get_station_list, error SSL: DH_KEY_TOO_SMALL

CODE EXAMPLE:

This part works ok,

however returns only 6 traces (2 stations N.IWEH N.IWSH)

from HinetPy import Client, win32
client = Client("username", "password")
client.doctor()
client.info("0101")
client.get_event_waveform(
"201001010000",
"201001020000",
minmagnitude=4.0,
maxmagnitude=7.0,
mindepth=0,
maxdepth=70)
data, ctable = client.get_continuous_waveform("0101", "2020301010000", 2)
win32.extract_sac(data, ctable, suffix="", outdir="SAC")

The following commands crash with error

SLError(1, '[SSL: DH_KEY_TOO_SMALL] dh key too small (_ssl.c:997)'):

stations = client.get_station_list("0101") # crash
client.select_stations("0101", ["N.AAKH", "N.ABNH"]) #crash
client.select_stations("0101", latitude=36, longitude=139, minradius=0, maxradius=3) # crash

possibly related issue

skip SSL verification by using verify=False.

Originally posted by @seisman in #1 (comment)

HOW / WHERE DO I SET verify = FALSE ? i tried this but no luck:

client = Client("stefanazzz", "S2ByXcxUurfz", verify=False)
--> TypeError: BaseClient.init() got an unexpected keyword argument 'verify'

stations = client.get_station_list("0101", verify = False)
--> TypeError: StationClient.get_station_list() got an unexpected keyword argument 'verify'

Many thanks,
Stefanazzz

Thanks Dr. Tian for preparing and sharing the HinetPy package for the seismology community, it is a useful tool designed to assist downloading the high quality HiNet data which facilitates tremendous scientific findings. When reviewing your submission to JOSS, I found following minor issues that I hope Dr. Tian could resolve:

1. Could you explain briefly in the paper which python packages or techniques are implemented in HinetPy that makes data quires possible.
2. Even though you made it clear in the installation part in the documentation that the users need to install win32tool from NIED, could you also briefly state that the win32 functionality of the HinetPy package is based on the win32tool installed from NIED in the paper?
3. Based on the 'community guideline' of the review checklist, the information about 2) Report issues or problems with the software and 3) Seek support are missing in the repository. Even though the issues are well maintained and answered timely by the authors.

That's all concerns I have for the author. Thanks again for the brilliant and open work.

Yifan Yu

Currently, client.py has too many functions and features, it can request and download continuous waveforms, download event catalog and mechanism catalog. After finishing #14, the code would be too big to maintain. It would be better to split client.py into several tiny clients. The modified codes should be backward compatible.

• get_station_list should return a list of stations, rather than printing station information
• get_selected_stations should return a list of stations
• select_stations should support selection based on lat/lon region.

Hinet website have a big update at Dec 1st, 2014. This update use SSL which makes the scripts fail.

Try to fix it.

Steps to reproduce the bug:

1. get one data
2. call win32.extract_sac() in a large loop.

Full code that generated the error

from HinetPy import Client, win32

client = Client("username", "password")
data, ctable = client.get_continuous_waveform('0120A', "2016-12-25T23:38:38.000000", 5)

Login succeeds with requests 2.14 and 2.16, fails with 2.17 and 2.18.

$ python test.py              
Traceback (most recent call last):
  File "test.py", line 5, in <module>
    client = Client("seisman", "seismanustc")
  File "/home/seisman/Script/HinetPy/HinetPy/client.py", line 87, in __init__
    self.login(user, password)
  File "/home/seisman/Script/HinetPy/HinetPy/client.py", line 124, in login
    raise requests.ConnectionError(msg)
requests.exceptions.ConnectionError: Unauthorized. Check your username and password!

Full error message

[2023-07-21 10:20:42] WARNING: Error in parsing channel information for NU.NIN.U (9181). Skipped.
[2023-07-21 10:20:42] WARNING: Original error message: could not convert string to float: '0.70-10'
[2023-07-21 10:20:42] WARNING: Error in parsing channel information for NU.NIN.N (9182). Skipped.
[2023-07-21 10:20:42] WARNING: Original error message: could not convert string to float: '0.70-10'
[2023-07-21 10:20:42] WARNING: Error in parsing channel information for NU.NIN.E (9183). Skipped.
[2023-07-21 10:20:42] WARNING: Original error message: could not convert string to float: '0.70-10'

I guess this message stems from the following code

The erroneous station is

# nu.nin Niino
9181  1  0  NU.NIN  U     7   20  1500.00  m/s    120.05  0.70-10  7.630e-06  34.6833 138.1312    55  0  0  Niino
9182  1  0  NU.NIN  N     7   20  1500.00  m/s    120.05  0.70-10  7.630e-06  34.6833 138.1312    55  0  0  Niino
9183  1  0  NU.NIN  E     7   20  1500.00  m/s    120.05  0.70-10  7.630e-06  34.6833 138.1312    55  0  0  Niino

which is from this channel table D20110514001274_20.txt

System information

• OS: Ubuntu 22.04
• Python version: Python 3.9.16
• HinetPy version: v0.7.2.dev15+g992f9b8

To reproduce the issue:

from HinetPy import win32
ctable="01_01_20130318.euc.ch"
win32.extract_pz(ctable,suffix="SAC_PZ", outdir="PZ/", filter_by_name="N.TKWH")

It seems Hinet may make some mistakes and give 0.70-10 in the channel table, resulting in following exceptions.

win32.extract_sac(data, ctable, suffix="sac", outdir=os.path.join(newfile,"SAC"))
  File "/usr/local/lib/python3.5/dist-packages/HinetPy/win32.py ", line 123, in extract_sac
    channels = _get_channels(ctable)
  File "/usr/local/lib/python3.5/dist-packages/HinetPy/win32.py ", line 248, in _get_channels
    damping=float(items[10]),
ValueError: could not convert string to float: '0.70-10'

Description of the problem

A ConnectionError interrupt the downloading. The data downloading code has run for two days before this error.

Full code that generated the error

import HinetPy
...
def download(...):
    client=HinetPy.Client(...)
    while True:
        try:
             client.get_event_waveform(...)
        except Exception as e:
            logger.error(f"{type(e).__name__}| Waiting for 10 seconds before next try")
            time.sleep(10)
            client = HinetPy.Client(user=username, password=password, retries=5)
        else:
            break

Full error message

  File "/home/zhongyiyuan/miniconda3/envs/seisbench_v0.4/lib/python3.9/site-packages/HinetPy-0.7.2.dev15+g992f9b8-py3.9.egg/HinetPy/client.py", line 140, in __init__
  File "/home/zhongyiyuan/miniconda3/envs/seisbench_v0.4/lib/python3.9/site-packages/HinetPy-0.7.2.dev15+g992f9b8-py3.9.egg/HinetPy/client.py", line 188, in login
  File "/home/zhongyiyuan/miniconda3/envs/seisbench_v0.4/lib/python3.9/site-packages/requests/sessions.py", line 600, in get
    return self.request("GET", url, **kwargs)
  File "/home/zhongyiyuan/miniconda3/envs/seisbench_v0.4/lib/python3.9/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/zhongyiyuan/miniconda3/envs/seisbench_v0.4/lib/python3.9/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/home/zhongyiyuan/miniconda3/envs/seisbench_v0.4/lib/python3.9/site-packages/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='hinetwww11.bosai.go.jp', port=443): Max retries exceeded with url: /auth/ (Caused

The error message suggests that this error is caused by

and

Question
Maybe I should use a try...except... around the Client initialization part like the following code?

while true:
    try:
        client = HinetPy.Client(user=username, password=password, retries=5)
    except:
        logger.error(f"{type(e).__name__}| Waiting for 10 seconds before next try")
        time.sleep(10)
        continue
    else:
        break

Is it equivalent to setting retries to a very large number?

System information

• OS: Ubuntu 22.04
• Python version: Python 3.9.16
• HinetPy version: v0.7.2.dev15+g992f9b8

I am a phd student, working with data from HiNet and FNet. I am currently retrieving the data, converting them with win2sac... And it seems to me that win32tools automatically fills gaps with zeros. I don't know if it is the right place to ask this kind of question but I am therefore wondering (first if I am right) how do you manage gaps in data ?
Thanks in advance for your answer,
Lucile Costes

Hi,

The circular region selection of stations appears to be broken. The example here:

>>> client.select_stations('0101', latitude=30, longitude139,
...                        minradius=0, maxradius=2)

produces the following error stack:

Traceback (most recent call last):
  File "download.py", line 285, in <module>
    hinet.download_event_data(events, args.maxradius)
  File "download.py", line 247, in download_event_data
    fn = self.event_to_stations(event, maxradius)
  File "download.py", line 225, in event_to_stations
    minradius=0, maxradius=2)
  File "/home/marius/.local/lib/python3.5/site-packages/HinetPy/client.py", line 1049, in select_stations
    stations_at_server = self.get_station_list(code)
  File "/home/marius/.local/lib/python3.5/site-packages/HinetPy/client.py", line 903, in get_station_list
    latitude, longitude, elevation = items[7], items[8], items[12]
IndexError: list index out of range

Also, there is a typo in the example (longitude139)

To all HinetPy users,

I plan to write a short paper about the HinetPy package and submit it to Journal of Open Source Software (JOSS). The submission requires at least two reviewers to help review and improve the quality of the package.

The reviewers should be HinetPy users and be familiar with Python.

Please let me know if you're interested in reviewing HinetPy.

Best,
Dongdong

Hi!

I tried to download continuous waveform data, but I got "ERROR: Data request fails after 3 retries.", and I got no data (.cnt) and channel table (.ch) in the working directory. Below is what I got:

>>> from HinetPy import Client, win32
>>> client = Client('MYUSENAME', 'MYPASSWORD')
>>> data, ctable = client.get_continuous_waveform('0101', '201604160100', 5)
[2019-07-31 11:12:29] INFO: 2016-04-16 01:00 ~5
[2019-07-31 11:12:29] INFO: [1/1] => 2016-04-16 01:00 ~5
[2019-07-31 11:13:03] ERROR: Data request fails after 3 retries.
[2019-07-31 11:13:03] ERROR: Fail to request some data. Skipped.

It seems that the request has successfully been sent to Hi-net, and I could manually (click and save) download the data via Status / Download page at Hi-net website https://hinetwww11.bosai.go.jp/auth/download/cont/cont_status.php?LANG=en, but it should be not convenient for me when downloading hours and days of long data...

It would be a great help if I could get any hints to solve this problem... Thanks!

• OS: macOS
• Python version: Python 3.6.1
• HinetPy version: 0.6.3

When using rdhinet.py to extract SAC files from cnt files, if the directory name contain underscore, it may be in a infinite loop and the computer seems to be dead.

This issue is reported by Jiayuan Yao.

The script seems to have a huge bug, I will check it ASAP.

FnetPy(https://github.com/seisman/FnetPy) is designed for F-net only. Users should use FnetPy when requesting F-net data, since HinetPy cannot handle instrumental responses of F-net correctly.

zenodo: https://zenodo.org/

Description of the desired feature
Downloading waveforms for a single event, given the event information

Description of the solution you'd like
get_event_waveform is used to download waveforms for a list of events. Is there any way to download waveforms for only one single event? Now I have downloaded a catalog of events, and I am trying to download waveforms corresponding to each event.

It seems that _request_event_waveform and _download_event_waveform can do this, but I don't know how to initialize an event. In particular, I am very confused about the event id, because I cannot find any event ID information from the event catalog (https://www.data.jma.go.jp/svd/eqev/data/bulletin/data/format/hypfmt_e.html)

Thank you for sharing such a convenient package.

It seems now we can download 60 minutes data in a single request for all networks.