Comments (18)
commented on July 19, 2024
I will try to reproduce the issue locally.
Can you provide os name, os version and synda installation method (source or system package) ?
from synda.
cmharr
commented on July 19, 2024
OS=TOSS 2.4-6, which is based heavily on RHEL 6.7. I'm using the CentOS 6.7 RPM install of Synda.
from synda.
commented on July 19, 2024
Can you run the command below on the remote machine where synda is installed
sudo sed -i -e '\|/usr/share/python/synda/sdt/bin/sddaemon.py start|a \ sleep 1' /etc/init.d/sdt
then retry starting the daemon using remote shell
The problem seems to be caused by the daemonization process which fails if the remote shell is closed too quickly.
from synda.
cmharr
commented on July 19, 2024
The command returns with that extra sleep statement; however the daemon doesn't start.
from synda.
commented on July 19, 2024
Does command below gives better results ?
ssh <remote_host> "sudo nohup service synda start"
Any error message in logfiles ?
/var/log/synda/sdt/*
Thanks
from synda.
cmharr
commented on July 19, 2024
My nodes aren't set up to be able to SSH to each other so I use rsh. Also SSH's -c option is to select a cipher, so I'm not sure if you're meaning to just run the sudo command or something else (assuming the former).
[root@aimsdtn5:~]# /etc/init.d/sdt status
Daemon not running
[root@aimsdtn5:~]# rsh e5 nohup service synda start
Starting synda daemon (sdt):
[root@aimsdtn5:~]# /etc/init.d/sdt status
Daemon not running
I see the following in transfer.log:
2016-02-22 15:02:50,035 INFO SDDAEMON-001 Daemon starting ...
2016-02-22 15:02:50,035 INFO SDTSCHED-533 Connected to /var/lib/synda/sdt/sdt.db
2016-02-22 15:02:50,035 INFO SDTSCHED-993 Starting watchdog..
2016-02-22 15:02:50,356 ERROR SYDLOGON-040 Exception occured while retrieving certificate (status=4)
2016-02-22 15:02:50,357 ERROR SDTSCHED-920 Error occured while retrieving ESGF certificate
2016-02-22 15:02:50,357 INFO SDDAEMON-010 Exception occured (SYDLOGON-001)
2016-02-22 15:02:50,357 INFO SDDAEMON-034 Daemon stopped
... and this in stacktrace.log:
'sdlogon.sh' script returned an error
status=4
stdout=2016-02-22 15:02:50 - ERR009 - Error occured while retrieving certificate
stderr=
from synda.
commented on July 19, 2024
Also SSH's -c option is to select a cipher,
Indeed, in fact I use "vagrant ssh" and I forgot to remove the "-c" option which doesn't have the same meaning when used with ssh directly.
Reading informations you provide, I think the daemon doesn't start because of a certificate problem.
Is openid configured in credentials.conf file on the remote host ?
Do the command below succeed on the remote host ?
synda certificate renew
Thanks
from synda.
cmharr
commented on July 19, 2024
I saw that certificate error in the log after your last suggestion but wasn't sure what to do with it. Now running the cert renew command, I see it fails because I believe our ESGF data node is having problems. I notified the admin for that node earlier today and he is doing an update tomorrow that he hopes will fix some of the problems. Here's the renew output:
[root@aimsdtn5:~]# synda certificate renew
'sdlogon.sh' script returned an error
status=4
stdout=echo XXXX | /usr/share/python/synda/sdt/bin/myproxyclient logon -T -b -S -s pcmdi.llnl.gov -p 7512 -l llnl_replication -o /var/tmp/synda/sdt/.esg/credentials.pem
2016-02-22 15:31:10 - ERR009 - Error occured while retrieving certificate
stderr=/usr/share/python/synda/sdt/lib/python2.6/site-packages/cryptography/__init__.py:26: DeprecationWarning: Python 2.6 is no longer supported by the Python core team, please upgrade your Python. A future version of cryptography will drop support for Python 2.6
DeprecationWarning
Traceback (most recent call last):
File "/usr/share/python/synda/sdt/bin/myproxyclient", line 9, in <module>
load_entry_point('MyProxyClient==1.4.3', 'console_scripts', 'myproxyclient')()
File "/usr/share/python/synda/sdt/lib/python2.6/site-packages/myproxy/script.py", line 196, in main
do_logon(myproxy, options)
File "/usr/share/python/synda/sdt/lib/python2.6/site-packages/myproxy/script.py", line 220, in do_logon
updateTrustRoots=options.trustroots)
File "/usr/share/python/synda/sdt/lib/python2.6/site-packages/myproxy/client.py", line 1462, in logon
conn.write('0')
File "/usr/share/python/synda/sdt/lib/python2.6/site-packages/OpenSSL/SSL.py", line 1271, in send
self._raise_ssl_error(self._ssl, result)
File "/usr/share/python/synda/sdt/lib/python2.6/site-packages/OpenSSL/SSL.py", line 1187, in _raise_ssl_error
_raise_current_error()
File "/usr/share/python/synda/sdt/lib/python2.6/site-packages/OpenSSL/_util.py", line 48, in exception_from_error_queue
raise exception_type(errors)
OpenSSL.SSL.Error: [('SSL routines', 'SSL3_READ_BYTES', 'tlsv1 alert unknown ca'), ('SSL routines', 'SSL3_WRITE_BYTES', 'ssl handshake failure')]
Error occurs while renewing certificate (code=SYDLOGON-001,message=Cannot retrieve certificate from ESGF (hostname=pcmdi.llnl.gov,port=7512))
from synda.
commented on July 19, 2024
Ok, so something seems wrong indeed with the certificate.
[root@aimsdtn5:
]# /etc/init.d/sdt status]# rsh e5 nohup service synda start
Daemon not running
[root@aimsdtn5:
Starting synda daemon (sdt):
[root@aimsdtn5:~]# /etc/init.d/sdt status
Daemon not running
I see here that you are starting the remote daemon,
but checking the status of the local daemon.
Is this wanted ?
Thanks
from synda.
cmharr
commented on July 19, 2024
Actually, it's the same node. I'm rsh-ing to itself to "remotely" start the daemon. (aimsdtn5 == e5)
from synda.
commented on July 19, 2024
Okay, got it :)
So yes the remote start problem is very likely caused by the certificate issue.
Let's see how it goes once the certificate is working
from synda.
commented on July 19, 2024
As cannot be reproduce for now, closed until new feedback from user.
from synda.
cmharr
commented on July 19, 2024
Jerome,
Sorry for the delay, but I'm not ready to close this. (I've either been unavailable or waiting for certificate updates.)
Currently, I'm failing to grab the certificate from my local index node and I have a couple questions. The data below warns about Python and SSL. Using the python packaged with Synda gives similar results. Should Python 2.6.6 still work? Any ideas on why the SSL handshake would fail?
Thanks
# synda certificate renew
'sdlogon.sh' script returned an error
status=4
stdout=echo XXXX | /usr/share/python/synda/sdt/bin/myproxyclient logon -T -b -S -s pcmdi.llnl.gov -p 7512 -l llnl_replication -o /var/tmp/synda/sdt/.esg/credentials.pem
2016-03-09 14:40:39 - ERR009 - Error occured while retrieving certificate
stderr=/usr/share/python/synda/sdt/lib/python2.6/site-packages/cryptography/__init__.py:26: DeprecationWarning: Python 2.6 is no longer supported by the Python core team, please upgrade your Python. A future version of cryptography will drop support for Python 2.6
DeprecationWarning
Traceback (most recent call last):
File "/usr/share/python/synda/sdt/bin/myproxyclient", line 9, in <module>
load_entry_point('MyProxyClient==1.4.3', 'console_scripts', 'myproxyclient')()
File "/usr/share/python/synda/sdt/lib/python2.6/site-packages/myproxy/script.py", line 196, in main
do_logon(myproxy, options)
File "/usr/share/python/synda/sdt/lib/python2.6/site-packages/myproxy/script.py", line 220, in do_logon
updateTrustRoots=options.trustroots)
File "/usr/share/python/synda/sdt/lib/python2.6/site-packages/myproxy/client.py", line 1462, in logon
conn.write('0')
File "/usr/share/python/synda/sdt/lib/python2.6/site-packages/OpenSSL/SSL.py", line 1271, in send
self._raise_ssl_error(self._ssl, result)
File "/usr/share/python/synda/sdt/lib/python2.6/site-packages/OpenSSL/SSL.py", line 1187, in _raise_ssl_error
_raise_current_error()
File "/usr/share/python/synda/sdt/lib/python2.6/site-packages/OpenSSL/_util.py", line 48, in exception_from_error_queue
raise exception_type(errors)
OpenSSL.SSL.Error: [('SSL routines', 'SSL3_READ_BYTES', 'tlsv1 alert unknown ca'), ('SSL routines', 'SSL3_WRITE_BYTES', 'ssl handshake failure')]
Error occurs while renewing certificate (code=SYDLOGON-001,message=Cannot retrieve certificate from ESGF (hostname=pcmdi.llnl.gov,port=7512))
root@aimsdtn5:sdt]# openssl version
OpenSSL 1.0.1e-fips 11 Feb 2013
from synda.
commented on July 19, 2024
Reopened.
I am a bit stuck as I can't reproduce the ssl error.
I installed a new Centos 6.7 - 64bit from scratch this morning:
- Synda 3.2 RPM for Centos 6.7
- Python 2.6.6
- OpenSSL 1.0.1e-fips 11 Feb 2013
So it seems the same config as you.
Then I ran
synda certificate renew
which complete successfully.
Maybe you can retry after removing the certificate folder, by running commands below:
sudo rm -rf /var/tmp/synda/sdt/.esg
sudo synda certificate renew
If it still fails, maybe ask about the certificate error on the [email protected] mailing list,
as someone may already faced this issue.
from synda.
cmharr
commented on July 19, 2024
Still no luck. I'll try posting to the list.
from synda.
cmharr
commented on July 19, 2024
I wiped out the node and re-installed the OS and synda and synda works again - except for the issue represented by this ticket. Whether running with the init script patch or with nohup (actually, with both), the command still hangs every time. The only output I get is in the transfer.log file:
2016-03-16 13:48:38,391 INFO SDDAEMON-001 Daemon starting ...
2016-03-16 13:48:38,391 INFO SDTSCHED-533 Connected to /var/lib/synda/sdt/sdt.db
2016-03-16 13:48:38,391 INFO SDTSCHED-993 Starting watchdog..
2016-03-16 13:48:39,676 INFO SDTSCHED-902 Transfer daemon is now up and running
from synda.
cmharr
commented on July 19, 2024
I ended up just putting an '&' at the end of the start command and it returns fine, so I'll just leave it at that. Thanks for looking into it.
from synda.
commented on July 19, 2024
Ok, I close the issue.
Do not hesitate to reopen.
Thanks for your feedback.
from synda.
Related Issues (20)
- synda remove fails to import sdsimplefilter HOT 2
- OAuth 2.0 authentication support needed for ESGF Future Architecture HOT 1
- Where can I find up to date documentation? HOT 4
- STAC ESGF Search Support for Synda
- transfer log is opened on multiple file descriptors HOT 1
- Synda reports lots of failed checksums HOT 4
- Unable to change setting of data_path in Synda HOT 5
- synda daemon succefully started but download does not work HOT 3
- synda daemon and synda reset not responsive HOT 11
- Download not working when `wget` is installed outside standard locations
- "%" in password is interpreted as formatting operator
- `synda queue` disables cursor on MacOS
- conda openssl requirement HOT 1
- add support for username in credentials.conf
- Synda time slice in selection file breaks whenever a file has a bad name
- Synda reset marks failed file downloads as `already installed`
- downloads stops when almost complete HOT 1
- synda on conda-forge
- Daemon stopping immediately HOT 1
- Datasets still have the status of 'empty' despite all of their files having the status 'done'
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from synda.