Comments (6)
You need to run the command with sudo
e.g.
sudo synda get pmip3.output.MPI-M.MPI-ESM-P.lgm.monClim.atmos.Aclim.r1i1p2.v20140428
Also because of servers downtime, you may need to change indexes in sdt.conf file
vi /etc/synda/sdt/sdt.conf
#indexes=esgf-data.dkrz.de
#default_index=esgf-data.dkrz.de
indexes=esgf-index1.ceda.ac.uk
default_index=esgf-index1.ceda.ac.uk
from synda.
You can run command below to display an explicit error message instead of Python Exception
sudo sed -i -e "s/install/get','install/" /usr/share/python/synda/sdt/bin/sdconst.py
I added the fix in the doc below
https://github.com/Prodiguer/synda/blob/master/sdt/doc/deb_install.md#patch
Regarding the fact that you need sudo to run 'synda get' command,
I opened the issue below:
It should be possible to use 'synda get' with normal user if we store ESGF credential for each user
(currently, only one ESGF credential is stored in /etc/synda/sdt)
from synda.
That's great, thanks. Using sudo
resolves the issue, but I would agree that superuser privileges should not be required to initiate a download.
I do have another issue now however...
When starting a test download, I am now seeing:
synda get cmip5.output1.MPI-M.MPI-ESM-MR.historical.day.atmos.day.r3i1p1.v20120503 tasmax
5 file(s) will be downloaded for a total size of 1.2 GB
[...]
Transfer failed with error 1 (did you subscribe to the required role/group ? (e.g. cmip5_research, cordex_research))
Download failed (http://aims3.llnl.gov/thredds/fileServer/cmip5_css02_data/cmip5/output1/MPI-M/MPI-ESM-MR/historical/day/atmos/day/r3i1p1/tasmax/1/tasmax_day_MPI-ESM-MR_historical_r3i1p1_19900101-19991231.nc)
When using a browser to download that dataset, I am redirected to log in, then when that completes the download starts. I have the role "CMIP5 Research"
Are there other conditions which could cause this error?
from synda.
I successfully downloaded the files using my openid, so I guess something must differ between my certificate and your certificate.
Can you print the output of the command below ?
(it to display roles informations associated with your certificate)
sudo openssl x509 -certopt no_subject,no_header,no_version,no_serial,no_signame,no_validity,no_issuer,no_pubkey,no_sigdump,no_aux -text -noout -in /var/tmp/synda/sdt/.esg/credentials.pem
If the certificate is missing, you can run this command to create it
$ sudo synda certificate renew
from synda.
It seems that it took a while for the role to propagate. I had just applied it a couple hours before attempting this but if I try now I do not receive an error.
To avoid using sudo
I actually ended up re-installing from source on a new Ubutnu 16.04 machine with python 2.7.11.
Upon completing that, a synda certificate renew
would result in this error:
'sdlogon.sh' script returned an error
status=4
stdout=/home/bveerman/sdt/bin/myproxyclient logon -T -s pcmdi.llnl.gov -p 7512 -l bveerman -o /home/bveerman/.esg/credentials.pem -b
2016-05-31 16:52:32 - ERR009 - Error occured while retrieving certificate
stderr=Traceback (most recent call last):
File "/home/bveerman/sdt/bin/myproxyclient", line 11, in <module>
sys.exit(main())
File "/home/bveerman/sdt/local/lib/python2.7/site-packages/myproxy/script.py", line 196, in main
do_logon(myproxy, options)
File "/home/bveerman/sdt/local/lib/python2.7/site-packages/myproxy/script.py", line 220, in do_logon
updateTrustRoots=options.trustroots)
File "/home/bveerman/sdt/local/lib/python2.7/site-packages/myproxy/client.py", line 1384, in logon
**getTrustRootsKw)
File "/home/bveerman/sdt/local/lib/python2.7/site-packages/myproxy/client.py", line 1533, in getTrustRoots
conn = self._initConnection(verifyPeerWithTrustRoots=(not bootstrap))
File "/home/bveerman/sdt/local/lib/python2.7/site-packages/myproxy/client.py", line 699, in _initConnection
context = SSL.Context(SSL.SSLv3_METHOD)
File "build/bdist.linux-x86_64/egg/OpenSSL/SSL.py", line 489, in __init__
File "build/bdist.linux-x86_64/egg/OpenSSL/_util.py", line 48, in exception_from_error_queue
OpenSSL.SSL.Error: [('SSL routines', 'SSL_CTX_new', 'null ssl method passed')]
Error occurs while renewing certificate (code=SYDLOGON-001,message=Cannot retrieve certificate from ESGF (hostname=pcmdi.llnl.gov,port=7512))
I then manually downloaded the credentials with a system (deb) install of MyProxyClient and synda was able to use that credentials.pem
and it worked.
I did notice that a source install of MyProxyClient would then fail to renew the credentials at version 1.3.1, but the upgrading the version to 1.4.3 solved the issues with MyProxyClient as well as synda certificate renew
.
Thanks for your help!
from synda.
It seems that it took a while for the role to propagate. I had just applied it a couple hours before attempting this but if I try now I do not receive an error.
Good to know
To avoid using sudo I actually ended up re-installing from source on a new Ubutnu 16.04 machine with python 2.7.11.
Well done, that's the way to go until I remove the sudo dependency in system packages
I did notice that a source install of MyProxyClient would then fail to renew the credentials at version 1.3.1, but the upgrading the version to 1.4.3 solved the issues with MyProxyClient as well as synda certificate renew.
Indeed, the myproxyclient version was frozen to the 1.3.1 for no reason.
I changed the installation script to use the latest instead.
Thanks for your feedback
from synda.
Related Issues (20)
- synda remove fails to import sdsimplefilter HOT 2
- OAuth 2.0 authentication support needed for ESGF Future Architecture HOT 1
- Where can I find up to date documentation? HOT 4
- STAC ESGF Search Support for Synda
- transfer log is opened on multiple file descriptors HOT 1
- Synda reports lots of failed checksums HOT 4
- Unable to change setting of data_path in Synda HOT 5
- synda daemon succefully started but download does not work HOT 3
- synda daemon and synda reset not responsive HOT 11
- Download not working when `wget` is installed outside standard locations
- "%" in password is interpreted as formatting operator
- `synda queue` disables cursor on MacOS
- conda openssl requirement HOT 1
- add support for username in credentials.conf
- Synda time slice in selection file breaks whenever a file has a bad name
- Synda reset marks failed file downloads as `already installed`
- downloads stops when almost complete HOT 1
- synda on conda-forge
- Daemon stopping immediately HOT 1
- Datasets still have the status of 'empty' despite all of their files having the status 'done'
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from synda.