relleums / qstat Goto Github PK
View Code? Open in Web Editor NEWparse Grid Engine qstat job info list into a list of python dicts
License: GNU General Public License v3.0
parse Grid Engine qstat job info list into a list of python dicts
License: GNU General Public License v3.0
Hi there! Nice piece of software I could really use it.
However it's not working on my server, I get this error:
qstat returncode: 2
qstat std output: b"qstat: invalid option -- 'm'\nusage: \n qstat [-f [-1]] [-W site_specific] [-x] [ job_identifier... | destination... ]\n qstat [-a|-i|-r|-e] [-u user] [-n [-1]] [-s] [-G|-M] [-R] [job_id... | destination...]\n qstat -Q [-f [-1]] [-W site_specific] [ destination... ]\n qstat -q [-G|-M] [ destination... ]\n qstat -B [-f [-1]] [-W site_specific] [ server_name... ]\n"
Traceback (most recent call last):
File "/home/eap/sRNAtoolbox/qtest.py", line 3, in
queue_info, job_info = qstat()
File "/opt/venv/sRNAtoolbox2017/lib/python3.5/site-packages/qstat/_tools.py", line 20, in qstat
xml = qstat2xml(qstat_path=qstat_path)
File "/opt/venv/sRNAtoolbox2017/lib/python3.5/site-packages/qstat/_tools.py", line 37, in qstat2xml
qstatxml = sp.check_output([qstat_path, xml_option], stderr=sp.STDOUT)
File "/usr/lib/python3.5/subprocess.py", line 626, in check_output
**kwargs).stdout
File "/usr/lib/python3.5/subprocess.py", line 708, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['qstat', '-xml']' returned non-zero exit status 2
Maybe this has to do with qstat version. Which one were you using when you developped this?
Thanks
When there is only one <job_list>
in either one of the job-lists, the xml
to dict
fails.
Hi,
Really like the package! Good work!
Only one tip...I'm calling the qsub using popen but if I'm calling qstat() right after, the package throws an exception as Grid Engine is still doing his job. I needed to add a sleep(2) in between.
I guess I need to check the popen until finished.
gold_auto_call = 'qsub -t 1-4 gold_auto.sh '
process = Popen(shlex.split(gold_auto_call), stdout=PIPE)
(output, err) = process.communicate()
exit_code = process.wait()
time.sleep(2)
queue_info, job_info = qstat()
all_jobs = queue_info + job_info
Just tested it with Open Cluster Scheduler and it works. Used a container image (hpcgridware/clusterscheduler-latest-ubuntu2204:eac1ec597) containing Open Cluster Scheduler (detailed description here: https://github.com/dgruber/drmaa/blob/master/dockerfiles/openclusterscheduler/README.md)
Maybe you can mention Open Cluster Scheduler support in your project?
Cheers
Daniel
python3 test.py
sleep is {'@state': 'running', 'JB_job_number': '4', 'JAT_prio': '0.55500', 'JB_name': 'sleep', 'JB_owner': 'root', 'state': 'r', 'JAT_start_time': '2024-06-28T11:59:43.834192', 'queue_name': 'all.q@master', 'slots': '1', 'tasks': '34'}
sleep is {'@state': 'running', 'JB_job_number': '4', 'JAT_prio': '0.55500', 'JB_name': 'sleep', 'JB_owner': 'root', 'state': 'r', 'JAT_start_time': '2024-06-28T11:59:43.904217', 'queue_name': 'all.q@master', 'slots': '1', 'tasks': '35'}
sleep is {'@state': 'running', 'JB_job_number': '4', 'JAT_prio': '0.55500', 'JB_name': 'sleep', 'JB_owner': 'root', 'state': 'r', 'JAT_start_time': '2024-06-28T11:59:43.915557', 'queue_name': 'all.q@master', 'slots': '1', 'tasks': '36'}
sleep is {'@state': 'running', 'JB_job_number': '4', 'JAT_prio': '0.55500', 'JB_name': 'sleep', 'JB_owner': 'root', 'state': 'r', 'JAT_start_time': '2024-06-28T11:59:43.926597', 'queue_name': 'all.q@master', 'slots': '1', 'tasks': '37'}
sleep is {'@state': 'running', 'JB_job_number': '4', 'JAT_prio': '0.55500', 'JB_name': 'sleep', 'JB_owner': 'root', 'state': 'r', 'JAT_start_time': '2024-06-28T11:59:43.941707', 'queue_name': 'all.q@master', 'slots': '1', 'tasks': '38'}
sleep is {'@state': 'running', 'JB_job_number': '4', 'JAT_prio': '0.55500', 'JB_name': 'sleep', 'JB_owner': 'root', 'state': 'r', 'JAT_start_time': '2024-06-28T11:59:43.952828', 'queue_name': 'all.q@master', 'slots': '1', 'tasks': '39'}
sleep is {'@state': 'running', 'JB_job_number': '4', 'JAT_prio': '0.55500', 'JB_name': 'sleep', 'JB_owner': 'root', 'state': 'r', 'JAT_start_time': '2024-06-28T11:59:43.964330', 'queue_name': 'all.q@master', 'slots': '1', 'tasks': '40'}
sleep is {'@state': 'running', 'JB_job_number': '4', 'JAT_prio': '0.55500', 'JB_name': 'sleep', 'JB_owner': 'root', 'state': 'r', 'JAT_start_time': '2024-06-28T11:59:43.979662', 'queue_name': 'all.q@master', 'slots': '1', 'tasks': '41'}
sleep is {'@state': 'running', 'JB_job_number': '4', 'JAT_prio': '0.55500', 'JB_name': 'sleep', 'JB_owner': 'root', 'state': 'r', 'JAT_start_time': '2024-06-28T11:59:43.990396', 'queue_name': 'all.q@master', 'slots': '1', 'tasks': '42'}
sleep is {'@state': 'running', 'JB_job_number': '4', 'JAT_prio': '0.55500', 'JB_name': 'sleep', 'JB_owner': 'root', 'state': 'r', 'JAT_start_time': '2024-06-28T11:59:44.001713', 'queue_name': 'all.q@master', 'slots': '1', 'tasks': '43'}
sleep is {'@state': 'pending', 'JB_job_number': '4', 'JAT_prio': '0.55500', 'JB_name': 'sleep', 'JB_owner': 'root', 'state': 'qw', 'JB_submission_time': '2024-06-28T11:59:27.208411', 'queue_name': None, 'slots': '1', 'tasks': '44-100:1'}
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.