QmemTools is a set of tools to monitor memory usage in an SGE cluster.
It will include more tools in the future but you can develop you own client
tools to fit your needs. Details about data structure returned in JSON by the
server are described below
Note: QmemTools has been tested on ubuntu, centos and redhat distributions.
QmemTools is compatible with POSIX system. Windows is not supported.
A daemon (qmemserver) generate and parse cached xml data from qhost and qstat
xml output command.
A client (ie: qmemview.py) request the server to receive JSON structured data
(refer to Data structure) and do something useful with it.
QmemTools requires following packages:
- python (>=2.4 and <3.0)
- web.py (>=0.34)
- simplejson (>=2.0.9)
- libxml2
sudo python setup.py install
After installation, a file is create in /etc/init.d/qmemserver, so if
you want to enable qmemserver at boot time you have to configure manually
according to your distribution.
The server configuration is locate in /etc/qmem/qmemserver.conf and contain two
sections, files and system which are described.
qmemserver.py handle requests and return JSON data as response.
Qmemserver accept the following url requests:
- http://localhost:8080/qhost for qhost data
- http://localhost:8080/qstat for qstat data
The JSON format for a successful request is:
{"success" : true, "message" : "", "data" : {json_data}}
The JSON format for a failed request is:
{"success" : false, "message" : "an_error_message", "data" : {}}
qmemview.py: display information about memory usage on your cluster
qmemview.py has the following options:
qmemview.py <url:port> : display all hosts qmemview.py <url:port> -h : display this help qmemview.py <url:port> -u : display all job details / host qmemview.py <url:port> -j <jobid>: display details for one job (set -u automatically) qmemview.py <url:port> -o <owner>: display owner's job details (set -u automatically) qmemview.py <url:port> -h <hostname> : display only selected host qmemview.py <url:port> -u -h <hostname> : display only selected host with job details for this host qmemview.py <url:port> -h <hostname> -j <jobid> : display only selected host with job details for jobid only <url:port> argument should be set to point on qmemserver address and port. ie: qmemview.py localhost:8080
- TIPS, for simplicity you can create a shell alias:
- alias qmemview="qmemview.py localhost:8080"
Returned by qhost:
qhost_data[hostname]['num_proc']:str
['mem_total']:str
['jobs'][jobid]['jobcount']:int
['master']:bool
['taskid']:list
['owner']:str
['jobname']:str
- qhost_data[hostname]['num_proc']
- number of processors on hostname
- qhost_data[hostname]['mem_total']
- total memory available on hostname
- qhost_data[hostname]['jobs']
- contain jobid running on hostname
- qhost_data[hostname]['jobs'][jobid]['jobcount']
- slots used by this job on hostname (don't rely on it for array task, use taskid list length instead)
- qhost_data[hostname]['jobs'][jobid]['master']
- is master run on hostname ?
- qhost_data[hostname]['jobs'][jobid]['taskid']
- list of taskid running on hostname
- qhost_data[hostname]['jobs'][jobid]['owner']
- owner of jobid
- qhost_data[hostname]['jobs'][jobid]['jobname']
- job name
Returned by qstat:
qstat_data[owner]['uid']:str
['jobs'][jobid]['requested_h_vmem_strval']:str
['requested_h_vmem_dblval']:float
['hostname'][hostname]['master']:str
['slave']:str
[taskid]:str
- qstat_data[owner]['uid']
- userid of owner
- qstat_data[owner]['jobs'][jobid]['requested_h_vmem_strval']
- requested h_vmem (string format) for owner's jobid
- qstat_data[owner]['jobs'][jobid]['requested_h_vmem_dblval']
- requested h_vmem (double format) for owner's jobid
- qstat_data[owner]['jobs'][jobid]['hostname']
- hostnames where jobid run
- qstat_data[owner]['jobs'][jobid]['hostname'][hostname]['master']
- memory consummed by jobid master on hostname
- qstat_data[owner]['jobs'][jobid]['hostname'][hostname]['slave']
- memory consummed by jobid slave on hostname
- qstat_data[owner]['jobs'][jobid]['hostname'][hostname][taskid]
- memory consummed by jobid task on hostname
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent