madmaze / pynmonanalyzer Goto Github PK

Hi,

pyNmonAnalyser fails to parse NMON files which have been generated under AIX Systems with the following error message:

WARNING - Discarding line with missing Timestamp ['TOP', '+PID', 'Time', '%CPU', '%Usr', '%Sys', 'Threads', 'Size', 'ResText', 'ResData', 'CharIO', '%RAM', 'Paging', 'Command', 'WLMclass']
Traceback (most recent call last):
File "./pyNmonAnalyzer.py", line 232, in
nmonAnalyzer = pyNmonAnalyzer(args)
File "./pyNmonAnalyzer.py", line 84, in init
self.processedData = self.nmonParser.parse()
File "/home/guilhem/Documents/nmon/nmonpython/pyNmonAnalyzer-master/pyNmonParser.py", line 112, in parse
self.processLine(bits[0],bits)
File "/home/guilhem/Documents/nmon/nmonpython/pyNmonAnalyzer-master/pyNmonParser.py", line 87, in processLine
col.append(line[n+1])
IndexError: list index out of range

I have tested with various NMON files coming different AIX and NMON versions with the same failure.

I have extracted 2 NMON AIX files sample available to download with this public link:

https://dl.dropboxusercontent.com/u/63061887/NMON/NMON_AIX_Extract.zip

Thank you for your help.

Guilhem

Is there any plan to add this tool to pypi?

This is a very important feature of the Java nmonvisualizer https://github.com/nmonvisualizer/nmonvisualizer , it seems like pyNmonAnalyzer is missing this. Create a new option to specify multiple files, call the parser iteratively, and join the data across days.

While using pyNmonAnalyzer, which has so far been a highly-useful tool, I've noticed that pyNmonPlotter tends to label disks/partitions with their ID (i.e. sda1, dm-0). Where I work, we are using a report.config file and trying to determine how to edit it and the python files to allow the use of actual disk/partition labels in the legend for each plot (i.e. root, swap, etc.). Is this possible? If so, how would I go about it?

We usually monitor multiple machines at one time during a test run. It would be ideal if we could plot them all together, and give the option to add/remove plot lines from the chart.

if config is missing, still creates directories and then ends

Should have an option to run with default and an option to print out the config file

• have added features
• should add troubleshooting section, mention issue with remove/local CSV files

• write a server which will connect to a named pipe then launches nmon to output into this named pipe. The server then processes that data and makes it available over an API which dygraphs can then dynamically load.

Hello,

While using pyNmonParser -b -o testReport -i test.nmon I'm getting this error:

INFO - Preparing CSV files..
INFO - Preparing interactive Report..
Traceback (most recent call last):
File "/usr/local/bin/pyNmonAnalyzer", line 11, in
sys.exit(pyNmonAnalyzer())
File "/usr/local/lib/python3.6/dist-packages/pynmonanalyzer/pyNmonAnalyzer.py", line 114, in init
self.processedData, self.args.dygraphLoc)
File "/usr/local/lib/python3.6/dist-packages/pynmonanalyzer/pyNmonAnalyzer.py", line 308, in buildInteractiveReport
reportConfig, self.args.outdir, data=data, dygraphLoc=dygraphLoc)
File "/usr/local/lib/python3.6/dist-packages/pynmonanalyzer/pyNmonReport.py", line 107, in createInteractiveReport
if max(numericArray) > localMax or localMax == None:
TypeError: '>' not supported between instances of 'float' and 'NoneType'

And while I can see the csv folder and its contents, the interactiveReport.html file is empty (0 Kb)

Can anyone tell me what could be the the reason behind this error?

i use the demo test.nmon
and new an html report ,but always 'loading'
it seems not load the js, right?

I use pip to install pyNmonAnalyzer matplotlib and numpy like this:

(pyNmonAnalyzer) [lujun9972@F31 work]$ pip install pyNmonAnalyzer matplotlib numpy
Collecting pyNmonAnalyzer
  Using cached pyNmonAnalyzer-1.0.8.tar.gz
Collecting matplotlib
  Using cached matplotlib-2.1.1-cp36-cp36m-manylinux1_x86_64.whl
Collecting numpy
  Using cached numpy-1.14.0-cp36-cp36m-manylinux1_x86_64.whl
Collecting six>=1.10 (from matplotlib)
  Using cached six-1.11.0-py2.py3-none-any.whl
Collecting pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 (from matplotlib)
  Using cached pyparsing-2.2.0-py2.py3-none-any.whl
Collecting python-dateutil>=2.0 (from matplotlib)
  Using cached python_dateutil-2.6.1-py2.py3-none-any.whl
Collecting cycler>=0.10 (from matplotlib)
  Using cached cycler-0.10.0-py2.py3-none-any.whl
Collecting pytz (from matplotlib)
  Using cached pytz-2017.3-py2.py3-none-any.whl
Installing collected packages: pyNmonAnalyzer, six, pyparsing, python-dateutil, cycler, numpy, pytz, matplotlib
  Running setup.py install for pyNmonAnalyzer ... done
Successfully installed cycler-0.10.0 matplotlib-2.1.1 numpy-1.14.0 pyNmonAnalyzer-1.0.8 pyparsing-2.2.0 python-dateutil-2.6.1 pytz-2017.3 six-1.11.0

then I try to export the default config like this:

(pyNmonAnalyzer) [lujun9972@F31 work]$ pyNmonAnalyzer --defaultConfig

There is an error reported:

Traceback (most recent call last):
  File "/home/lujun9972/work/pyNmonAnalyzer/bin/pyNmonAnalyzer", line 11, in <module>
    load_entry_point('pyNmonAnalyzer==1.0.8', 'console_scripts', 'pyNmonAnalyzer')()
  File "/home/lujun9972/work/pyNmonAnalyzer/lib/python3.6/site-packages/pkg_resources/__init__.py", line 565, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/home/lujun9972/work/pyNmonAnalyzer/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2631, in load_entry_point
    return ep.load()
  File "/home/lujun9972/work/pyNmonAnalyzer/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2291, in load
    return self.resolve()
  File "/home/lujun9972/work/pyNmonAnalyzer/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2297, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/home/lujun9972/work/pyNmonAnalyzer/lib/python3.6/site-packages/pynmonanalyzer/__init__.py", line 18, in <module>
    from pynmonanalyzer import pyNmonAnalyzer as pna
  File "/home/lujun9972/work/pyNmonAnalyzer/lib/python3.6/site-packages/pynmonanalyzer/pyNmonAnalyzer.py", line 25, in <module>
    import pyNmonParser
ModuleNotFoundError: No module named 'pyNmonParser'

what is the problem?

I am using python3.6

Thanks

at the moment interactive pyNmonAnalyzer reports point at dygraphs.org. Enable an option to pass in alternative location such as ./dygraphs.js

Using the most recent git version as of 10/9/2013:

When I create the interactive graph, the HTML display is incorrect.

The graphs are labeled CPU, DISKBUSY, and MEM, which is what I want to see, but the data are incorrect.

The headings for the CPU graph are the disk names 'sda1, etc' (and also do not reflect the actual disks (vda1 - vda8). The graph itself and its values appear correct.

The graph labeled DISKBUSY is actually the memory graph, but has the correct items laid out.

The graph labeled MEM is actually the network graph, but has the correct items laid out.

The JavaScript from the source shows:

var dataSources=[
"csv/DISKBUSY.csv",
"csv/MEM.csv",
"csv/NET.csv" ];

I imagine it should be CPU, MEM, NET as far as the 3 graphs that are laid out in the output.

I tried editing the resulting HTML to put CPU_ALL.csv, MEM.csv, and NET.csv in the right places, but the graphs failed to load after that point.

as u see the image

last logs:
dates.py line 254,
dt = datetime.datetime.fromorddinal(ix).replace(tzinfo=UTC)
ValueError: ordinal must be >= 1

I downloaded your master copy zip from github web. I unzip, did some test play around using your test nmon file and it's working. But when I feed it with my nmon data from AIX system, it emits errors. Sorry, I can't paste the entire log, but I am seeing this:

INFO - Preparing graphs..
/usr/lib/pymodules/python2.7/matplotlib/axes.py:4486: UserWarning: No labeled objects found. Use label='...' kwarg on individual plots.
warnings.warn("No labeled objects found. "
Traceback (most recent call last):
File "/home/daixtr/src/py/pymonizer/pyNmonAnalyzer-master/pyNmonAnalyzer.py", line 231, in
nmonAnalyzer = pyNmonAnalyzer(args)
File "/home/daixtr/src/py/pymonizer/pyNmonAnalyzer-master/pyNmonAnalyzer.py", line 91, in init
self.buildReport()
File "/home/daixtr/src/py/pymonizer/pyNmonAnalyzer-master/pyNmonAnalyzer.py", line 182, in buildReport
outFiles = nmonPlotter.plotStats(reportConfig)
File "/home/daixtr/src/py/pymonizer/pyNmonAnalyzer-master/pyNmonPlotter.py", line 89, in plotStats
fname = self.plotStat(data, xlabel="Time", ylabel="Disk Busy (%)", title="Disk Busy vs Time", yrange=[0,105])
File "/home/daixtr/src/py/pymonizer/pyNmonAnalyzer-master/pyNmonPlotter.py", line 195, in plotStat
plt.savefig(outFilename)
File "/usr/lib/pymodules/python2.7/matplotlib/pyplot.py", line 471, in savefig
return fig.savefig(_args, *_kwargs)
File "/usr/lib/pymodules/python2.7/matplotlib/figure.py", line 1185, in savefig
self.canvas.print_figure(_args, *_kwargs)
File "/usr/lib/pymodules/python2.7/matplotlib/backend_bases.py", line 2021, in print_figure
*_kwargs)
File "/usr/lib/pymodules/python2.7/matplotlib/backends/backend_agg.py", line 469, in print_png
FigureCanvasAgg.draw(self)
File "/usr/lib/pymodules/python2.7/matplotlib/backends/backend_agg.py", line 421, in draw
self.figure.draw(self.renderer)
File "/usr/lib/pymodules/python2.7/matplotlib/artist.py", line 55, in draw_wrapper
draw(artist, renderer, *args, *_kwargs)
File "/usr/lib/pymodules/python2.7/matplotlib/figure.py", line 898, in draw
func(_args)
File "/usr/lib/pymodules/python2.7/matplotlib/artist.py", line 55, in draw_wrapper
draw(artist, renderer, *args, *_kwargs)
File "/usr/lib/pymodules/python2.7/matplotlib/axes.py", line 1997, in draw
a.draw(renderer)
File "/usr/lib/pymodules/python2.7/matplotlib/artist.py", line 55, in draw_wrapper
draw(artist, renderer, _args, *_kwargs)
File "/usr/lib/pymodules/python2.7/matplotlib/axis.py", line 1041, in draw
ticks_to_draw = self._update_ticks(renderer)
File "/usr/lib/pymodules/python2.7/matplotlib/axis.py", line 931, in _update_ticks
tick_tups = [ t for t in self.iter_ticks()]
File "/usr/lib/pymodules/python2.7/matplotlib/axis.py", line 881, in iter_ticks
majorLabels = [self.major.formatter(val, i) for i, val in enumerate(majorLocs)]
File "/usr/lib/pymodules/python2.7/matplotlib/dates.py", line 340, in call
raise ValueError('DateFormatter found a value of x=0, which is an illegal date. This usually occurs because you have not informed the axis that it is plotting dates, eg with ax.xaxis_date()')
ValueError: DateFormatter found a value of x=0, which is an illegal date. This usually occurs because you have not informed the axis that it is plotting dates, eg with ax.xaxis_date()

I am trying to use pyNmonAnalyser-0.1 to read nmon files. It is discarding all lines that start with TOP with messages like the following

'Discarding line with missing Timestamp ['TOP', '0001639', 'T0002', '0.30', '0.00', '0.30', '0', '0', '0', '0', '0', '0', '0', 'flush-253:0']’

Apparently it does not have support to chart the cpu/memory usage of top processes.

self.processedData should be cleared in init, otherwise it will cause subsequent files unable to load new label because of the data in it, triggers warning at line 92

What are the log levels?

Is there a silent option? I plan on running this via cron and I'd rather not fill up /var/log/cron with the TOP warnings.

/pyNmonAnalyzer.py -b -x node001.nmon
Traceback (most recent call last):
File "./pyNmonAnalyzer.py", line 24, in
import pyNmonPlotter
File "/root/pyNmonPlotter.py", line 20, in
import matplotlib.pyplot as plt
File "/usr/lib64/python2.6/site-packages/matplotlib/pyplot.py", line 78, in
new_figure_manager, draw_if_interactive, show = pylab_setup()
File "/usr/lib64/python2.6/site-packages/matplotlib/backends/init.py", line 25, in pylab_setup
globals(),locals(),[backend_name])
File "/usr/lib64/python2.6/site-packages/matplotlib/backends/backend_gtkagg.py", line 10, in
from matplotlib.backends.backend_gtk import gtk, FigureManagerGTK, FigureCanvasGTK,
File "/usr/lib64/python2.6/site-packages/matplotlib/backends/backend_gtk.py", line 8, in
import gtk; gdk = gtk.gdk
File "/usr/lib64/python2.6/site-packages/gtk-2.0/gtk/init.py", line 64, in
_init()
File "/usr/lib64/python2.6/site-packages/gtk-2.0/gtk/init.py", line 52, in _init
_gtk.init_check()
RuntimeError: could not open display

I tried to generate the graphs but they do not show any results
Command used: pyNmonAnalyzer -b -o testReport1 -i <nmon_file>

Often JSON output could be useful, add an option to produce JSON output