toolsforexperiments / plottr Goto Github PK
View Code? Open in Web Editor NEWA flexible plotting and data analysis tool.
Home Page: https://github.com/toolsforexperiments/plottr
License: MIT License
A flexible plotting and data analysis tool.
Home Page: https://github.com/toolsforexperiments/plottr
License: MIT License
To generate a quick overview of all measurement data, it has become common practice to copy paste the graphs in to the onenote. Now this is done by selecting a windows over the plots, which is not so practical.
So with this, could it be possible add two buttons below the graph showing one for a tight layout and one to copy the plot to the clipboard.
Thanks in forward!
I tried to install plottr in a fresh environment that qcodes was already installed there.
I couldn't run plottr-inspectr and this error returned:
raise PythonQtError('No Qt bindings could be found') qtpy.PythonQtError: No Qt bindings could be found
Then, I installed pyqt5 in the environment to be able to run the plottr.
Shouldn't pyqt5 to be a required package in the installation of plottr, if for any reason, qcodes doesn't install pyqt5 in the environment?
I'm trying to plot an unfinished run with three axes (say two RF powers and an RF frequency), something like
sweep(station.hetsrc.RF.power,spec_powers)(sweep(station.qubsrc.RF.power,qub_powers)(
ss.measure_soft_time_avg_spec(spec_frequencies, spec_src, setup=False,
setup_awg=False)))
where we have different dimensions for each of these sweeps, say 10, 20, and 50 points (not sure if that is relevant). This results in a crash with the following error:
python apps\inspectr.py --dbpath=d:\OneDrive\Setups\LK1\LK1_data\experiments.db
Traceback (most recent call last):
File "d:\onedrive\setups\lk1\lk1_code\code\plottr\plottr\plot\mpl.py", line 238, in setData
self._plot2d(data, ax, axesNames[0], axesNames[1], dn)
File "d:\onedrive\setups\lk1\lk1_code\code\plottr\plottr\plot\mpl.py", line 197, in _plot2d
im = ppcolormesh_from_meshgrid(ax, x, y, z)
File "d:\onedrive\setups\lk1\lk1_code\code\plottr\plottr\plot\mpl.py", line 73, in ppcolormesh_from_meshgrid
if np.any(np.isnan(x)) or np.any(np.isnan(y)):
TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''
This only occurs when the measurement is still on-going, so I am guessing that it might be related to the guessing of the shape?
We want to implement a standardized way to analyze data (for fitting, etc.).
Some notes are prototyped already.
We need:
should allow setting log levels, etc.
needs a unified structure for how an app looks like?
When I have a dataset with 1 record, being a blob (in this case of frequencies and magnitudes), plottr is not able to plot this
The warning is something like this:
py.warnings ¦ _showwarnmsg ¦ \plottr\plottr\plot\mpl.py:877: ComplexWarning: Casting complex values to real discards the imaginary part
If a dataset has complex values, plottr terminal throws these kinds of warning messages when plottr loads that dataset for plotting. It is not a big issue, but it would be nice to somehow handle that.
The README.md refers to a file plottr.exe which does not exist. The docs should perhaps refer to plottr.py which contains main. In any event, extensions .exe and .bat are meaningless on linux systems.
When refreshing the data, sometimes plottr gives a traceback similar to the following form
Traceback (most recent call last):
File "plottr\plottr\apps\inspectr.py", line 425, in plotRun
fc, win = autoplotQcodesDataset(pathAndId=(self.filepath, runId))
File "plottr\plottr\apps\autoplot.py", line 238, in autoplotQcodesDataset
win = QCAutoPlotMainWindow(fc, pathAndId=pathAndId)
File "plottr\plottr\apps\autoplot.py", line 213, in __init__
self.setDefaults()
File "plottr\plottr\apps\autoplot.py", line 182, in setDefaults
self.fc.nodes()['Dimension assignment'].dimensionRoles = drs
File "plottr\plottr\node\node.py", line 40, in wrap
self.ui.optSetters[optName](val)
File "plottr\plottr\node\dim_reducer.py", line 627, in setRoles
self.widget.setRole(dimName, role)
File "plottr\plottr\node\dim_reducer.py", line 336, in setRole
super().setRole(dim, role, **kw)
File "plottr\plottr\node\dim_reducer.py", line 276, in setRole
super().setRole(dim, role)
File "plottr\plottr\node\dim_reducer.py", line 202, in setRole
self.choices[dim]['roleSelectionWidget'].setCurrentText(role)
KeyError: 'NW_bias'
Where NW_bias
is one of the parameters that i am measuring
The current backend for inspectr is the qcodes database format. We would like to use other sources of datasets as well. Can the plottr\apps\apps\inspectr.py
be updates so that the inspectr code is separated from the qcodes database backend and it is better documented what would have to be implemented to create a new backend?
I used the following code:
import numpy as np
import qcodes as qc
from qcodes import (
Measurement,
experiments,
initialise_or_create_database_at,
new_experiment,
)
from qcodes.dataset.plotting import plot_dataset
station = qc.Station()
initialise_or_create_database_at("~/plottr_failing.db")
exp = new_experiment('dataset_context_manager2',
sample_name="no sample1")
meas = Measurement()
meas.write_period = 2
N = 5
meas.register_custom_parameter(name='bfield',
label='B', unit='Tesla',
paramtype='numeric')
meas.register_custom_parameter(name='phase',
label='Phase', unit='rad',
paramtype='numeric',
setpoints=['bfield'])
meas.register_custom_parameter(name='time',
label='Time', unit='s',
paramtype='numeric')
meas.register_custom_parameter(name='signal',
label='Signal', unit='V',
paramtype='numeric',
setpoints=('time','bfield'))
with meas.run() as datasaver:
for bf in np.linspace(0, 25, 10):
bfield = bf
phase = np.random.rand()
time_data = np.linspace(0, 1, N)
for time in time_data:
signal_data = np.sin(2*np.pi*time) + 0.025*np.random.randn(N)
datasaver.add_result(('time', time), ('signal', signal_data),('bfield', bfield), ('phase', phase))
dataset = datasaver.dataset # convenient to have for plotting
to generate dataset which looks like this:
But Plottr crashes and gives the following error
Qcodes Logfile : C:\Users\a-halakh\.qcodes\logs\200402-41928-qcodes.log
2020-04-02 19:36:48,803 ¦ plottr ¦ ERROR ¦ grid ¦ process ¦ 299 ¦ Unknown data type <class 'plottr.data.datadict.DataDictBase'>.
2020-04-02 19:36:48,810 ¦ plottr ¦ ERROR ¦ grid ¦ process ¦ 299 ¦ Unknown data type <class 'plottr.data.datadict.DataDictBase'>.
2020-04-02 19:36:48,816 ¦ plottr ¦ ERROR ¦ grid ¦ process ¦ 299 ¦ Unknown data type <class 'plottr.data.datadict.DataDictBase'>.
Traceback (most recent call last):
File "c:\users\a-halakh\documents\microsoft\plottr\plottr\apps\inspectr.py", line 438, in plotRun
fc, win = autoplotQcodesDataset(pathAndId=(self.filepath, runId))
File "c:\users\a-halakh\documents\microsoft\plottr\plottr\apps\autoplot.py", line 253, in autoplotQcodesDataset
win = QCAutoPlotMainWindow(fc, pathAndId=pathAndId)
File "c:\users\a-halakh\documents\microsoft\plottr\plottr\apps\autoplot.py", line 226, in __init__
self.setDefaults()
File "c:\users\a-halakh\documents\microsoft\plottr\plottr\apps\autoplot.py", line 189, in setDefaults
self.fc.nodes()['Dimension assignment'].dimensionRoles = drs
File "c:\users\a-halakh\documents\microsoft\plottr\plottr\node\node.py", line 40, in wrap
self.ui.optSetters[optName](val)
File "c:\users\a-halakh\documents\microsoft\plottr\plottr\node\dim_reducer.py", line 627, in setRoles
self.widget.setRole(dimName, role)
File "c:\users\a-halakh\documents\microsoft\plottr\plottr\node\dim_reducer.py", line 336, in setRole
super().setRole(dim, role, **kw)
File "c:\users\a-halakh\documents\microsoft\plottr\plottr\node\dim_reducer.py", line 276, in setRole
super().setRole(dim, role)
File "c:\users\a-halakh\documents\microsoft\plottr\plottr\node\dim_reducer.py", line 202, in setRole
self.choices[dim]['roleSelectionWidget'].setCurrentText(role)
KeyError: 'bfield'
c:\code\plottr\plottr\plot\mpl.py:477: MatplotlibDeprecationWarning: Passing non-integers as three-element position specification is deprecated since 3.3 and will be removed two minor releases later.
self.axes.append(self.fig.add_subplot(nrows, ncols, i))
Max refresh time is now 99s, I think a third digit might be nice.
One issue that crops up fairly often with lockin measurements (though other measurements probably suffer from similar issues) is that a transient completely ruins the color scale and leaves 99% of the data at one end of the color bar. It would be nice to be able to set the range of the color bar manually in the GUI and have the color scale in the plot change accordingly.
I would like an option that two lineplots can be plotted side by side, instead of multiple lines in 1 plot
Users report that loading a .db
file into inspectr
takes ~seconds when the number of runs in the database is ~300-400. When the number of runs in ~1000, it takes ~2 minutes.
It should be easy to improve this via smarted SQL requests.
plottr crashes when one tries to plot a measurement that was recorded without prior definition of a station in qcodes. Once a station is defined, plottr can plot the new measurements as usual.
Here is a sample code that first creates a measurement without having defined a station (id1) and a second one after defining a station (id2).
from functools import partial
import os
import numpy as np
from qcodes.dataset.sqlite.database import initialise_or_create_database_at
from qcodes.dataset.experiment_container import load_or_create_experiment
from qcodes.tests.instrument_mocks import DummyInstrument
from qcodes.dataset.measurements import Measurement
from qcodes.dataset.plotting import plot_dataset
from qcodes import Station
tutorial_db_path = os.path.join(os.getcwd(), 'plottr_crash.db')
initialise_or_create_database_at(tutorial_db_path)
load_or_create_experiment("plottr", sample_name="no sample")
# preparatory mocking of physical setup
dac = DummyInstrument('dac', gates=['ch1', 'ch2'])
dmm = DummyInstrument('dmm', gates=['v1', 'v2'])
def gauss_model(x0: float, y0: float, sigma: float, noise: float=0.0005):
"""
Returns a generator sampling a gaussian. The gaussian is
normalised such that its maximal value is simply 1
"""
while True:
(x, y) = yield
model = np.exp(-((x0-x)**2+(y0-y)**2)/2/sigma**2)*np.exp(2*sigma**2)
noise = np.random.randn()*noise
yield model + noise
# and finally wire up the dmm v1 to "measure" the gaussian
gauss = gauss_model(0.1, 0.2, 0.25)
next(gauss)
def measure_gauss(dac):
val = gauss.send((dac.ch1.get(), dac.ch2.get()))
next(gauss)
return val
dmm.v1.get = partial(measure_gauss, dac)
def do1d(param_set, start, stop, num_points, delay, *param_meas):
meas = Measurement()
meas.register_parameter(param_set) # register the first independent parameter
output = []
param_set.post_delay = delay
# do1D enforces a simple relationship between measured parameters
# and set parameters. For anything more complicated this should be reimplemented from scratch
for parameter in param_meas:
meas.register_parameter(parameter, setpoints=(param_set,))
output.append([parameter, None])
with meas.run() as datasaver:
for set_point in np.linspace(start, stop, num_points):
param_set.set(set_point)
for i, parameter in enumerate(param_meas):
output[i][1] = parameter.get()
datasaver.add_result((param_set, set_point),
*output)
dataset = datasaver.dataset # convenient to have for plotting
return dataset
dataset = do1d(dac.ch1, 0, 1, 10, 0.01, dmm.v1, dmm.v2)
# Add all instruments to the station
station = Station(dac, dmm)
dataset = do1d(dac.ch1, 0, 1, 10, 0.01, dmm.v1, dmm.v2)
Plottr crashes with the following output on the console:
(qcodes) E:>python .\plottr\apps\inspectr.py
Traceback (most recent call last):
File "e:\plottr\plottr\apps\inspectr.py", line 438, in plotRun
fc, win = autoplotQcodesDataset(pathAndId=(self.filepath, runId))
File "e:\plottr\plottr\apps\autoplot.py", line 253, in autoplotQcodesDataset
win = QCAutoPlotMainWindow(fc, pathAndId=pathAndId)
File "e:\plottr\plottr\apps\autoplot.py", line 229, in init
self.snapshotWidget.loadSnapshot(self.loaderNode.dataSnapshot)
File "e:\plottr\plottr\gui\widgets.py", line 150, in loadSnapshot
items = dictToTreeWidgetItems(snapshotDict)
File "e:\plottr\plottr\gui\widgets.py", line 163, in dictToTreeWidgetItems
for k, v in d.items():
AttributeError: 'NoneType' object has no attribute 'items'
In the previous version of plottr the measurement ID was included in the title of the plot, but now it is only visible in the window frame. This makes logging/saving screenshots of the data a little more involved. Would it be possible to add this again?
The following test fails
https://github.com/data-plottr/plottr/blob/e47c9941bc178a1d5f0d2e810776cacfd9f14132/test/pytest/test_ddh5.py#L84
FAILED [100%]
test/pytest/test_ddh5.py:83 (test_loader_node)
qtbot = <pytestqt.qtbot.QtBot object at 0x121e2a7d0>
def test_loader_node(qtbot):
dds.DDH5Loader.useUi = False
x = np.arange(3)
y = np.repeat(np.linspace(0, 1, 5).reshape(1, -1), 3, 0)
z = np.arange(y.size).reshape(y.shape)
data = dd.DataDict(
x=dict(values=x, unit='A',
__info__='careful!',
__moreinfo__='more words in here'),
y=dict(values=y, unit='B'),
z=dict(values=z, axes=['x', 'y'], unit='C'),
__desc__='some description',
)
assert data.validate()
dds.datadict_to_hdf5(data, FN, append_mode=dds.AppendMode.new)
assert _clean_from_file(dds.datadict_from_hdf5(FN)) == data
fc = linearFlowchart(('loader', dds.DDH5Loader))
node = fc.nodes()['loader']
assert fc.outputValues()['dataOut'] is None
node.filepath = FN
> out = fc.outputValues()['dataOut'].copy()
E AttributeError: 'NoneType' object has no attribute 'copy'
test_ddh5.py:109: AttributeError
I would like to be able to plot any two arrays of the same shape against eachother even if both are dependent or independent variables. For instance I am monitoring resistance vs temperature and i have as independent variable a counter or time. I would like to also be able to plot R vs T.
I'm running into a problem when I try to plot a specific type of VNA measurement; plottr will plot the first trace just fine, but once the second one begins it crashes. Opening it again still leads to a crash. Once the measurement has finished I can plot it just fine.
The error message I receive is
Traceback (most recent call last):
File "d:\onedrive\setups\lk1\lk1_code\code\plottr\plottr\plot\mpl.py", line 238, in setData
self._plot2d(data, ax, axesNames[0], axesNames[1], dn)
File "d:\onedrive\setups\lk1\lk1_code\code\plottr\plottr\plot\mpl.py", line 197, in _plot2d
im = ppcolormesh_from_meshgrid(ax, x, y, z)
File "d:\onedrive\setups\lk1\lk1_code\code\plottr\plottr\plot\mpl.py", line 74, in ppcolormesh_from_meshgrid
x, y = interp_meshgrid_2d(x, y)
File "d:\onedrive\setups\lk1\lk1_code\code\plottr\plottr\utils\data.py", line 16, in interp_meshgrid_2d
yy2 = pd.DataFrame(yy).interpolate(axis=0).values
File "C:\ProgramData\Anaconda3\envs\qcodes\lib\site-packages\pandas\core\generic.py", line 6015, in interpolate
raise TypeError("Cannot interpolate with all NaNs.")
TypeError: Cannot interpolate with all NaNs.
The measurement code used to generate the measurement is
setup_zerospan_trace(5.345737e9, 51)
sweep_obj = sweep(twpa_pump.power, np.arange(8.0, 12.0, .2))(
sweep(twpa_pump.frequency, np.arange(7.8e9, 8.1e9, 1.0e6)))(
measure(get_SNR)
)
result = do_experiment(f"TWPA_SNR/{SAMPLE}", sweep_obj, live_plot=False)
Here get_SNR
is defined as
@getter(('signal', 'dB'), ('noise', 'dB'), ('SNR', 'dB'))
def get_SNR():
time.sleep(0.01)
mag, phase = take_trace(plot=False)
sig = np.abs(mag) * np.exp(-1j*phase)
lin_mean = np.abs(sig.mean())
lin_std = np.abs(sig.std())
return 20*np.log10(lin_mean), 20*np.log10(lin_std), 20*np.log10(lin_mean/lin_std)
where take_trace
is
def take_trace(chan='S21', plot=False):
"""
Get the data of a currently measured trace.
The trace has to be setup already.
Returns magnitude (in dB) and phase (in rad).
If plot is true, make a simple plot of the magnitude vs frequency.
"""
vna = qc.Station.default.vna
trace = getattr(vna.channels, chan)
fvals = np.linspace(trace.start(), trace.stop(), trace.npts())
mag, phase = trace.trace_mag_phase()
if plot:
fig, ax = plt.subplots(1, 1)
ax.plot(fvals*1e-9, 20*np.log10(mag))
ax.grid(dashes=[1,1])
ax.set_xlabel('Frequency (GHz)')
ax.set_ylabel('Magnitude (dBm)')
return mag, phase
and setup_zerospan_trace
is
def setup_zerospan_trace(frq, npts, chan='S21', span=1.):
"""
Setup the VNA to measure a trace around <frq> with <npts> samples
in a very narray frequency span.
"""
vna = qc.Station.default.vna
trace = getattr(vna.channels, chan)
trace.center(frq)
trace.span(span)
trace.npts(npts)
trace.autoscale()
if not vna.rf_power():
vna.rf_on()
expanding should only be done on-demand, i.e., after extracting data to work with.
in the current way, this can result in a lot of redundant data.
While launching the inspectr (from within the plottr root), I'm just getting the error:
File "apps/inspectr.py", line 15, in <module>
from .. import log as plottrlog
ValueError: attempted relative import beyond top-level package
I'm using Python 3.6.8 in an Anaconda env. Am I missing something stupid (I'm new here!)?
I ran plottr from the command line and tried viewing some old 2D datasets. They open fine in the old version of plottr, but in the new version they only show up properly if I select 'no grid' and then zoom in on the data. The top right plot window shows the output with 'no grid' selected. I've also attached the db file with these 2D maps.
i.e., only give the function, computation of values on demand
Original request:
Plottr does not show plots for multiple measured parts of data
Do a measurement of several dependent parameters, only the first one will show up in plottr
Added comment:
correct, but "first" is a default selection. User can select which dependent parameter to plot during the live plotting. So perhaps this is not a bug, but a feature request to plot dependent parameters at the same time on different plots :)
Feel free to discuss/close/implement/etc...
it should not be too hard to allow easier addition of data to datadict.
For example, we want to be able to add scalar data, and datadict should recognize that this is one new row.
include tool widgets that allow:
Since for now plottr is used as a plotting tool together with inspectr to browse through the database, it would be nice to have a small widget that would show the snapshot next to the plot of the run.
Hello, new here. I run into the sqlite3.OperationalError when running plottr. It may not open the database file and could not realize live plotting. How could I make it work?
Thanks!
@wpfff
File "c:\users\zhouz\anaconda3\lib\site-packages\qcodes\dataset\sqlite\database.py", line 136, in connect
sqlite3_conn = sqlite3.connect(name, detect_types=sqlite3.PARSE_DECLTYPES,
sqlite3.OperationalError: unable to open database file
Tried to integrate plottr (v.0.1.0) to qcodes (v.0.1.11) according to this, but it looks like there is some module structure mismatch (?) between them. According to default qcodestc.json, qcodes tries to call plottr.qcodes_dataset.QcodesDatasetSubscriber
, but (1) currently qcodes_dataset.py
is located under plottr/data
folder, and (2) I'm not able to find QcodesDatasetSubscriber
anywhere. So the subscription to the plottr has been failed.
I'm new in the field, so any help would be appreciated
Using a dataset created with the following code:
import numpy as np
import qcodes as qc
qc.config.core.db_location = "./test.db"
qc.initialise_database()
exp = qc.load_or_create_experiment("categorical_data", sample_name="no sample")
m = qc.Measurement(exp=exp)
m.register_custom_parameter('x')
m.register_custom_parameter('label', paramtype='text')
m.register_custom_parameter('y', setpoints=('x', 'label'))
def f(x, label):
if label == 'trig':
return np.sin(x * 2 * np.pi)
elif label == 'gauss':
return np.exp(-x**2)
with m.run() as datasaver:
for x in np.linspace(-1, 1, 10):
for label in ['trig', 'gauss']:
datasaver.add_result(('x', x),
('label', label),
('y', f(x, label)))
When opening the dataset with Autoplot, it crashes with the following stack trace:
Traceback (most recent call last):
File "plottr/plottr/apps/inspectr.py", line 425, in plotRun
fc, win = autoplotQcodesDataset(pathAndId=(self.filepath, runId))
File "plottr/plottr/apps/autoplot.py", line 237, in autoplotQcodesDataset
win = QCAutoPlotMainWindow(fc, pathAndId=pathAndId)
File "plottr/plottr/apps/autoplot.py", line 213, in __init__
self.setDefaults()
File "plottr/plottr/apps/autoplot.py", line 182, in setDefaults
self.fc.nodes()['Dimension assignment'].dimensionRoles = drs
File "plottr/plottr/node/node.py", line 40, in wrap
self.ui.optSetters[optName](val)
File "plottr/plottr/node/dim_reducer.py", line 627, in setRoles
self.widget.setRole(dimName, role)
File "plottr/plottr/node/dim_reducer.py", line 336, in setRole
super().setRole(dim, role, **kw)
File "plottr/plottr/node/dim_reducer.py", line 276, in setRole
super().setRole(dim, role)
File "plottr/plottr/node/dim_reducer.py", line 202, in setRole
self.choices[dim]['roleSelectionWidget'].setCurrentText(role)
KeyError: 'label'
The (or at least one) correct behavior would be to add the categorical dimension as a slider, which works if the data has at least one 'numerical' dimension (if not, then I'm not sure what should happen).
Found this bug when loading a new db-file:
When the checkbox "Auto-plot new" is enabled in the main plottr window and a new db-file is loaded, it will load all the experiments from that db-file.
For now it is easy to circumvent by unclicking the box, but I assume this is unwanted behaviour.
Everything is running smoothly, dual plotting is possible.
But when clicking on the parameter name, it causes a crash.
Only error notification is:
TypeError: native Qt signal is not callable
Hey @wpfff
Currently the plot axes are labelled with '{param_name}_{unit}'. This is a bit different to plot_by_id
which will use the parameter label. This is kind of nice when you use a station yaml file to relabel things from 'ch2.source_voltage' to 'Gate Voltage' or similar.
Would this be an easy thing to add?
example:
When users have data where the axes are not ordered from slowest to fastest (wrt to the axis specification of the data), then currently this cannot be specified manually.
This needs to be included. The shape-guessing function does that already, but we also want a manual one.
instead individual consumers/nodes/... should use what they want instead directly.
Plottr cannot plot sweeps where the dependent variable is swept back and forth in the same measurement
copy the one from instrumentserver?
Can we add an option to input extra info about the experiment somewhere on the plot?
Currently missing:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.