neutrons / pystog Goto Github PK

Need to include the ability for integrating the g(r) to get n(r). Would have multiple inputs for the real-space functions that convert to g(r) and then do calculation.

@tacaswell suggested this to me for doing image comparison in testing for matplotlib instead of just the current mock testing. Need to investigate:

https://github.com/matplotlib/pytest-mpl

To better improve the design and checks for the higher-level classes, specifically the Converter class, would like to explore the idea of a Function class that acts like a factory.

The idea of the class would be something like:

sq = Function(x, y, error=e, "S(Q)") # x,y,e are the Q, S(Q), and E[S(Q)], respectively
dcs = sq.to_DCS()
gr = sq.to_GofR(r) # r is the real space domain

Yet, the initial issue I see is that this requires both the capabilities of the Converter and Transformer into one class.

Python 2 is deprecated, remove support

Currently getting that we cannot import pystog from the binder built notebooks under the tutorials directory:

May want to look into repo2docker tool to test and understand why this is happening:
https://repo2docker.readthedocs.io/en/latest/getting-started/index.html

Steps to setup the CD:

• Setup the automatic versioning, via something like versioneer
• Use existing materials to setup the CD with Travis-CI here

Since Travis is dropping support, switching to Travis CI.

See it being similar to ADDIE's CI since we will be adding a conda package we want to upload once Issue #30 is closed (soon)
https://github.com/neutrons/addie/blob/master/.github/workflows/actions.yml

Will also want to deploy anaconda package to the neutrons anaconda channel:
https://anaconda.org/neutrons

This can be a problem in a back transform after fourier filtering. Introduces a high-freq that could mess with reciprocal space.

Related, look into "flat-lining" the fourier filtered bit

There is a numpy_store branch that was the initial work to drop Pandas dependency and just use numpy for the function store.

This will help with dependency reduction.

I used this script to compare the Fourier transform algorithm PDFFourierTransform of Mantid and pystog.

from mantid.simpleapi import CreateWorkspace, PDFFourierTransform
import numpy as np
from pystog import Transformer
transformer = Transformer()

xx = np.arange(0, 10, 0.1)
yy = np.exp(-(2.0 * xx)**2)
ws = CreateWorkspace(DataX=xx, DataY=yy, DataE=yy, UnitX='MomentumTransfer')
Rt = PDFFourierTransform(ws, InputSofQType='Q[S(Q)-1]', PDFType='G(r)')

y = Rt.extractY()[0]
x = Rt.extractX()[0]
e = Rt.extractE()[0]

r, gr, dgr = transformer.F_to_G(xx, yy, x, yy)

After c80d57b this shows that you get the same G(r) (comparing y and gr) and delta-G(r) (comparing e and dgr).

But if you change the input q to not start at 0 (xx=np.arange(0.1,10,0.1)) then the results differ.

Getting this error when I run the files attached below:

loading config from 'base_1_pystog.json'
Traceback (most recent call last):
  File "/home/ntm/.local/share/virtualenvs/pystog-_b47w3aQ/bin/pystog_cli", line 11, in <module>
    load_entry_point('pystog', 'console_scripts', 'pystog_cli')()
  File "/home/ntm/direnv/python3/pystog/pystog/cli.py", line 57, in pystog_cli
    stog._add_keen_fq(q, sq)
  File "/home/ntm/direnv/python3/pystog/pystog/stog.py", line 1238, in _add_keen_fq
    q, fq, self.df_sq_master, self.fq_title)
  File "/home/ntm/direnv/python3/pystog/pystog/stog.py", line 1436, in add_to_dataframe
    df = pd.concat([df, df_temp], axis=1)
  File "/home/ntm/.local/share/virtualenvs/pystog-_b47w3aQ/lib/python3.6/site-packages/pandas/core/reshape/concat.py", line 229, in concat
    return op.get_result()
  File "/home/ntm/.local/share/virtualenvs/pystog-_b47w3aQ/lib/python3.6/site-packages/pandas/core/reshape/concat.py", line 426, in get_result
    copy=self.copy)
  File "/home/ntm/.local/share/virtualenvs/pystog-_b47w3aQ/lib/python3.6/site-packages/pandas/core/internals/managers.py", line 2065, in concatenate_block_managers
    return BlockManager(blocks, axes)
  File "/home/ntm/.local/share/virtualenvs/pystog-_b47w3aQ/lib/python3.6/site-packages/pandas/core/internals/managers.py", line 114, in __init__
    self._verify_integrity()
  File "/home/ntm/.local/share/virtualenvs/pystog-_b47w3aQ/lib/python3.6/site-packages/pandas/core/internals/managers.py", line 311, in _verify_integrity
    construction_error(tot_items, block.shape[1:], self.axes)
  File "/home/ntm/.local/share/virtualenvs/pystog-_b47w3aQ/lib/python3.6/site-packages/pandas/core/internals/managers.py", line 1691, in construction_error
    passed, implied))
ValueError: Shape of passed values is (3526, 3), indices imply (3138, 3)

Per the original StoG, you could:

• enter rmax to "delete" / flat-line a region from 0 -> rmax
• you also entered a peak_min and peak_max to "save" the peak region so it did not get flat-lined

Would like to add this same functionality along with a different input mechanism:

• Enter a list of [(rmin, rmax)] pairs where you flat-line the regions between these.

Then, to replicate the old behavior, you would enter the following list:

[(0, `peak_min`), (`peak_max`, `rmax`)]

A positive to integrating PyStoG into Mantid Framework will be that Workspaces have the uncertainty and thus can be calculated during the transformations.

Yet, if given in the input, PyStoG should also include this capability. The source code for PDFFourierTransform algorithm should be the starting place for insight into the implementation.

The currently known "missing" real space functions that are needed, in order, are:

• D(r)
• T(r)
• rho(r)
• R(r)
• N(r)

Lots of function names and variables do not comply with PEP8 naming conventions.
Can use the PyCQA pep8-naming tool to find the offenders.
Once complete, can add to the linting stage of CI

Right now, we are just pushing the python 3.7 package version to anaconda:
https://anaconda.org/neutrons/pystog/files

This is because it is coupled to the PyPi build, which only needs one version of python to build against.
https://github.com/neutrons/pystog/blob/master/.github/workflows/cd.yml#L17-L40

We should probably instead create two separate CD yamls, one for PyPi (i.e. cd_pypi.yml) and one for Anaconda (i.e. anaconda.yml).
Then, we would have a matrix of OS that exist just like they do in the ci.yml to upload multiple python versions to Anaconda.

Add a conda recipe and upload to anaconda.
Would like to use the neutrons anaconda channel

Here is the rebin code I have that did this previously. Want it for smoothing via coarsening the binning:

#!/usr/bin/env python
from __future__ import (absolute_import, division, print_function)

import numpy as np
import argparse
import matplotlib.pyplot as plt

def rebin2_rmc(xin, yin, minx, xdiv, maxx):
    xout  = np.arange(minx, maxx+2.*xdiv, xdiv)
    yout  = np.zeros_like(xout)
    ynorm = np.zeros_like(xout)
    print('Number of points ', len(xout))

    for x, y in zip(xin, yin):
        if minx <= x and x<= maxx:
            idx = int((x-minx)/xdiv)
            scale1=1.-(x-xout[idx])/xdiv
            scale2=1.-scale1

            yout[idx]   = yout[idx]   + y*scale1
            yout[idx+1] = yout[idx+1] + y*scale2

            ynorm[idx]   = ynorm[idx]   + scale1
            ynorm[idx+1] = ynorm[idx+1] + scale2

    yout = yout / ynorm

    return xout, yout

https://docs.codecov.io/docs/about-the-codecov-bash-uploader

Similar to the other tutorials, need to include a FourierFilter class notebook. Refactor the class if needed.

Need to refactor the io away from the initial replication of stog into one that can accommodate many different inputs, instead of just the one current form used via RMCProfile.

Probably best way is to use what is already available in pandas