openpiv / openpiv-python Goto Github PK

I am failing when trying to run this line of the tutorial:

u, v = openpiv.filters.replace_outliers( u, v, method='localmean', max_iter=10, kernel_size=2)

The traceback is:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-17-93f7bad645cf> in <module>()
     15 u, v, mask = openpiv.validation.sig2noise_val( u, v, sig2noise, threshold = 1.5 )
     16 
---> 17 u, v = openpiv.filters.replace_outliers( u, v, method='localmean', max_iter=10, kernel_size=2)
     18 # u, v = u.astype('float'), v.astype('float')
     19 

C:\Users\PrakashLab\Anaconda\lib\site-packages\openpiv-0.19-py2.7-win-amd64.egg\openpiv\filters.pyc in replace_outliers(u, v, method, max_iter, tol, kernel_size)
    120     """
    121 
--> 122     u = replace_nans( u, method=method, max_iter=max_iter, tol=tol, kernel_size=kernel_size )
    123     v = replace_nans( v, method=method, max_iter=max_iter, tol=tol, kernel_size=kernel_size )
    124 

openpiv\src\lib.pyx in openpiv.lib.replace_nans (openpiv/src\lib.c:1862)()

ValueError: Buffer dtype mismatch, expected 'DTYPEi_t' but got 'long long'

I am running Windows 8. I tried using ".astype" to cast u and v to float32, double, and every option I could think of, but none seemed to solve the issue. Thank you very much for any advice.

Hi guys,

thank you for OpenPIV it is a very useful piece of software!

I would like to process those images:
https://public.iscpif.fr/~reuillon/m1.jpg
https://public.iscpif.fr/~reuillon/m2.jpg

For that purpose I use this script (from the tutorial 1):
import openpiv.tools
import openpiv.process
import openpiv.scaling

frame_a = openpiv.tools.imread( 'm1.jpg' )
frame_b = openpiv.tools.imread( 'm2.jpg' )

u, v, sig2noise = openpiv.process.extended_search_area_piv( frame_a, frame_b, window_size=24, overlap=12, dt=0.02, search_area_size=64, sig2noise_method='peak2peak' )

x, y = openpiv.process.get_coordinates( image_size=frame_a.shape, window_size=24, overlap=12 )

u, v, mask = openpiv.validation.sig2noise_val( u, v, sig2noise, threshold = 1.3 )

u, v = openpiv.filters.replace_outliers( u, v, method='localmean', max_iter=10, kernel_size=2)

x, y, u, v = openpiv.scaling.uniform(x, y, u, v, scaling_factor = 96.52 )

openpiv.tools.save(x, y, u, v, mask, 'exp1_001.txt' )

openpiv.tools.display_vector_field('exp1_001.txt', scale=100, width=0.0025)

However I got this error message:
Traceback (most recent call last):
File "tutorial-part1.py", line 8, in
u, v, sig2noise = openpiv.process.extended_search_area_piv( frame_a, frame_b, window_size=24, overlap=12, dt=0.02, search_area_size=64, sig2noise_method='peak2peak' )
File "process.pyx", line 18, in openpiv.process.extended_search_area_piv (openpiv/src/process.c:1386)
ValueError: Buffer has wrong number of dimensions (expected 2, got 3)

I have also tried with B&W version of the images but I obtain the same error message.

Any idea what goes wrong?

Cheers,
Romain

Hi,

Installing openpiv on my Ubuntu 14.04 I got the following problem: the default encoding was not utf8. I fixed it with a sys.setdefaultencoding('utf8')

Also, filters has been deprecated in skimage, filter must be loaded in tools.py instead

Thanks to all the team for the hard work on openpiv.

Best,
JB

I think it's a serious bug or at least it is not explained well (or is it? where?) I was puzzled recently why all the quiver on image we show are opposite. and now it's clear that we deliberately violate the image coordinate system. So it's a sort of convention but why not to keep the image coordinate system, which means that 0,0 is the top , then if I want to know what is the vector at some precise pixel location, I don't need to guess why it's flipped but just to give that pixel location. wouldn't it be straightforward?

Thanks for the monumental effort in porting OpenPIV to [P|C]ython, Alex and company. Sadly, pip-based installation on Ubuntu Linux appears to be... problematic.

Specifically, the following Cython-related non-human-readable exception is raised:

$ pip2 install OpenPIV
Collecting OpenPIV
  Downloading OpenPIV-0.20.5.tar.gz (22.3MB)
    100% |████████████████████████████████| 22.3MB 76kB/s 
    Complete output from command python setup.py egg_info:
    Unable to find pgen, not compiling formal grammar.
    warning: no files found matching '*.pyx' under directory 'Cython/Debugger/Tests'
    warning: no files found matching '*.pxd' under directory 'Cython/Debugger/Tests'
    warning: no files found matching '*.h' under directory 'Cython/Debugger/Tests'
    warning: no files found matching '*.pxd' under directory 'Cython/Utility'
    /tmp/easy_install-AvP7gM/Cython-0.25.2/Cython/Plex/Scanners.c:4:20: fatal error: Python.h: No such file or directory
    compilation terminated.
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-HYAigT/OpenPIV/setup.py", line 15, in <module>
        Distribution(dict(setup_requires='Cython'))
      File "/usr/local/lib/python2.7/dist-packages/setuptools/dist.py", line 318, in __init__
        self.fetch_build_eggs(attrs['setup_requires'])
      File "/usr/local/lib/python2.7/dist-packages/setuptools/dist.py", line 375, in fetch_build_eggs
        replace_conflicting=True,
      File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 851, in resolve
        dist = best[req.key] = env.best_match(req, ws, installer)
      File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1123, in best_match
        return self.obtain(req, installer)
      File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1135, in obtain
        return installer(requirement)
      File "/usr/local/lib/python2.7/dist-packages/setuptools/dist.py", line 443, in fetch_build_egg
        return cmd.easy_install(req)
      File "/usr/local/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 673, in easy_install
        return self.install_item(spec, dist.location, tmpdir, deps)
      File "/usr/local/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 699, in install_item
        dists = self.install_eggs(spec, download, tmpdir)
      File "/usr/local/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 880, in install_eggs
        return self.build_and_install(setup_script, setup_base)
      File "/usr/local/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 1119, in build_and_install
        self.run_setup(setup_script, setup_base, args)
      File "/usr/local/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 1107, in run_setup
        raise DistutilsError("Setup script exited with %s" % (v.args[0],))
    distutils.errors.DistutilsError: Setup script exited with error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-HYAigT/OpenPIV/

The Unable to find pgen, not compiling formal grammar. line is, of course, dangerously suspicious. My intuitive reading of this exception is that the Python 2.7 version of Cython is either not currently installed on my system or is installed improperly. I assume the latter, as the equally suspicious line /tmp/easy_install-AvP7gM/Cython-0.25.2/Cython/Plex/Scanners.c:4:20: fatal error: Python.h: No such file or directory suggests that pip2 did at least attempt to install Cython into a temporary directory but failed, probably due to Python 2.7 development headers not having been installed.

A similar issue elsewhere suggests that manually installing Cython via apt-get on Debian-based Linux distributions (e.g., Ubuntu) is the ideal working solution:

sudo apt-get install cython

Let's give that a whirl.

https://github.com/OpenPIV/openpiv-python/blob/master/openpiv/tools.py#L98

Hi, its me again :-)
I am currently experimenting with the find_subpixel_peak_position function.
The problem of that function is if the find_first_peak() function returns a corner value of the correlation matrix then the sub-pixel peak position is set to (0,0) which in this sense is not correct at all.
I am proposing therefore for corner cases a change of the found peak value by 1 px to the inside so the calculation can be made.
I will try to request that change through github and push the changed pyprocess.py to the repository if you allow.

Cheers,
Ben

Another minor but (...hopefully) trivially fixable installation woe. If the Python 2.7 version of SciPy is not currently installed, OpenPIV installation superficially succeeds but ultimately fails on the first runtime importation of OpenPIV: e.g.,

>>> import openpiv.tools
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/openpiv/tools.py", line 28, in <module>
    import scipy.misc
ImportError: No module named scipy.misc

The underlying issue is that the install_requires keyword argument passed to the setuptools.setup() call in the setup.py script fails to specify SciPy as a runtime dependency. Ideally, this argument should probably resemble:

setup(
        .
        .
        .
        install_requires = ['scipy', 'numpy', 'cython'],
)

Easy come, easy go!

Hi,
running the tutorial part 1, I get the following error:

Traceback (most recent call last):
  File "/Users/tf/Downloads/firefox/tutorial-part1/tutorial-part1.py", line 8, in <module>
u, v, sig2noise = openpiv.process.extended_search_area_piv( frame_a, frame_b, window_size=24, overlap=12, dt=0.02, search_area_size=64, sig2noise_method='peak2peak' )
  File "process.pyx", line 23, in openpiv.process.extended_search_area_piv (openpiv/src/process.c:2392)
ValueError: Buffer dtype mismatch, expected 'DTYPEi_t' but got 'unsigned char'

Hi,

is it possible to read in im7 (LaVision) files?

Best,
Lukas

@TKaeufer - could you please add an example how to use windef when images are already loaded and not provided as file path in settings. e.g. sometimes I'd like to work with movies and I use pims.Video to read those - it's much more convenient that converting it to frames and storing as images.

Thanks
Alex

Hi,
I just installed openpiv for python in a linux environment. The installation went fine, but when trying to use openpiv, I got a problem with the following line in tools.py:

/.../openpiv/openpiv-python/openpiv/tools.py in ()
---> 33 from skimage import filters, io

Is it a typo? Should the line say the following instead(?):

from skimage import filter, io

Best Regards Trygve

I noticed another error while running tutorial_part3:
AttributeError Traceback (most recent call last)
in ()

AttributeError: module 'openpiv.pyprocess' has no attribute 'piv'

There is "extended_search_area_piv" in pyprocess:

I can modify this, for example:
u, v= openpiv.pyprocess.piv( frame_a, frame_b, corr_method='direct', window_size=24, overlap=12, dt=0.02, sig2noise_method='peak2peak' )
to
u, v, sig2noise= openpiv.pyprocess.extended_search_area_piv( frame_a, frame_b, corr_method='direct', window_size=24, overlap=12, dt=0.02, sig2noise_method='peak2peak' )

It seems to work.

read about fused types in cython for both 32 bit and 64 bit programming

the warning appears in multi-pass_tutorial

btw, the name shall be also fixed (mutli instead of multi)

The bot created this issue to inform you that pyup.io has been set up on this repo.
Once you have closed it, the bot will open pull requests for updates as soon as they are available.

Hi,

I donwloaded this sample: http://www.openpiv.net/openpiv-python/_downloads/tutorial-part13.zip

And I got this error:

 python2 tutorial-part1.py                                                                                                                        13:47
Traceback (most recent call last):
  File "tutorial-part1.py", line 1, in <module>
    import openpiv.tools
  File "/usr/lib/python2.7/site-packages/openpiv/__init__.py", line 21, in <module>
    import openpiv.process
  File "process.pyx", line 11, in init openpiv.process (openpiv/src/process.c:25074)
ImportError: No module named progressbar

Am trying to install with Python 2.7.11 (on a Mac OSX 10.11.6) and getting the following error during build:

clang: error: no such file or directory: 'openpiv/src/process.c'
clang: error: no input files
error: command '/usr/bin/clang' failed with exit status 1

Have been trying to work around this, but not making any progress. Any ideas? Thanks.

Python doesn't care for the way the validity of a value is being evaluated:

Traceback (most recent call last):
File "pypiv_test.py", line 43, in
u, v, mask = openpiv.validation.global_val( u, v, (-10, 200), (-30, 30) )
File "/usr/local/lib/python2.6/dist-packages/openpiv/validation.py", line 65, in global_val
ind = u < u_thresholds[0] or u > u_thresholds[1] or v < v_thresholds[0] or v > v_thresholds[1]
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

It seems that because u and v are arrays, it is not interested in evaluating it.

Is your feature request related to a problem? Please describe.
In some cases, the flow is primarily in the horizontal or vertical direction with a significantly smaller component in another direction. The best example is a laminar flow in the micro-channel - the flow is all horizontal ($u \gg v$). In such cases, square interrogation windows are less optimal and a rectangular interrogation window would be appropriate.

Describe the solution you'd like
See openpiv-matlab for the rectangular window solution https://github.com/OpenPIV/openpiv-matlab/blob/master/cross_correlate_rect.m

Describe alternatives you've considered
An alternative can be using windef with the largest size suitable for the horizontal component and the smallest size to the vertical component. With the deformation, the window might get to what we need. Also, use the overlap extensively.

Additional context
The discussion started at the openpiv_tk_gui repository

https://openpiv.readthedocs.io/en/latest/src/api_reference.html#the-openpiv-tools-module

The links to in detail API references are not active.

I'd expect there [2] instead of [1] for 3D case - right ? @AndreasBauerGit ?

https://gist.github.com/alexlib/5aa1e09750fcf6995a39943e4cc8c17a

In pyprocess.extended_search_area_piv() there are these lines that are redundant:

u = np.zeros((n_rows, n_cols))
v = np.zeros((n_rows, n_cols))
sig2noise = np.zeros((n_rows, n_cols)) 

u, v = np.zeros((n_rows, n_cols)), np.zeros((n_rows, n_cols))
# if we want sig2noise information, allocate memory
if sig2noise_method is not None:
    sig2noise = np.zeros((n_rows, n_cols))

After downloading and installing. I tried the tutorial and received the following error:

--------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-8-0211776e7460> in <module>()
      7 
      8 
----> 9 u, v, sig2noise = openpiv.process.extended_search_area_piv( frame_a, frame_b, window_size=24, overlap=12, dt=0.02, search_area_size=64, sig2noise_method='peak2peak' )
     10 
     11 

/usr/local/lib/python2.7/dist-packages/openpiv/process.so in openpiv.process.extended_search_area_piv (openpiv/src/process.c:2390)()

ValueError: Buffer dtype mismatch, expected 'DTYPEi_t' but got 'unsigned char'

OS: Linux Mint 17 Qiana

The jupyter notebook in the multi-pass_tutorial is no longer working.

1. Some missing imports
2. The PIV module no longer exists
3. There is a bug in WiDIM that I cannot seem to figure out (IndexError: only integers, slices (:), ellipsis (...), numpy.newaxis (None) and integer or boolean arrays are valid indices)

I would be happy to update the tutorial once I know how to fix the issue with WiDIM.

Aaaaaand... here we go again.

The openpiv.tools submodule imports skimage, the top-most package for the external scikit-image project. That project is thus a hard dependency of OpenPIV and should be listed as such in this project's setup.py script.

Since OpenPIV requires the skimage.filters submodule first introduced with scikit-image 0.12.0 (...I think?), it would probably be helpful to constrain this dependency in a version-specific manner: e.g.,

install_requires = ['scipy','numpy','cython','scikit-image >= 0.12.0'],

And thanks for your continual efforts on the Python front. Python is really the scientific future. It's great to see OpenPIV adopting and embracing the new standard! 🥇

in the tools.py, the import
from builtins import range
resulted in an exception. not sure if this is needed. After commnting out, the tutorial-part1 notebook worked fine.
(using python2.7. Installed through the anaconda)

Alex,

I would like to know what is your plan regarding the OpenPiv website.

I think that for the first release the web site at openpiv.net should be significantly updated in order for people to know what is the current progress of the project, what is the state of its three components, ... Furthermore a clear link to http://alexlib.github.com/openpiv-python/ should be made so that people and developers can be directed to the documentation we are writing.

One of the things that i find difficult at the beginning was in fact to know what was the state of the project. We should make people understand that we are working behind the scenes for a great release and, to this end, the website should be more clear and concise.

A further note. What do you think about the role of the page at http://alexlib.github.com/openpiv-python/ ? should it be only a documentation page for the python version or should we built a full website using sphinx for openpiv and move it to openpiv.net?
I don't know how did you develop the current site at openpiv.net, but sphinx is very capable and can do nice things, (but of course requires work.)

Final note: can you modify my name to Davide Lasagna on openpiv.net?

Best,

Davide

Dear @alexlib,

I evaluated images using

• process.extended_search_area_piv, and
• process.WiDIM

Compared to the extended_search_area_piv, the widim result is biased to positive values. This is visible in the vector plots of the same Poiseuille-Flow:
https://github.com/OpenPIV/openpiv_tk_gui/blob/master/tst_widim/vec_extd.png
https://github.com/OpenPIV/openpiv_tk_gui/blob/master/tst_widim/vec_widim.png
As well as from the plotted velocity profiles:
https://github.com/OpenPIV/openpiv_tk_gui/blob/master/tst_widim/profile_extd.png
https://github.com/OpenPIV/openpiv_tk_gui/blob/master/tst_widim/profile_widim.png

All files necessary to reproduce the result are collected here:
https://github.com/OpenPIV/openpiv_tk_gui/tree/master/tst_widim

I used the following parameters:
a) WiDIM
overlap: 16
corr_window: 32
dt: 1.0
subpixel_method: gaussian
sig2noise_method: peak2peak
evaluation_method: widim
coarse_factor: 2

b) Extended Search Area
evaluation_method: extd_search_area
overlap: 16
corr_window: 32
dt: 1.0
subpixel_method: gaussian
sig2noise_method: peak2peak
search_area: 64

I would not expect exactly the same result, but a more random distribution of the error.

I am on Ubuntu Linux 18.04 using the OpenPivTkGui, Version 0.1.14.

I think Numpy community invested a lot of time in properly working masking arrays - it's just a view (memory efficient thing) ot the same array with additional mark of what is 'masked' (as nan) and what is not. one can also use np.nan as athe 'fill' value that fills the masked regions, but it's irrelevant. The question here is that if it's possible to use masked arrays through C interface if we need it later for speed.

The tutorial describes how to cross-correlate only 1 image pair. Is there a function that will allow processing let's say 500 image pairs?

I was testing this library with 2 sample images where I created some dots moving towards the centre of the image. I noticed that the sign of the V vectors was incorrect, and noticed the minus sign in the code for extended_search_area_piv() .

v[I,J] = -( (i_peak - corr.shape[0]/2) - (search_area_size-window_size)/2 ) / dt

Is there a reason why the minus sign is there?

Best

Hi,
I am very new to PIV and coding in general so please bear with me. I have a dataset that is a video taken by a single camera under constant led illumination. I have broken them down into frames (dt=.06) and am trying to generate the velocity vectors, so far unsuccessfully. (the vectors don't seem to represent fluid motion in the actual video).
video:

Code:
piv initial2.txt

Is there any way to save the displayed vector fields? Also, for outliers and sig2noise methods, are there any other options than local mean and peak2peak?

Thank you in advance

Please. 😄

I noticed there may be a bug in preprocess.py.

In the code, it is like this:
imcopy = np.copy(image)
# stretch the histogram
image = exposure.rescale_intensity(img_as_float(image), in_range=(0, 1))

However, this causes this error while running masking_tutorial:
78 bw = (blur_edges > threshold)
79 bw = binary_fill_holes(bw)
---> 80 imcopy -= blurback
81 imcopy[bw] = 0.0
82 elif method is 'intensity':

TypeError: Cannot cast ufunc subtract output from dtype('float64') to dtype('uint8') with casting rule 'same_kind'

I can fix this by:
#imcopy = np.copy(image)
# stretch the histogram
image = exposure.rescale_intensity(img_as_float(image), in_range=(0, 1))
imcopy = np.copy(image)

Describe the bug
get_field_shape is a function that returns the number of rows and columns in the vector field. it is different in process.pyx and pyprocess.py and it is incompatible if one mixes the two cases. process.pyx should be a faster option, but this is a very fast NumPy function so there is no need for Cython speed-up here. It's also used only

To Reproduce
Steps to reproduce the behaviour:
Run

from openpiv import process, pyprocess
process.get_field_shape??
pyprocess.get_field_shape??

Expected behavior
Remove the process.pyx get_field_shape and use pyprocess.py only is impossible as process.pyx uses this function also in the extended_search_piv. Some decision is needed whether the field shape should take into account the search_area_size or not.

I'm trying to use process.WiDIM for some multi-pass PIV, but I'm unclear how to change some parameters like the sign2noise threshold in particular. Looking in process.c I found: "...the sig2noise for which the threshold is 1.5" but does that mean I can't adjust it?
Thanks for the help!

Travis unfortunately cannot cover all use cases.

I realized image pairs analyzed with versions 0.21.8 and lower give different results than versions 0.22.0 ad 0.22.2. The results I am getting from versions 0.22.0 and later are physically not acceptable, while the earlier ones are all correct. It seems that the changes to replace_nans cause this issue.

When trying to reproduce tutorials, I get an error with tools.display_vector_field when I swap 'on_img = True'. Attached is the script that I am working from. Below is the error.

testing.zip

Traceback (most recent call last):
File "test.py", line 26, in
tools.display_vector_field('test1.vec', on_img = True, scale=75, width=0.0035)
File "/home/user/miniconda3/lib/python3.7/site-packages/openpiv/tools.py", line 78, in display_vector_field
im = imread(image_name)
File "/home/user/miniconda3/lib/python3.7/site-packages/openpiv/tools.py", line 119, in imread
im = _imread(filename)
File "/home/user/miniconda3/lib/python3.7/site-packages/matplotlib/image.py", line 1417, in imread
with Image.open(fname) as image:
File "/home/user/miniconda3/lib/python3.7/site-packages/PIL/Image.py", line 2609, in open
fp = builtins.open(filename, "rb")
FileNotFoundError: [Errno 2] No such file or directory: 'None'

In [40]: a = np.random.randint(0, 2**8, (32,32) ).astype(np.int32)

In [41]: b = np.random.randint(0, 2**8, (32,32) ).astype(np.int32)

In [42]: corr_np =  np.fft.fftshift( np.fft.irfft2( np.fft.rfft2(a, s=(64,64) )*np.conj( np.fft.rfft2(b, s=(64,64)) )) .real, axes=(0,1) )

In [43]: import scipy.fftpack as f

In [44]: corr_scipy =  f.fftshift( f.ifft2( f.fft2(a, shape=(64,64) )*np.conj( f.fft2(b, shape=(64,64)) )) .real, axes=(0,1) )

In [45]: timeit f.fftshift( f.ifft2( f.fft2(a, shape=(64,64) )*np.conj( f.fft2(b, shape=(64,64)) )) .real, axes=(0,1) )
1000 loops, best of 3: 726 us per loop

In [46]: timeit  np.fft.fftshift( np.fft.irfft2( np.fft.rfft2(a, s=(64,64) )*np.conj( np.fft.rfft2(b, s=(64,64)) )) .real, axes=(0,1) )
1000 loops, best of 3: 453 us per loop

Can you test this on your machine as well? should we change back to numpy.fft?

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

It seems the MANIFEST file is committed. However that actually gets regenerated everytime a package is built. What is the reason for including this file? Can it be dropped?

gurney-flap is very noisy and needs both validation and filtering. let's use it as a demo/tutorial of the new algorithms.

Currently working with the code for another project and saw that you have a function moving_window_array in the pyprocess.py which is unused.
Do you plan with it in the future?