We need to improve the coverage of the project to at least to 80%.

Because Pareto. 😄

`
for img in imglist:

        newcat = img._best_srcs['sources']
        newcat = append_fields(newcat, 'sourceid',
                               np.repeat(0, len(newcat)),
                               usemask=False,
                               dtypes=int)

        ids, mask = matching(mastercat, newcat, masteridskey='sourceid',
                             angular=False, radius=1., masked=True)

        for i in range(len(ids[mask])):
            print ids[mask][i]
            newcat[mask]['sourceid'][i] = ids[mask][i]
            print newcat[mask]['sourceid'][i]

newcat[mask]['sourceid'] = np.array(ids[mask])

`

This is on lines 222 of properimage/utils.py

Trying to fill the values of newcat, but there is no traceback, it doesn't fail, but it is not working neither.

Something related to the creation of the field 'sourceid' must be wrong.

Any ideas?

Tests seems to fail randomly from time to time.

It may be a typo. Numpy.fft has irfft2 but not rifft2

>>> from properimage import single_image as s
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "properimage/single_image.py", line 74, in <module>
    _ifftwn = np.fft.rifft2
AttributeError: 'module' object has no attribute 'rifft2'

Hello,

Would you mind versioning commits? Especially those that break API. It'd be much appreciated if you could do so.

Thanks,
M.

Implement a default behavior to the difference with the __sub__

So instead of execute img_a.diff(img_b) you can run img_a - img_b

Remove prints or turn into logging calls.

Modules plot, utils, numpydb and simtools should be private (prefixed with underscore _) if they will not be part of the public API.

class Bunch is defined in single_image.py and single_image_psfs.py but never used anywhere else? I checked with a cat/grep.

We should delete that if it's really unused.

Installed fresh in a virtualenv, gives the following error when imported:

>>> import ProperImage
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named ProperImage
>>> import properimage
>>> from properimage import propersubtract as ps
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "properimage/propersubtract.py", line 30, in <module>
    from . import propercoadd as pc
  File "properimage/propercoadd.py", line 35, in <module>
    from . import utils
  File "properimage/utils.py", line 34, in <module>
    import matplotlib.pyplot as plt
ImportError: No module named matplotlib.pyplot

Is it necessary for us to align images? I think that should be left to the user to use whichever software is more convenient for the situation.
Same thing with cropping and removal of cosmic rays.
IMHO, it seems too specific and unrelated to proper subtraction/coaddition.

Create a .asarray method to convert an image to a basic np.ndarray

Consider renaming the functions with a full word (verb preferably) that describes what it does.
My suggestion is subtract and coadd or stack.

It's all BLK100.

Try this experimentally first

After you run
afields, psfbasis = img.get_varpsf(inf_loss)
if you re-run it with new inf_loss value it does nothing.
Which is a bug. This is supposed to work.

I think these lines should not be there? I never used ipdb, so not sure what they do.

https://github.com/toros-astro/ProperImage/blob/4689c8d8fb3f62dc02dcac609c702b4711c723d5/properimage/propersubtract.py#L105-L109

and

https://github.com/toros-astro/ProperImage/blob/4689c8d8fb3f62dc02dcac609c702b4711c723d5/properimage/propersubtract.py#L149-L152

Hi,

I noticed today a bunch of hidden dotted files:

._120634801176SingleImage.dat
._120634801176SingleImage.map
...

Could this be from properimage not cleaning it up?

If there's anything worth keeping in those two modules, we can move it to utils or single_image.
Transparency could be a function to single_image.

Thumbs up if agree, thumbs down if don't ;)

• Integrate black into tox with flake8-black
• Ad pyproject.toml to configure black

I think the method returning the ndarray pixels from SingleImage should be just data instead of pixeldata, since that seems to be the standard in masked numpy arrays, CCDData, NDData and HDUList.

Likewise it should have a property mask that returns the mask as a ndarray, but I think that is already implemented.

I don't know what you guys think.

Provide a accessor to the most common plot of the image through a plot accesor attribute.

So instead of run (img is a SingleImage instance)

plt.imshow(img)

we can use

img.plot() 
# or
img.plot.imshow()

Another method can be img.plot.background()

Function chunk_it defined in single_image is only used in propercoadd module and nowhere else.
We should move it there.

https://github.com/toros-astro/ProperImage/blob/abdce1b1ee2c7dbd07129115c77aa6f5d058d051/properimage/single_image.py#L1100

We had written code as -

import os
import builtins
import pickle
import sys
sys.tracebacklimit=0
import traceback
import io
from logging import Logger

safe_builtins = {
        'range',
        'complex',
        'set',
        'frozenset'
        }

class RestrictedUnpickler(pickle.Unpickler):
  def find_class(self, module, name):
  # Only allow safe classes from builtins.
   if module == "builtins" and name in safe_builtins:
       return getattr(builtins, name)
    # Forbid everything else.
   raise pickle.UnpicklingError("global '%s.%s' is forbidden" %
   (module, name))

def restricted_loads(s):
    """Helper function analogous to pickle.loads()."""
    return RestrictedUnpickler(io.BytesIO(s)).load()

def func1(a): 
  try: 
    x= restricted_loads(pickle.dumps(a))
    return x 
  except pickle.UnpicklingError : 
    s= pickle.UnpicklingError("unsupported persistent object") 
    return s

def func2(s):
  try:
    x=restricted_loads(pickle.dumps(s))
    return s[x]
  except pickle.UnpicklingError :
    s=traceback.format_exc()
    return s
    
if __name__ == "__main__":
       a=range(int(input())) 
       b=func1(a)
       print(b)
       y=tuple(input())
       z=func2(y)
       print(z)

Getting
TypeError: tuple indices must be integers or slices, not tuple

Input (stdin)

50 
"a", "b", "c", "d", "e", "f", "g", "h"

Your Output (stdout)
range(0, 50)

Expected Output

range(0, 50) 
Traceback (most recent call last): 
_pickle.UnpicklingError: global 'builtins.slice' is forbidden

I think we should reorder the documentation, focusing on the main functionality and leaving SingleImage further down the docs.

There is an implementation for a numpy array storage, that uses pickle.
This is critical so we don't keep in memory stamps of every star.
The current implementation though, requires the creation of files, which for many reasons can persist later as garbage in the local directory (see #11).

Using diskcache library could avoid such issue, and clean the base code of additional modules.

We need to define a better repr method

Integrate sphinx build into tox and travis

Could be a module operations, although the main functions that user will use should be imported at package level in __init__.

Remove psf stuff from SingleImage and move to file psf.py inside properimage package.

single_image_psfs inherits from single_image, so IMHO should be under the same module.
Also, reduces namespace noise.

If setting the stamp_shape parameter when making an instance of SingleImage, then set to never update it. Related to #8

There seems to be a number of issues that make properimage not work with astropy > 3.1.
I've set max version for astropy at 3.1 in setup.py

Add docstring to __init__.py. Basically the first paragraph of the docs.
Should I add @BrunoSanchez 's email?

Dear developer, thank you very much for this awesome project ! I started to read the mentionned paper (the 2002 one about spatially varying psf, and the newer ones about statistical hypothesis testing on differntial imaging).

It is really outstanding work ! Thank you very much for making it available here !

I am not done yet on reading the paper about the psf decomposition, but I was wondering if it was possible to get a reconstruction of the psf at various location of the image, such that one can then compute some quality metric maps across the image.

Which for instance would include:
-anisotropic gaussian matrix estimation
and from there:
-fwhm map
-ellipticity map

I was eager to see what will be in https://properimage.readthedocs.io/en/latest/topics/PSF.html
But there is nothing yet, what do you intend to put there ?

Use crossmatch from astropy to drop astroML dependency.

Used in utils.py

https://github.com/toros-astro/ProperImage/blob/f4e78943cc1306ec464c6cc8e688b57b55d9b64a/properimage/utils.py#L45

and in no other file.

photutils is marked as a dependency in setup.py but it's never imported.

The documentation says MIT license, but the LICENSE says BSD3

If I understand correctly, apparently since python 3.0 pickle/cpickle is just pickle.

A common pattern in Python 2.x is to have one version of a module implemented in pure Python, with an optional accelerated version implemented as a C extension; for example, pickle and cPickle. This places the burden of importing the accelerated version and falling back on the pure Python version on each user of these modules. In Python 3.0, the accelerated versions are considered implementation details of the pure Python versions. Users should always import the standard version, which attempts to import the accelerated version and falls back to the pure Python version. The pickle / cPickle pair received this treatment.

So these lines

Should be just

import pickle

I'll add something similar to what's on setup to grab the version number from __init__.py.

I have this dependency error right after installation in a clean virtualenv.

>>> from properimage import single_image as s
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "properimage/single_image.py", line 61, in <module>
    from astroscrappy import detect_cosmics
ImportError: No module named astroscrappy

I checked and it seems that astroscrappy is not listed in requirements.txt or setup.py

It seems like it was left behind by mistake.

https://github.com/toros-astro/ProperImage/blob/abdce1b1ee2c7dbd07129115c77aa6f5d058d051/properimage/propersubtract.py#L406-L413

I know this may have changed at some point, but I think that SingleImage's stamp_size should always keep stamps that size, or we should remove the ability of the user to specify it.

Otherwise it's an unexpected result if psf estimation is returned with a different size.

If we're dropping support for py2.7 (are we?) we can remove the dependency for six.
It's used only to import range in single_image and single_image_psfs.

• Extract properimage.test outside properimage source.
• Remove pytest as dependency.
• Rename outside test folder to draft.
• Add parallel into pytest