lightkurve / lightkurve Goto Github PK

When I tried to use this command:
tpf.plot(frame=970, aperture_mask=tpf.pipeline_mask,mask_color='red')

I see the following errror:

AttributeError: Unknown property mask_color

Would you please look at it.

I'm finding many of the tutorial plots do not appear on my system, probably due to a missing %matplotlib inline. When I run %matplotlib inline, the figures appear inline, as desired.

See title.

To study e.g. flare energies, users sometimes want to convert Kepler's relative photometry (electrons/s) into absolute fluxes. Although the literature on Kepler's zeropoint is limited, it would be useful to offer a function to convert fluxes into absolute units (e.g. Kp) using a best-available estimate of the zeropoint.

The GO site has historically listed a flux "zero"-point of 1.74 × 10^5 e-/s for a G2V-type star with Kp=12 (source: https://keplergo.arc.nasa.gov/CalibrationSN.shtml).

But it is available at MAST:

It would be useful for people to be able to see light curves with dates instead of MJD. We can use astropy.time.Time and datetime to create a tpf.time_readable or something similar. These objects also plot up nicely.

@dfm pointed out that better centroid estimates may be achieved by fitting a 2D polynomial to the central pixels of the stars, and returning the peak position.

The TargetPixelFile.centroids method could take a centroid-estimating method as a parameter.

We received the following bug report via e-mail:

===
Hello Kepler Science Time,

I am going through the 'Science with lightkurve' tutorial 'How to perform aperture photometry with custom apertures'. (http://lightkurve.keplerscience.org/tutorials/2.05-making-custom-apertures.html)

When I run the code in the tutorial in my notebook I am getting an error.

#Build the light curve
pipeline_lc = tpfs[0].to_lightcurve().flatten()
for tpf in tpfs:
    pipeline_lc = pipeline_lc.append(tpf.to_lightcurve().flatten())

#Clean the light curve
pipeline_lc = pipeline_lc.remove_nans().remove_outliers()

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-14-4ea5577ff332> in <module>()
      2 pipeline_lc = tpfs[0].to_lightcurve().flatten()
      3 for tpf in tpfs:
----> 4     pipeline_lc = pipeline_lc.append(tpf.to_lightcurve().flatten())
      5 
      6 #Clean the light curve

/Users/wesleysandidge/anaconda2/lib/python2.7/site-packages/lightkurve/lightcurve.pyc in flatten(self, window_length, polyorder, return_trend, break_tolerance, **kwargs)
    235             trend_signal[l:h] = signal.savgol_filter(x=lc_clean.flux[l:h],
    236                                                      window_length=window_length,
--> 237                                                      polyorder=polyorder, **kwargs)
    238         trend_signal = np.interp(self.time, lc_clean.time, trend_signal)
    239         flatten_lc = copy.deepcopy(self)

/Users/wesleysandidge/anaconda2/lib/python2.7/site-packages/scipy/signal/_savitzky_golay.pyc in savgol_filter(x, window_length, polyorder, deriv, delta, axis, mode, cval)
    335     if mode == "interp":
    336         if window_length > x.size:
--> 337             raise ValueError("If mode is 'interp', window_length must be less "
    338                              "than or equal to the size of x.")
    339 

ValueError: If mode is 'interp', window_length must be less than or equal to the size of x.

Is this a known error? What should I do to fix it?

Should this be accessible yet? i.e, should this work?

tpf = KeplerTargetPixelFile.from_archive(249914869, campaign=15)

I get

WARNING: NoResultsWarning: Query returned no results. [astroquery.mast.core]
---------------------------------------------------------------------------
ArchiveError                              Traceback (most recent call last)
<ipython-input-2-215e0a00620d> in <module>()
----> 1 tpf = KeplerTargetPixelFile.from_archive(249914869, campaign=15)

~/anaconda3/lib/python3.6/site-packages/lightkurve/targetpixelfile.py in from_archive(target, cadence, quarter, month, campaign, **kwargs)
    196                                "".format(len(products), target))
    197         elif len(products) < 1:
--> 198             raise ArchiveError("No Target Pixel File found for {} at MAST.".format(target))
    199         path = download_products(products)[0]
    200         return KeplerTargetPixelFile(path, **kwargs)

ArchiveError: No Target Pixel File found for 249914869 at MAST.

I want to use K2-33b for a tutorial but from_archive() can't find the EPIC ID: 205117205.

It can find it if I use the RA and Dec.

Some hlsps have not made it to MAST because it's a pain to put light curves into a fits file. If we can implement it it will lower the barrier for entry for the hlsps.

Currently you can't edit the flux of a tpf. It would be nice to have this feature so we can see the residuals by doing e.g.

(tpf - model).plot()

If scale="log" is used in tpf.plot(), we should let the ticks in the colorbar use log steps as well. Right now the colorback ticks look funny:

This isn't actually an issue. I just wanted to say congratulations, @barentsen!

https://www.sciencefriday.com/segments/after-finding-thousands-of-exoplanets-kepler-rides-into-the-sunset/

If you take the campaign specification out, the command works.

After installing lightkurve on Anaconda python 3.6 (pip install lightkurve) and downloading a TPF FITS file from MAST (https://archive.stsci.edu/pub/k2/target_pixel_files/c15/249900000/14000/ktwo249914869-c15_spd-targ.fits.gz), I am unable to open with lightkurve

tpf = KeplerTargetPixelFile.from_fits_images(['ktwo249914869-c15_spd-targ.fits'])
...
FileNotFoundError: [Errno 2] No such file or directory: '/Users/sl/anaconda3/lib/python3.6/site-packages/lightkurve/data/tpf-ext0-header.txt'

I tried downloading tpf-ext0-header.txt etc, from kadenza after which the above command works, but when I produce a lightcurve it is empty...

Currently the CBV corrector requires input light curves with the same number of data points as the CBV vectors. This is problematic as some of the light curve data is null and might be removed. We should change the CBV corrector so as to remove points from each vector that correspond to missing light curve data.

This would not necessarily replace k2-pix, because the latter provides a command-line interface etc... Instead, perhaps lightkurve could provide the very basic functionality which k2-pix imports.

See the screenshots below:

lc.stitch fails, it doesn't iterate through the list that is passed. I think there needs to be a check on the input for hasattr('__iter__') before we do anything with it.

I'm not sure how to fix this. The plot method has xlabel as a keyword, but the default is Tim - BJDREF. This is usually fine but if you've folded it this should be Phase.

The only solution I can think of is to have an attribute which describes the quantity in flux and time which would then be used as x and y labels in all the plots.

Possible small tweaks to improve SFFcorrector():

• Return the fitted long-term trend that is applied (currently a b-spline)
• Implement "Break point as a function of the campaign number"
• The bspline should take in raw time, not time relative to t0 (user may mask time later, losing reproducibility).
• s variable name, although faithful to the publication notation, causes some ambiguity with arclength() function

Over in #41 the following outstanding improvements to KeplerTargetPixelFile.from_fits_images() were identified:

• The WCS is not carried through yet.
• The performance is slow; using fitsio may be better.
• The unit test coverage is not complete.
• Support for flux errors is missing.

These would print valuable information. We think:

• ID
• Instrument
• Time of observation
• Flux 25th, median and 75th percentiles

We could also improve repr and str along these lines

Consider making a [clickable?] menu of all methods and attributes at the top of the page.
I was looking for wcs and it did not appear in the table at the top, so I gave up and had to ask Christina.

Currently KeplerCBVCorrector only accepts a LightCurveFile as an input. It should be able to work from a LightCurve generated from a tpf file e.g. tpf.to_lightcurve() still has a channel and a quarter.

This will mean people can use their models with lightkurve with no headaches.

The exception said scipy.interpolate.BSpline not found. Upgrading from scipy 0.17 to scipy 0.19 resolved the problem for the user.

I need this in my life.

I propose we add a Jupyter notebook widget to lightkurve that allows a Target Pixel File to be inspected in detail. The principal use case is to enable users to investigate why certain cadences appear as outliers in a lightcurve. @gully seems interested in working on this!

Proposed specifications:

• The widget shall allow the user to select a cadence using e.g. a scrollbar and an input box.
• For the selected cadence, the widget shall display the pixel data as an image.
• The widget shall also display the lightcurve and highlight the data point that corresponds to the user-select cadence.
• The widget shall display the list of quality flags set for the user-selected cadence (by calling KeplerQualityFlags.decode()).

Stretch goals (less important):

• The user shall be able to select to display FLUX, FLUX_BKG, RAW_CNTS (e.g. using a dropdown).
• The user shall be able to play the entire target pixel file as a movie using "play" and "pause" buttons.
• The user shall be able to interactively change the aperture mask to understand how it changes the lightcurve in real time.

@gully Feel free to think about and edit the specs! The best way to start may be to mock up a non-functional prototype interface that enables a discussion about the exact UI and features.

Snippets to recreate below. I ran into this problem with WASP-47, but not with HATS-11 (216414930).

lctpf = KeplerTargetPixelFile.from_archive("206103150", cadence="lc",
                                          quality_bitmask='hardest');
lc_m = lctpf.to_lightcurve(aperture_mask=lctpf.pipeline_mask)

lc = lc_m.remove_nans().remove_outliers()

#Remove long term trends
#lc = lc.flatten(window_length=401)
lc = lc.flatten()

ValueError                                Traceback (most recent call last)
<ipython-input-28-5b942d6160ae> in <module>()
      4 #Remove long term trends
      5 #lc = lc.flatten(window_length=401)
----> 6 lc = lc.flatten()
      7 
      8 #ax = lc_m[(lc_m.time>mystart) & (lc_m.time<myend)].plot(label='raw')

~/Documents/Documents/K2/learning_lightkurve/latest_lightkurve/lightkurve/lightkurve/lightcurve.py in flatten(self, window_length, polyorder, return_trend, break_tolerance, **kwargs)
    235             trend_signal[l:h] = signal.savgol_filter(x=lc_clean.flux[l:h],
    236                                                      window_length=window_length,
--> 237                                                      polyorder=polyorder, **kwargs)
    238         trend_signal = np.interp(self.time, lc_clean.time, trend_signal)
    239         flatten_lc = copy.deepcopy(self)

~/anaconda3/lib/python3.6/site-packages/scipy/signal/_savitzky_golay.py in savgol_filter(x, window_length, polyorder, deriv, delta, axis, mode, cval)
    335     if mode == "interp":
    336         if window_length > x.size:
--> 337             raise ValueError("If mode is 'interp', window_length must be less "
    338                              "than or equal to the size of x.")
    339 

ValueError: If mode is 'interp', window_length must be less than or equal to the size of x.

%debug

See #88.

To make sure non-Kepler or non-TESS users don't immediately switch off when they see the docs, let's add a tutorial that focuses on the generic functionality of the LightCurve class.

pathname = 'https://archive.stsci.edu/pub/k2/eng/long_cadence/kplr060021426-2014044044430_lpd-targ.fits'
tpf = KeplerTargetPixelFile(pathname)
lc = tpf.to_lightcurve(aperture_mask=tpf.pipeline_mask)

results in:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-37-8299474187b3> in <module>()
----> 1 lc = tpf.to_lightcurve(aperture_mask=tpf.pipeline_mask)

~/GitHub/lightkurve/lightkurve/targetpixelfile.py in to_lightcurve(self, aperture_mask)
    435                                 campaign=self.campaign,
    436                                 quarter=self.quarter,
--> 437                                 mission=self.mission,
    438                                 cadenceno=self.cadenceno)
    439 

~/GitHub/lightkurve/lightkurve/targetpixelfile.py in mission(self)
    368     def mission(self):
    369         """Mission name"""
--> 370         return self.header(ext=0)['MISSION']
    371 
    372     def to_fits(self):

~/anaconda3/lib/python3.6/site-packages/astropy/io/fits/header.py in __getitem__(self, key)
    143         else:
    144             keyword = key
--> 145         card = self._cards[self._cardindex(key)]
    146         if card.field_specifier is not None and keyword == card.rawkeyword:
    147             # This is RVKC; if only the top-level keyword was specified return

~/anaconda3/lib/python3.6/site-packages/astropy/io/fits/header.py in _cardindex(self, key)
   1652 
   1653         if not indices:
-> 1654             raise KeyError("Keyword {!r} not found.".format(keyword))
   1655 
   1656         try:

KeyError: "Keyword 'MISSION' not found."

I'm addicted to slicing and masking in LightCurve objects. Please can we have it in target pixel files too?

In PR #41 there is a problem with the number of time stamps not being returned. See below:

Not sure what's going on here, can't see why it would only return 4 of the time stamps requested.

We propose a long-awaited feature: the ability to inspect the target-pixel file (TPF) at a cadence-of-interest, identified interactively from inspection of the associated lightcurve.

The motivation for this feature arises in many scenarios: "Is this blip I see in my lightcurve astrophysical or some instrumental noise source?" may be the most common. Inspecting the cause of centroid shifts may be another niche, but powerful application-- did a supernova go off in the outskirts of your galaxy's target pixel postage stamp? If so, you may witness a spatially segregated temporal rise in flux that pulls the centroid location away from the galaxy center and towards the supernova-- a sure sign that you have discovered a genuine transient. A similar centroid-offset phenomenon can distinguish unresolved background eclipsing binaries from singleton transiting-planet-host stars. Finally, the effortless feedback of pixel data instills an intuitive feel for the connection of lightcurve and pixels, perhaps even more-so than the existing k2flix gif-animations, that are not seemlessly registered to their associated lightcurves.

The successful design of this feature hinges upon the TPF image being displayed effortlessly and instantly. That is: latency is near-zero---much less than human reaction time--- and no typing or clicking-through is needed. We also demand interactivity: a user may wish to zoom in on a region of interest while maintaining the linked, high-performance display.

These user experience objectives drove us to select a design that adopts the javascript framework bokeh. We investigated a matplotlib design, but we observed much higher latency, with less customization out-of-the-box.

The baseline design is simple. The setting is a Jupyter notebook. The user executes:

tpf.quick_look()

which yields a lightcurve plot that is wider than it is tall, adjacent to a nominally square image window for the TPF. A slider for cadence sits below the lightcurve window. The slider instantly updates a vertical bar on the lightcurve window, and updates the pixel image in the TPF window.

The design offers many avenues for additional functionality and custom behavior.

First, the baseline design automatically computes the default (raw, simple aperture photometery [SAP]) flux using a bare to_lightcurve() method. This SAP lightcurve may be adequate for some purposes, but likely the y-range will be overly large, and astrophysical signals of interest may be lost beneath the large motion-induced noise. Instead, the user may instruct the use of a pre-processed lightcurve that has already dropped outliers and/or applied motion-detrending and custom background subtraction:

lc_raw = tpf.to_lightcurve()
lc_cln = my_custom_cleaning_function(lc_raw)
# lc_cln may have fewer elements than lc_raw!
tpf.quick_look(lc=lc_cln)

In this case, the user will witness a nice, cleaned lightcurve with the raw pixel-level data on its right. Sliding the toolbar over missing cadences may simply jump over those cadences in the TPF, or show them anyway. We haven't decided what the default behavior should be, but probably the former?

Currently it does not and I want to correct my own light curve:

Currently, the remove_outliers() method in lightcurve does not support user-defined masks. With some discussion, Geert and I think that we could support direct masking, e.g.:

lc_masked = lc[mask]

where mask is a boolean mask. Geert points out that we'd have to implement dunder methods:
https://docs.python.org/3/reference/datamodel.html#object.__getitem__

A backup approach could be to support an input_mask kwarg in remove_outliers:

def remove_outliers(self, sigma=5., return_mask=False, input_mask=None, **kwargs):
        # [...]
        new_lc = copy.copy(self)
        if input_mask is not None:
            outlier_mask = input_mask
            #some checks to make sure the input_mask is the correct shape...
        else:
            outlier_mask = sigma_clip(data=new_lc.flux, sigma=sigma, **kwargs).mask
        new_lc.time = self.time[~outlier_mask]
        new_lc.flux = self.flux[~outlier_mask]

Suggestion from @jessie-dotson: let's add a 1-page cheat sheet to the docs that summarizes the key features?

Currently one must input detector column and row, which is proving problematic for stars in crowded superstamp regions (e.g., M67) that go across channel boundaries where the detector coordinates are discontinuous. It would be a lot easier to use the local coordinate system of the TPF to specify star positions (though one would still use the detector coordinates to retrieve the PRF model).

...and explain in the docs what the benefits are of contributing to lightkurve, which includes being added to the AUTHORS file etc.

https://github.com/KeplerGO/lightkurve/blob/master/lightkurve/utils.py#L187
https://github.com/KeplerGO/lightkurve/blob/master/lightkurve/utils.py#L257

Although our current logo is cool, I feel we need a less trembling one.

A KeplerLightCurve can get poscorr values either from a tpf or a lcf. We've been asked if we can have these carry over in a property.

Issue #83 reminded me that we do not appear to run the tutorial notebooks as part of Travis. We should investigate whether we can!

lc.correct(method='sff')

will break if there are NaNs in the lc object. lc.remove_nans() will not work, as it doesn't remove the nans from the centroids in the header.

We should have a tool that can cut out a region from an arbitrary list of standard FITS images and return a TPF. This would enable lightkurve to be used with data from any telescope, or e.g. the k2superstamp data set (@amcody may already have code to do this).

This function can likely use AstroPy's astropy.nddata.utils.Cutout2D function.

Example use:

images = os.listdir('k2superstamp/m67/*.fits')
tpf = TPFCreator(images=images, position=(x,y), size=10)

That's what I get with the current arguments: