spacetelescope / costools Goto Github PK
View Code? Open in Web Editor NEWTools for HST/COS.
Home Page: http://costools.readthedocs.io/
License: BSD 3-Clause "New" or "Revised" License
Tools for HST/COS.
Home Page: http://costools.readthedocs.io/
License: BSD 3-Clause "New" or "Revised" License
Following the example posted here does not work as intended due to what I think is a problem between costools and astropy. Have a look at the following output from the python terminal:
Python 2.7.13 (default, Jan 19 2017, 14:48:08)
[GCC 6.3.0 20170118] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from costools import timefilter
The following tasks in the costools package can be run with TEAL:
splittag timefilter x1dcorr
>>> timefilter.TimelineFilter("lcw301b2q_corrtag_b.fits")
input = /home/paw/science/betapic/COS/data/2015_1/lcw301b2q_corrtag_b.fits
GTI: start stop
0.032 555.040
0.0 % of DQ column flagged with 2048
556 rows in TIMELINE
column beginning middle end
time 0.03 278.03 555.03
sun_alt 28.93 12.25 -4.28
target_alt 42.10 51.64 58.93
longitude 121.77 139.87 156.68
latitude -28.06 -25.08 -19.78
shift1 -510.47 -510.47 -510.47
column min max median
shift1 -510.47 -510.47 -510.47
Ly_alpha 1 373 237.5
darkrate 0 1.7952e-05 8.0782e-06
<costools.timefilter.TimelineFilter object at 0x7f1d207d3f90>
>>> timefilter.TimelineFilter("lcw301b2q_corrtag_b.fits", "new_file.fits", "Ly_alpha < 300.")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/costools/timefilter.py", line 435, in __init__
self.setDqFlag()
File "/usr/local/lib/python2.7/dist-packages/costools/timefilter.py", line 800, in setDqFlag
gti = self.recomputeExptime()
File "/usr/local/lib/python2.7/dist-packages/costools/timefilter.py", line 994, in recomputeExptime
self.fd[self.events_hdunum].header.update(exptime_key, exptime)
File "/usr/local/lib/python2.7/dist-packages/astropy/io/fits/header.py", line 1115, in update
'a keyword, value, and comment string.'.format(idx))
ValueError: Header update sequence item #0 is invalid; the item must either be a 2-tuple containing a keyword and value, or a 3-tuple containing a keyword, value, and comment string.
>>>
Anybody know what is wrong here?
When making final, public releases of this software please use the Github release feature (https://github.com/spacetelescope/costools/releases/new). This will trigger the build and upload of the corresponding package at the tag specified to PyPI.
Hi! I've used the costools.splittag.splittag
routine to extract the time-tag observations in the past. For a recent HST visit, it seems to fail for 1 out of the 5 orbits we got. I'm getting the following error:
~/Documents/GitHub/cos_flares/src/cos_reduction.py in split_corrtag(self, output_path, increment, starttime, endtime, time_list)
103 endtime=endtime, time_list=time_list)
104
--> 105 splittag.splittag(corrtag_b, os.path.join(output_path,
106 'split_{}'.format(self.rootname[i])),
107 increment=increment, starttime=starttime,
~/opt/anaconda3/lib/python3.9/site-packages/costools/splittag.py in splittag(infiles, outroot, starttime, increment, endtime, time_list, verbosity)
152 time_list = None
153
--> 154 SPLT.splittag(infiles=infiles, outroot=outroot,
155 starttime=starttime, increment=increment, endtime=endtime,
156 time_list=time_list, verbosity=verbosity)
~/opt/anaconda3/lib/python3.9/site-packages/calcos/splittag.py in splittag(infiles, outroot, starttime, increment, endtime, time_list, verbosity)
50 inlist = glob.glob(infiles)
51 for input in inlist:
---> 52 splitOneTag(input, outroot, starttime, increment, endtime,
53 time_list, verbosity)
54
~/opt/anaconda3/lib/python3.9/site-packages/calcos/splittag.py in splitOneTag(input, outroot, starttime, increment, endtime, time_list, verbosity)
133 ofd.append(out_gti_hdu)
134
--> 135 out_timeline_hdu = createNewTimeline(timeline_hdu, t0, t1)
136 if out_timeline_hdu is not None:
137 ofd.append(out_timeline_hdu)
~/opt/anaconda3/lib/python3.9/site-packages/calcos/splittag.py in createNewTimeline(timeline_hdu, t0, t1)
673 # of the input TIMELINE table.
674 # This implicitly assumes that the time increment is one second.
--> 675 (i_start, i_end)= ccos.range(time_col, t0, math.ceil(t1) + 0.1)
676 out_nrows = i_end - i_start
677 else:
RuntimeError: (30, 60.1) does not overlap the time array
Any insight into what's causing this bug would be greatly appreciated -- thanks!!
After install costools using pip3 (using Python 3.5.3) I get the following error when I try to import costools:
ImportError: No module named 'ccos'
ccos is called in the file lib/costools/timefilter.py at line 48.
Tyler Desjardins mentions that we should consider moving emails from help[at]stsci.edu
to point to the web portal where possible and appropriate. For HST (or any non-JWST), it is https://hsthelp.stsci.edu . For JWST, it is https://jwsthelp.stsci.edu . Please update info in setup.py
, setup.cfg
, documentation, etc as appropriate.
Please close this issue if it is irrelevant to your repository. This is an automated issue. If this is opened in error, please let pllim know!
No tests!
Python 2 will not be maintained past Jan 1, 2020 (see https://pythonclock.org/). Please remove all Python 2 compatibility and move this package to Python 3 only.
For conda recipe (including astroconda-contrib
), please include the following to prevent packaging it for Python 2 (https://conda.io/docs/user-guide/tasks/build-packages/define-metadata.html?preprocessing-selectors#skipping-builds):
build:
skip: true # [py2k]
Please close this issue if it is irrelevant to your repository. This is an automated issue. If this is opened in error, please let pllim know!
https://github.com/spacetelescope/stenv/actions/runs/8744041473/job/23996161017#step:11:71
_________________ ERROR collecting costools/tests/test_noop.py _________________
ImportError while importing test module '/home/runner/work/stenv/stenv/costools/costools/tests/test_noop.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../../micromamba/envs/stenv-Linux-py3.12/lib/python3.12/importlib/__init__.py:90: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
costools/__init__.py:3: in <module>
from pkg_resources import get_distribution, DistributionNotFound
E ModuleNotFoundError: No module named 'pkg_resources'
could this usage of pkg_resources
be replaced by importlib
?
When I tried to install costools
from source, I got this error:
Running calcos-3.0/setup.py -q bdist_egg --dist-dir /var/folders/jp/y94kfdg15xjgpplw8yrqt3jc0000gn/T/easy_install-3xpdprnf/calcos-3.0/egg-dist-tmp-fkcknzz_
error: Setup script exited with error: each element of 'ext_modules' option must be an Extension instance or 2-tuple
I am using Anaconda Python 3 on macOS. I managed to circumvent the error by installing calcos
from source before costools
.
I'm attempting to produce some sub-exposure COS x1d files and am hitting a TypeError when I try to run x1dcorr
. splittag
works fine and produces the relevant corrtag files. But trying to take one set of the subexposure corrtag files and process them into an x1d like so
from costools import x1dcorr
x1dcorr.x1dcorr(["test_1_corrtag_a.fits", "test_1_corrtag_b.fits"], outdir="test/", update_input=True, find=True, verbosity=2)
throws this
Input /Users/parke/Google Drive/Research/dimming/data/test_1_corrtag_a.fits
OutFlt test/test_1_flt_a.fits
OutCounts test/test_1_counts_a.fits
x1d test/test_1_x1d_a.fits
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-18-66f3f7e5ffe2> in <module>
----> 1 x1dcorr.x1dcorr(["test_1_corrtag_a.fits", "test_1_corrtag_b.fits"], outdir="test/", update_input=True, find=True, verbosity=2)
~/anaconda3/envs/astroconda/lib/python3.6/site-packages/costools/x1dcorr.py in x1dcorr(input, outdir, update_input, find, cutoff, location, extrsize, verbosity)
183 location=locn, extrsize=extrsz,
184 find_target={"flag": find, "cutoff": cutoff},
--> 185 verbosity=verbosity)
186
187
~/anaconda3/envs/astroconda/lib/python3.6/site-packages/calcos/x1d.py in extractSpec(inlist, outdir, update_input, location, extrsize, find_target, verbosity)
205 x1d_ab_list[i])
206 is_wavecal = makeFltCounts(cal_ver, corrtag_list[i],
--> 207 flt_list[i], counts_list[i])
208 extract.extract1D(flt_list[i], counts_list[i], x1d_ab_list[i],
209 location=location, extrsize=extrsize,
~/anaconda3/envs/astroconda/lib/python3.6/site-packages/calcos/x1d.py in makeFltCounts(cal_ver, corrtag, flt, counts)
446 reffiles["brftab"])
447 dq_array = timetag.doDqicorr(events, corrtag, info, switches, reffiles,
--> 448 phdr, headers[1], minmax_shift_dict)
449
450 timetag.writeImages(x, y, epsilon, dq,
TypeError: doDqicorr() missing 2 required positional arguments: 'traceprofile' and 'gti'
I think I will be able to run calcos instead as a work around, but it would be nice to be able to use x1dcorr directly in the future so that I don't have to worry about asn files and such.
When following the instructions posted in the costools documentation, I try to extract a 1D 1xdcorr spectrum from corrtag files but run into an error concerning a missing file. I am guessing I have to add this file and place it somewhere for the code to run. Any ideas?
Python 2.7.13 (default, Jan 19 2017, 14:48:08)
[GCC 6.3.0 20170118] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from costools import x1dcorr
The following tasks in the costools package can be run with TEAL:
splittag timefilter x1dcorr
>>> x1dcorr.x1dcorr("lc9e01iaq_corrtag_b.fits", outdir="sliced/", update_input=True, find=True, cutoff=5., verbosity=2)
Input /home/paw/science/betapic/COS/data/2014/lc9e01iaq_corrtag_b.fits
OutFlt sliced/lc9e01iaq_flt_b.fits
OutCounts sliced/lc9e01iaq_counts_b.fits
x1d sliced/lc9e01iaq_x1d_b.fits
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/costools/x1dcorr.py", line 185, in x1dcorr
verbosity=verbosity)
File "/usr/local/lib/python2.7/dist-packages/calcos/x1d.py", line 207, in extractSpec
flt_list[i], counts_list[i])
File "/usr/local/lib/python2.7/dist-packages/calcos/x1d.py", line 443, in makeFltCounts
timetag.setActiveArea(events, info, reffiles["brftab"])
File "/usr/local/lib/python2.7/dist-packages/calcos/timetag.py", line 368, in setActiveArea
cosutil.activeArea(info["segment"], brftab)
File "/usr/local/lib/python2.7/dist-packages/calcos/cosutil.py", line 975, in activeArea
brf_info = getTable(brftab, {"segment": segment}, exactly_one=True)
File "/usr/local/lib/python2.7/dist-packages/calcos/cosutil.py", line 452, in getTable
fd = fits.open(table, mode="copyonwrite")
File "/usr/local/lib/python2.7/dist-packages/astropy/io/fits/hdu/hdulist.py", line 160, in fitsopen
lazy_load_hdus, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/astropy/io/fits/hdu/hdulist.py", line 413, in fromfile
lazy_load_hdus=lazy_load_hdus, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/astropy/io/fits/hdu/hdulist.py", line 998, in _readfrom
fileobj = _File(fileobj, mode=mode, memmap=memmap, cache=cache)
File "/usr/local/lib/python2.7/dist-packages/astropy/utils/decorators.py", line 507, in wrapper
return function(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/astropy/io/fits/file.py", line 150, in __init__
self._open_filename(fileobj, mode, overwrite)
File "/usr/local/lib/python2.7/dist-packages/astropy/io/fits/file.py", line 506, in _open_filename
self._file = fileobj_open(self.name, IO_FITS_MODES[mode])
File "/usr/local/lib/python2.7/dist-packages/astropy/io/fits/util.py", line 395, in fileobj_open
return open(filename, mode)
IOError: [Errno 2] No such file or directory: '$lref/x1u1459il_brf.fits'
>>>
Hi @philhodge,
(cc @stsci-hack)
A quick scan of our entire codebase returned the usage of astropy.io.fits.new_table
in the costools.timefilter
module here:
costools/lib/costools/timefilter.py
Line 1083 in d8aefbf
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.