Giter VIP home page Giter VIP logo

asilib's Introduction

Test python package DOI

asilib

asilib is an open source package providing data access and analysis tools for the world's all-sky imager (ASI) data.

Note

The asilib code on PyPI moved from aurora-asi-lib to asilib. To update to version >=0.22.0, run

  1. python3 -m pip uninstall aurora-asi-lib and
  2. python3 -m pip install asilib.

The purpose of this project is to combine data from numerous observational ASI arrays into a single unified framework and is thus not associated with the development and operations of all sky cameras, or the curation of ASI datasets. All data is publicly available and is provided as-is. Please give appropriate credit and coordinate with instrument teams with regards to data issues and/or interpretation. See the Acknowledgements section for more information. Currently the supported camera systems (arrays) are:

Documentation | Code on GitHub | PyPI archive | Zenodo archive

A geographic map showing the spatial coverage (field of view rings) of all imagers supported by asilib.

An asilib collage showing fisheye images, mapped images, and a keogram from the THEMIS and REGO imagers at RANK.

And you can animate images & conjunctions!

See more examples in the online documentation

Your contributions and ideas are always welcome. The easiest way is to submit Issues or Pull Requests.

Acknowledgments

Are in the Acknowledgements section of the online documentation.

asilib's People

Contributors

cassandraauri avatar mshumko avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

asilib's Issues

BUG: Runtime error when using animate_fisheye()

Describe the issue:

When using this simpe code snippet where rego is from \asilib\asi\rego.py you get a runetime error from anaconda multiprocessing

RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

Reproduce the code example:

import asilib

import rego  # just copy of rego.py
from datetime import datetime, timedelta
time_range = (datetime(2021, 3, 18, 8, 0),
              datetime(2021, 3, 18, 8, 30))
asi = rego.rego('GILL', time_range=time_range)
# And run either one of these two commands
asi.animate_fisheye()  # animate fisheye images

Error message:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\1101w\anaconda3\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\1101w\anaconda3\lib\multiprocessing\spawn.py", line 125, in _main
    prepare(preparation_data)
  File "C:\Users\1101w\anaconda3\lib\multiprocessing\spawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Users\1101w\anaconda3\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
  File "C:\Users\1101w\anaconda3\lib\runpy.py", line 265, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "C:\Users\1101w\anaconda3\lib\runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "C:\Users\1101w\anaconda3\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "c:\Users\1101w\Clone\Programming-8\Summer 2023\helptest.py", line 9, in <module>
    asi.animate_fisheye()  # animate fisheye images
  File "C:\Users\1101w\anaconda3\lib\site-packages\asilib\imager.py", line 251, in animate_fisheye
    for _ in movie_generator:
  File "C:\Users\1101w\anaconda3\lib\site-packages\asilib\imager.py", line 382, in animate_fisheye_gen
    for i, (image_time, image) in _progressbar:
  File "C:\Users\1101w\anaconda3\lib\site-packages\asilib\utils.py", line 134, in progressbar
    for i, item in enumerate(iterator):
  File "C:\Users\1101w\anaconda3\lib\site-packages\asilib\imager.py", line 1232, in __iter__
    for file_times, file_images in self.iter_files():
  File "C:\Users\1101w\anaconda3\lib\site-packages\asilib\imager.py", line 1276, in iter_files
    times, images = self._data['loader'](path)
  File "c:\Users\1101w\Clone\Programming-8\Summer 2023\regotest.py", line 326, in _load_rego_pgm
    images, meta, problematic_file_list = rego_imager_readfile.read(str(path))
  File "C:\Users\1101w\anaconda3\lib\site-packages\rego_imager_readfile\_rego.py", line 35, in read
    pool = Pool(processes=workers)
  File "C:\Users\1101w\anaconda3\lib\multiprocessing\context.py", line 119, in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild,
  File "C:\Users\1101w\anaconda3\lib\multiprocessing\pool.py", line 212, in __init__
    self._repopulate_pool()
  File "C:\Users\1101w\anaconda3\lib\multiprocessing\pool.py", line 303, in _repopulate_pool
    return self._repopulate_pool_static(self._ctx, self.Process,
  File "C:\Users\1101w\anaconda3\lib\multiprocessing\pool.py", line 326, in _repopulate_pool_static
    w.start()
  File "C:\Users\1101w\anaconda3\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "C:\Users\1101w\anaconda3\lib\multiprocessing\context.py", line 326, in _Popen
    return Popen(process_obj)
  File "C:\Users\1101w\anaconda3\lib\multiprocessing\popen_spawn_win32.py", line 45, in __init__
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "C:\Users\1101w\anaconda3\lib\multiprocessing\spawn.py", line 154, in get_preparation_data
    _check_not_importing_main()
  File "C:\Users\1101w\anaconda3\lib\multiprocessing\spawn.py", line 134, in _check_not_importing_main
    raise RuntimeError('''
RuntimeError:
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.

Runtime information:

0.14.3
3.8.3 (default, Jul 2 2020, 17:30:36) [MSC v.1916 64 bit (AMD64)]
Windows-10-10.0.19041-SP0

Context for the issue:

When testing TREX I was trying to figure out how imager.py was fetching the infromation (as it couldnt find the TREx file) fixing this would help me implement TREx into imager.py and then eventually a way to figure out a way to implement TREx into asilib.animate_fisheye_generator()

BUG: error in time slicing in the get_points() function

Describe the issue:

I'm trying to get the pixel intensities at specific (lat,lon) points across a time range from a single THEMIS ASI imager. Whether using either a time range or a single time in the aurora.asi.themis() function, I get an error in the get_points() function stating that the time slicing cannot be 0. I do NOT have an error using a time or time range in the aurora.asi.themis() function when plotting a keogram, only when trying to get the pixel intensities.

Reproduce the code example:

import asilib
import asilib.asi

location_code = 'FYKN'
map_alt_km = 150
min_elevation = 2
time = datetime.datetime(2012,2,15,6,0,0)
time_range = [datetime.datetime(2012,2,15,6,0,0),datetime.datetime(2012,2,15,12,0,0)]

_imagers = asilib.asi.themis(location_code,time=time,alt=map_alt_km)
asis = asilib.Imagers(_imagers)

lat_lon_points,intensities = asis.get_points(min_elevation=min_elevation)
print(lat_lon_points)
print(intensities)
print(lat_lon_points.shape)

Error message:

Traceback (most recent call last):
  File "/Users/kat/Documents/UAH_Research/python_summary/asi_data.py", line 34, in <module>
    lat_lon_points,intensities = asis.get_points(min_elevation=min_elevation)
  File "/Users/kat/miniconda3/lib/python3.10/site-packages/asilib/imagers.py", line 217, in get_points
    if len(self.imagers[0].meta['resolution']) == 3: # RGB
  File "/Users/kat/miniconda3/lib/python3.10/site-packages/asilib/imager.py", line 1122, in __getitem__
    start_time, end_time = self._convert_slice(_slice)
  File "/Users/kat/miniconda3/lib/python3.10/site-packages/asilib/imager.py", line 1198, in _convert_slice
    raise ValueError(f'The slice must be [time] or [start_time:end_time], not {_slice}.')
ValueError: The slice must be [time] or [start_time:end_time], not 0.

Runtime information:

0.20.4
3.10.10 (main, Mar 21 2023, 13:41:39) [Clang 14.0.6 ]
macOS-10.15.7-x86_64-i386-64bit

Context for the issue:

No response

help

Hello, what version of python does this use? 3.7.0?

Scatterplotting on axes after calling from animate_map_gen()

            >>> from datetime import datetime
            >>> import matplotlib.pyplot as plt
            >>> import asilib
            >>> import asilib.asi
            >>>
            >>> location = 'FSMI'
            >>> time_range = (datetime(2015, 3, 26, 6, 7), datetime(2015, 3, 26, 6, 12))
            >>> asi = asilib.asi.themis(location, time_range=time_range)
            >>> ax = asilib.map.create_map(lon_bounds=(-120, -100), lat_bounds=(55, 65))
            >>> plt.tight_layout()
            >>>
            >>> gen = asi.animate_map_gen(overwrite=True, ax=ax)
            >>>
            >>> for image_time, image, ax, im in gen:
            >>>     # Add your code that modifies each image here...
            >>>     # To demonstrate, lets annotate each frame with the timestamp.
            >>>     # We will need to delete the prior text object, otherwise the current one
            >>>     # will overplot on the prior one---clean up after yourself!
            >>>     if 'text_obj' in locals():
            >>>             ax.texts.remove(text_obj)
            >>>     text_obj = ax.text(0, 0.9, f'THEMIS-{location} at {image_time:%F %T}',
            >>>             transform=ax.transAxes, color='white', fontsize=15)
            >>>
            >>> print(f'Animation saved in {asilib.config["ASI_DATA_DIR"] / "animations" / asi.animation_name}')

Calling animate_map_gen() returns 4 objects: a datetime.datetime of the current image, an ndarray corresponding to the image pixels, an plt.Axes object corresponding to the axis where the image is plotted, and a plt.Imshow object to add colorbars and so on.

Calling an extra scatterplot, like ax.scatterplot(x,y,c=z) where x and y are array-like objects of longitudes and latitudes and z is a quantity I want to plot, results in a singular spot being plotted on the image, even when x,y,z have more than one element. Why is this happening and is there a workaround?

To reproduce this issue, you can run the following code:

from datetime import datetime
import matplotlib.pyplot as plt
import asilib
import asilib.asi

# Sample Data
latitudes = [60, 61, 62]  
longitudes = [-110, -111, -112]  
values = [10, 20, 30] 

# Configuration for the map
location = 'FSMI'
time_range = (datetime(2015, 3, 26, 6, 7), datetime(2015, 3, 26, 6, 12))
asi = asilib.asi.themis(location, time_range=time_range)
ax = asilib.map.create_map(lon_bounds=(-120, -100), lat_bounds=(55, 65))
plt.tight_layout()

# Generate the animation frames
gen = asi.animate_map_gen(overwrite=True, ax=ax)

text_obj = None  # Initialize text_obj outside the loop
for image_time, image, ax, im in gen:
    # Add scatter plot on each frame
    scatter_plot = ax.scatter(longitudes, latitudes, c=values, cmap='viridis', s=100) 

    # Example of adding time-stamped annotations
    if text_obj:
        text_obj.remove()  # Remove the previous text object from the axes
    text_obj = ax.text(0, 0.9, f'THEMIS-{location} at {image_time:%F %T}',
                       transform=ax.transAxes, color='white', fontsize=15)

# Save the final animation
print(f'Animation saved in {asilib.config["ASI_DATA_DIR"] / "animations" / asi.animation_name}')

and the result is
Screenshot 2024-04-12 at 5 29 04 PM

Bug or misunderstanding of Conjunction.lla_footprint()

Describe the issue:

I am following the up to date Conjunction example. That example works with a toy satellite but using empheramis data, I am unsure of how to translate the altitude of the satellite to the footprint altitude.

I have initialized the Conjunction class with conj=asilib.Conjunction(asi, (time, (lat_sat,lon_sat,alt_sat)) then using
conj.lla_footprint(alt=110) which returns None and messes up map_azel( see error)

In conjunction_py the comments are out of date so I am unsure what the proper return should be and am quite shocked to see its just one Non. instead an array of length alt_sat of Nones.

Reproduce the code example:

from datetime import datetime, timedelta
import string
import matplotlib.pyplot as plt
import matplotlib.dates
import matplotlib.patches
import matplotlib.gridspec
import numpy as np
import pyaurorax.ephemeris as ae
import asilib
import asilib.asi
time_range = (datetime(2021, 3, 18, 8, 10),
              datetime(2021, 3, 18, 8, 30))

area_box_km = (20, 20)
#skymap_dict = asilib.load_skymap(asi_array_code, location_code, time_range[0])
print(asilib.config['ASI_DATA_DIR'])
global platforms
platforms = ["swarma"]
sky_map_values = ['trex_nir', 'rabb']  
rearth = 6378.1370


def emph():
    print(platforms, "test")

    def coordinates_extrapolation(array):
        n = int((time_range[1] - time_range[0]
                 ).total_seconds() / 3)  # 3 second␣
        latvelocitytotal, lonvelocitytotal = [], []
        cadence = int(60/3)
        # every 3 seconds theres a time component
        time = np.array([time_range[0] + timedelta(seconds=i*3)
                         for i in range(n)])
        for i in range(len(platforms)):
            latvelocity = []
            lonvelocity = []
            for j in range(len(array[0][0])-1):
                latvelocity.append(np.linspace(
                    array[0][i][j], array[0][i][j+1], cadence))
                lonvelocity.append(np.linspace(
                    array[1][i][j], array[1][i][j+1], cadence))
            latvelocitytotal.append(np.reshape(latvelocity, -1))
            lonvelocitytotal.append(np.reshape(lonvelocity, -1))
        print("ree")

        def altitude():
            altitude_total = []
            altepop = []
            altswarma = []
            altswarmb = []
            altvelocityepop = []
            altvelocityswarma = []
            altvelocityswarmb = []
            altswarmc = []
            altvelocityswarmc = []
            for i in range(len(platforms)):

                if(platforms[i] == "epop"):
                    for k in range(len(em1.data)):
                        if(em1.data[k].data_source.platform == "epop"):
                            altepop.append(
                                em1.data[k].metadata["radial_distance"]-rearth)
                    for j in range(len(altepop)-1):
                        altvelocityepop.append(np.linspace(
                            altepop[j], altepop[j+1], 20))
                    altvelocityepop = np.reshape(altvelocityepop, -1)
                    altitude_total.append(altvelocityepop)

                elif(platforms[i] == "swarma"):
                    for k in range(len(em1.data)):
                        if(em1.data[k].data_source.platform == "swarma"):
                            altswarma.append(
                                em1.data[k].metadata["radial_distance"]-rearth)
                    for j in range(len(altswarma)-1):
                        altvelocityswarma.append(np.linspace(
                            altswarma[j], altswarma[j+1], 20))
                    altvelocityswarma = np.reshape(altvelocityswarma, -1)
                    altitude_total.append(altvelocityswarma)

                elif(platforms[i] == "swarmb"):
                    for k in range(len(em1.data)):
                        if(em1.data[k].data_source.platform == "swarmb"):
                            altswarmb.append(
                                em1.data[k].metadata["radial_distance"]-rearth)
                    for j in range(len(altswarmb)-1):
                        altvelocityswarmb.append(np.linspace(
                            altswarmb[j], altswarmb[j+1], 20))
                    altvelocityswarmb = np.reshape(altvelocityswarmb, -1)
                    altitude_total.append(altvelocityswarmb)

                elif(platforms[i] == 'swarmc'):
                    for k in range(len(em1.data)):
                        if(em1.data[k].data_source.platform == "swarmc"):
                            altswarmc.append(
                                em1.data[k].metadata["radial_distance"]-rearth)
                    for j in range(len(altswarmc)-1):
                        altvelocityswarmc.append(np.linspace(
                            altswarmc[j], altswarmc[j+1], 20))
                    altvelocityswarmc = np.reshape(altvelocityswarmc, -1)
                    altitude_total.append(altvelocityswarmc)
                else:
                    pass

            return altitude_total

            # for i in range(len(platforms)):
            # for j in range(len(platforms)):
        altitude_array = altitude()
        return [time, latvelocitytotal, lonvelocitytotal, altitude_array]

    # image_bounds = imager_bounds()
    global em1
    em1 = ae.search(time_range[0], time_range[1],
                    platforms=platforms)
    lattotal, lontotal = [], []
    for k in range(len(platforms)):
        latstarts, lonstarts = [], [],
        for i in range(len(em1.data)):  # 3 spacecraft
            # sees if the data corresponds to the correct space-craft
            if(em1.data[i].data_source.platform == platforms[k]):
                latstarts.append(em1.data[i].location_geo.lat)  # starting
                lonstarts.append((em1.data[i].location_geo.lon))  # ending
        lattotal.append(latstarts)
        lontotal.append(lonstarts)
    return coordinates_extrapolation(
        np.array([lattotal, lontotal]))

 # [time, satellite_lattitute,.] #add everything in main


def main():
    print(__name__)

    save_file = []
    asi_array_code = sky_map_values[0]
    location_code = sky_map_values[1]
    fig, ax = plt.subplots(
        3, 1, figsize=(7, 10), gridspec_kw={'height_ratios': [4, 1, 1]}, constrained_layout=True
    )

    # retrieves emphermaris data in the format [time, [sattelite_longitudes], [satellite_lattitudes], [satelliite_alitiudes]]
    data = emph()
    sat_time = data[0]

    # Finds the footprint of selected region with each selected spacecraft

    def animator():

        for i, (time, image, _, im) in enumerate(movie_generator):
            # Plot the entire satellite track, its current location, and a 20x20 km box
            # around its location.
            ax[1].clear()
            ax[2].clear()
            for j in range(len(sat_azel_pixels_total)):
                ax[0].plot(sat_azel_pixels_total[j][:, 0],
                           sat_azel_pixels_total[j][:, 1], 'blue')
                ax[0].scatter(sat_azel_pixels_total[j][i, 0], sat_azel_pixels_total[j][i, 1],
                              c='red', marker='o', s=50)
                ax[0].contour(area_mask_total[j][i, :, :],
                              levels=[0.99], colors=['yellow'])

                ax[1].plot(sat_time, nearest_pixel_intensity_total[j])
                ax[2].plot(sat_time, area_intensity_total[j])

                # Plot the ASI intensity along the satellite path
            vline1 = ax[1].axvline(time, c='b')
            vline2 = ax[2].axvline(time, c='b')

            # Annotate the location_code and satellite info in the top-left corner.
            location_code_str = (
                f'{asi_array_code}/{location_code} '
                f'LLA=({asi.meta["lat"]:.2f}, '
                f'{asi.meta["lon"]:.2f}, {asi.meta["alt"]:.2f})'
            )
            text_obj = ax[0].text(
                0,
                1,
                location_code_str,
                va='top',
                transform=ax[0].transAxes,
                color='red',
            )
            ax[1].set(ylabel='ASI intensity\nnearest pixel [counts]')
            ax[2].set(xlabel='Time',
                      ylabel='ASI intensity\n10x10 km area [counts]')

    def ASI_logic():

        alt = 110  # footprint value
        if(asi_array_code.lower() == 'themis'):
            asi = asilib.asi.themis(
                location_code, time_range=time_range, alt=alt)
        elif(asi_array_code.lower() == 'rego'):
            asi = asilib.asi.rego(
                location_code, time_range=time_range, alt=alt)
            print("test")
        elif(asi_array_code.lower() == 'trex_nir'):
            asi = asilib.asi.trex.trex_nir(
                location_code, time_range=time_range, alt=alt)
        return asi
    asi = ASI_logic()
    # Initiate the movie generator function. Any errors with the data will be␣

    movie_generator = asi.animate_fisheye_gen(  # initaliziation
        ax=ax[0], azel_contours=True, overwrite=True, cardinal_directions='NE'
    )
    # Use the generator to get the images and time stamps to estimate mean the ASI
    # brightness along the satellite path and in a (20x20 km) box.
    sat_azel_pixels_total, nearest_pixel_intensity_total, area_intensity_total, area_mask_total = [], [
    ], [], []  # Creates empty arrays for data from satellites to go, which is there rendered in animator

    def Animation_logic_generator():
        nonlocal sat_azel_pixels_total, nearest_pixel_intensity_total, area_intensity_total, area_mask_total

        for i in range(len(platforms)):  # length of spacecraft

            conjunction_obj = asilib.Conjunction(asi, (sat_time, np.array(  # lat_sattelite, long, alt
                [data[1][i], data[2][i], data[3][i]]).T))
            print(data[3][i])
            blah = conjunction_obj.lla_footprint(alt=110)
            print(blah)
            # Normally the satellite time stamps are not the same as the ASI.
            # You may need to call Conjunction.interp_sat() to find the LLA coordinates
            # at the ASI timestamps.
            # Map the satellite track to the imager's azimuth and elevation coordinates and
            # image pixels. NOTE: the mapping is not along the magnetic field lines! You need
            # to install IRBEM and then use conjunction.lla_footprint() before
            # calling conjunction_obj.map_azel.
            sat_azel, sat_azel_pixels = conjunction_obj.map_azel()
            nearest_pixel_intensity = conjunction_obj.intensity(box=None)
            area_intensity = conjunction_obj.intensity(box=(10, 10))
            area_mask = conjunction_obj.equal_area(box=(10, 10))

            # Need to change masked NaNs to 0s so we can plot the rectangular area contours.
            area_mask[np.where(np.isnan(area_mask))] = 0
            sat_azel_pixels_total.append(sat_azel_pixels)
            nearest_pixel_intensity_total.append(nearest_pixel_intensity)
            area_intensity_total.append(area_intensity)
            area_mask_total.append(area_mask)
        # sat_azel_pixels, area_box_mask_2, asi_brightness_2 for each satellite
    Animation_logic_generator()
    animator()

    print(
        f'Movie saved in {asilib.config["ASI_DATA_DIR"] / "animations"}', )

    movie_container = 'mp4'
    movie_address = f'{time_range[0].strftime("%Y%m%d_%H%M%S")}_' \
        f'{time_range[1].strftime("%H%M%S")}_' \
        f'{asi_array_code.lower()}_{location_code.lower()}_fisheye.{movie_container}'

    movie_address_total = asilib.config["ASI_DATA_DIR"] / \
        'animations'/movie_address
    print(movie_address_total)
    save_file.append(movie_address_total)
    return save_file


if __name__ == '__main__':  # arg parse look up
    test = main()

Error message:

Traceback (most recent call last):
  File "c:/Users/1101w/Clone/Programming-8/Summer 2023/AnimationSattelite.py", line 262, in <module>
    test = main()
  File "c:/Users/1101w/Clone/Programming-8/Summer 2023/AnimationSattelite.py", line 243, in main
    Animation_logic_generator()
  File "c:/Users/1101w/Clone/Programming-8/Summer 2023/AnimationSattelite.py", line 231, in Animation_logic_generator
    sat_azel, sat_azel_pixels = conjunction_obj.map_azel()
  File "C:\Users\1101w\anaconda3\lib\site-packages\asilib\conjunction.py", line 315, in map_azel
    az, el, _ = pymap3d.geodetic2aer(
  File "C:\Users\1101w\anaconda3\lib\site-packages\pymap3d\aer.py", line 110, in geodetic2aer
    e, n, u = geodetic2enu(lat, lon, h, lat0, lon0, h0, ell, deg=deg)
  File "C:\Users\1101w\anaconda3\lib\site-packages\pymap3d\enu.py", line 199, in geodetic2enu
    x1, y1, z1 = geodetic2ecef(lat, lon, h, ell, deg=deg)
  File "C:\Users\1101w\anaconda3\lib\site-packages\pymap3d\ecef.py", line 68, in geodetic2ecef
    lat, ell = sanitize(lat, ell, deg)
  File "C:\Users\1101w\anaconda3\lib\site-packages\pymap3d\utils.py", line 62, in sanitize
    raise ValueError("-pi/2 <= latitude <= pi/2")
ValueError: -pi/2 <= latitude <= pi/2
PS C:\Users\1101w\Clone\Programming-8>

Runtime information:

0.16.0
3.8.3 (default, Jul 2 2020, 17:30:36) [MSC v.1916 64 bit (AMD64)]
Windows-10-10.0.19041-SP0

Context for the issue:

No response

BUG: Keogram not acceptating datetime.datetime objects

Describe the issue:

When putting in a datetime to keogram, it errors.

Reproduce the code example:

from datetime import datetime
import asilib
import asilib.asi

time_range = [datetime(2021, 3, 18, 8), datetime(2021, 3, 18, 9)]
axes, img1 = asilib.plot_keogram("rego", "fsmi", time_range, map_alt=110)

Error message:

Traceback (most recent call last):
  File "c:/Users/1101w/Clone/Programming-8/Summer 2023/keogram_test.py", line 10, in <module>
    axes, img1 = asilib.plot_keogram("rego", "fsmi", time_range, map_alt=110)
  File "C:\Users\1101w\anaconda3\lib\site-packages\asilib\plot\plot_keogram.py", line 105, in plot_keogram
    keo_df = keogram(asi_array_code, location_code, time_range, map_alt, path=path, aacgm=aacgm)
  File "C:\Users\1101w\anaconda3\lib\site-packages\asilib\analysis\keogram.py", line 62, in keogram
    return keo.keogram(map_alt, path, aacgm)
  File "C:\Users\1101w\anaconda3\lib\site-packages\asilib\analysis\keogram.py", line 232, in keogram
    [
  File "C:\Users\1101w\anaconda3\lib\site-packages\asilib\analysis\keogram.py", line 233, in <listcomp>
    (f - i).total_seconds()
AttributeError: 'numpy.timedelta64' object has no attribute 'total_seconds'

Runtime information:

0.20.5
3.8.3 (default, Jul 2 2020, 17:30:36) [MSC v.1916 64 bit (AMD64)]
Windows-10-10.0.19041-SP0

Context for the issue:

Can't use keograms with existing code and probably an easy fix.

BUG: Cannot uninstall opencv-python 4.6.0, RECORD file not found. Hint: The package was installed by conda.

Describe the issue:

It seems installation in a cloud environment (using HelioCloud) has an issue related to how opencv is installed. The package attempts to install it, but it does not recognize the package already installed by conda.

Reproduce the code example:

import os

os.system("pip install aurora-asi-lib")

Error message:

Collecting aurora-asi-lib
  Using cached aurora_asi_lib-0.20.1-py3-none-any.whl (3.4 MB)
Collecting opencv-python<5.0,>=4.8.0
  Using cached opencv_python-4.8.1.78-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (61.7 MB)
Requirement already satisfied: numpy in /srv/conda/envs/notebook/lib/python3.10/site-packages (from aurora-asi-lib) (1.23.5)
Requirement already satisfied: requests in /srv/conda/envs/notebook/lib/python3.10/site-packages (from aurora-asi-lib) (2.28.1)
Collecting ffmpeg-python
  Using cached ffmpeg_python-0.2.0-py3-none-any.whl (25 kB)
Requirement already satisfied: scipy in /srv/conda/envs/notebook/lib/python3.10/site-packages (from aurora-asi-lib) (1.9.3)
Collecting matplotlib<4.0,>=3.7
  Using cached matplotlib-3.8.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.6 MB)
Requirement already satisfied: cdflib>=0.3.20 in /srv/conda/envs/notebook/lib/python3.10/site-packages (from aurora-asi-lib) (0.4.9)
Requirement already satisfied: pyshp in /srv/conda/envs/notebook/lib/python3.10/site-packages (from aurora-asi-lib) (2.3.1)
Requirement already satisfied: rego-imager-readfile<2.0,>=1.2.0 in /srv/conda/envs/notebook/lib/python3.10/site-packages (from aurora-asi-lib) (1.2.2)
Collecting trex-imager-readfile<2.0,>=1.4.4
  Using cached trex_imager_readfile-1.5.1-py3-none-any.whl (15 kB)
Requirement already satisfied: beautifulsoup4 in /srv/conda/envs/notebook/lib/python3.10/site-packages (from aurora-asi-lib) (4.11.1)
Requirement already satisfied: aacgmv2 in /srv/conda/envs/notebook/lib/python3.10/site-packages (from aurora-asi-lib) (2.6.2)
Requirement already satisfied: themis-imager-readfile<2.0,>=1.2.1 in /srv/conda/envs/notebook/lib/python3.10/site-packages (from aurora-asi-lib) (1.3.0)
Requirement already satisfied: wheel in /srv/conda/envs/notebook/lib/python3.10/site-packages (from aurora-asi-lib) (0.38.4)
Requirement already satisfied: pymap3d in /srv/conda/envs/notebook/lib/python3.10/site-packages (from aurora-asi-lib) (3.0.1)
Requirement already satisfied: pandas in /srv/conda/envs/notebook/lib/python3.10/site-packages (from aurora-asi-lib) (1.5.2)
Requirement already satisfied: kiwisolver>=1.0.1 in /srv/conda/envs/notebook/lib/python3.10/site-packages (from matplotlib<4.0,>=3.7->aurora-asi-lib) (1.4.4)
Requirement already satisfied: contourpy>=1.0.1 in /srv/conda/envs/notebook/lib/python3.10/site-packages (from matplotlib<4.0,>=3.7->aurora-asi-lib) (1.0.6)
Requirement already satisfied: packaging>=20.0 in /srv/conda/envs/notebook/lib/python3.10/site-packages (from matplotlib<4.0,>=3.7->aurora-asi-lib) (21.3)
Requirement already satisfied: python-dateutil>=2.7 in /srv/conda/envs/notebook/lib/python3.10/site-packages (from matplotlib<4.0,>=3.7->aurora-asi-lib) (2.8.2)
Requirement already satisfied: cycler>=0.10 in /srv/conda/envs/notebook/lib/python3.10/site-packages (from matplotlib<4.0,>=3.7->aurora-asi-lib) (0.11.0)
Requirement already satisfied: pyparsing>=2.3.1 in /srv/conda/envs/notebook/lib/python3.10/site-packages (from matplotlib<4.0,>=3.7->aurora-asi-lib) (3.0.9)
Requirement already satisfied: pillow>=6.2.0 in /srv/conda/envs/notebook/lib/python3.10/site-packages (from matplotlib<4.0,>=3.7->aurora-asi-lib) (9.2.0)
Requirement already satisfied: fonttools>=4.22.0 in /srv/conda/envs/notebook/lib/python3.10/site-packages (from matplotlib<4.0,>=3.7->aurora-asi-lib) (4.38.0)
Requirement already satisfied: h5py<4.0.0,>=3.1.0 in /srv/conda/envs/notebook/lib/python3.10/site-packages (from trex-imager-readfile<2.0,>=1.4.4->aurora-asi-lib) (3.7.0)
Requirement already satisfied: soupsieve>1.2 in /srv/conda/envs/notebook/lib/python3.10/site-packages (from beautifulsoup4->aurora-asi-lib) (2.3.2.post1)
Requirement already satisfied: future in /srv/conda/envs/notebook/lib/python3.10/site-packages (from ffmpeg-python->aurora-asi-lib) (0.18.2)
Requirement already satisfied: pytz>=2020.1 in /srv/conda/envs/notebook/lib/python3.10/site-packages (from pandas->aurora-asi-lib) (2022.6)
Requirement already satisfied: certifi>=2017.4.17 in /srv/conda/envs/notebook/lib/python3.10/site-packages (from requests->aurora-asi-lib) (2023.5.7)
Requirement already satisfied: charset-normalizer<3,>=2 in /srv/conda/envs/notebook/lib/python3.10/site-packages (from requests->aurora-asi-lib) (2.1.1)
Requirement already satisfied: idna<4,>=2.5 in /srv/conda/envs/notebook/lib/python3.10/site-packages (from requests->aurora-asi-lib) (3.4)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /srv/conda/envs/notebook/lib/python3.10/site-packages (from requests->aurora-asi-lib) (1.26.11)
Requirement already satisfied: six>=1.5 in /srv/conda/envs/notebook/lib/python3.10/site-packages (from python-dateutil>=2.7->matplotlib<4.0,>=3.7->aurora-asi-lib) (1.16.0)
Installing collected packages: opencv-python, ffmpeg-python, trex-imager-readfile, matplotlib, aurora-asi-lib
  Attempting uninstall: opencv-python
    Found existing installation: opencv-python 4.6.0

ERROR: Cannot uninstall opencv-python 4.6.0, RECORD file not found. Hint: The package was installed by conda.

Runtime information:

print(sys.version)
3.10.8 | packaged by conda-forge | (main, Nov 22 2022, 08:23:14) [GCC 10.4.0]


print(platform.platform())
Linux-5.4.238-148.346.amzn2.x86_64-x86_64-with-glibc2.35

Context for the issue:

No response

Issue Importing asilib.asi

Steps to reproduce:

Installed using: python3 -m pip install aurora-asi-lib
OS: Ventura 13.4 AND Windows 10 19045.3086
Mac: Python Version 3.9.12, Windows: 3.10.11

Error message:

When using "import asilib.asi" -> ModuleNotFoundError: No module named 'asilib.asi'

When using "import asilib" -> AttributeError: module 'asilib' has no attribute 'asi'

Additional information:

The code example I'm using from the documentation website (creation of a keogram):

import matplotlib.pyplot as plt

import asilib.asi

location_code = 'GILL'
time_range = ['2015-02-02T10', '2015-02-02T11']

asi = asilib.asi.rego(location_code, time_range=time_range, alt=230)
ax, p = asi.plot_keogram(color_map='Greys_r')
plt.xlabel('Time')
plt.ylabel('Geographic Latitude [deg]')
plt.colorbar(p)
plt.tight_layout()
plt.show()

The code I wrote based on the above example(I know it's far from the most efficient):
import matplotlib.pyplot as plt

import asilib.asi

loop = 1
while loop == 1:
print("Enter 4 letter camera location code (format: XXXX):")
location_code = input()
location_code = location_code.upper()
print('Enter the start date (format: YYYY-MM-DD):')
startDate = input()
print('Enter the start time in UT (format: 00):')
startTime = input()
startTime = 'T' + startTime
print('Enter the end time in UT (format: 00):')
endTime = input()
endTime = 'T' + endTime
endDate = startDate + endTime
startDate = startDate + startTime
time_range = [startDate, endDate]

asi = asilib.asi.themis(location_code, time_range, alt=110)
ax, p = asi.plot_keogram(color_map='Greys_r', aacgm=True)
plt.xlabel('Time')
plt.ylabel('Magnetic Latitude [deg]')
plt.colorbar(p)
plt.tight_layout()
plt.show()
print("Create another keogram? Y/N")
loopCheck = input()
loopCheck = loopCheck.lower()
if loopCheck[0] == 'y':
    loop = 1
else:
    loop = 0

Implementing custom satellites+data in fisheye animations when working with custom satellite and data

Two-line element data of scientific satellites are often published online and users can derive LLA positions as functions of time of the satellites based on this information. If I want to plot the trajectory of a satellite and the custom data it records on an ASI image, how would I go about doing this? In the example given,

from datetime import datetime

import numpy as np
import matplotlib.pyplot as plt

import asilib
import asilib.asi


# ASI parameters
location_code = 'RANK'
alt=110  # km
time_range = (datetime(2017, 9, 15, 2, 32, 0), datetime(2017, 9, 15, 2, 35, 0))

fig, ax = plt.subplots(
    3, 1, figsize=(7, 10), gridspec_kw={'height_ratios': [4, 1, 1]}, constrained_layout=True
)

asi = asilib.asi.themis(location_code, time_range=time_range, alt=alt)

# Create the fake satellite track coordinates: latitude, longitude, altitude (LLA).
# This is a north-south satellite track oriented to the east of the THEMIS/RANK
# imager.
n = int((time_range[1] - time_range[0]).total_seconds() / 3)  # 3 second cadence.
lats = np.linspace(asi.meta["lat"] + 5, asi.meta["lat"] - 5, n)
lons = (asi.meta["lon"] - 0.5) * np.ones(n)
alts = alt * np.ones(n)  # Altitude needs to be the same as the skymap.
sat_lla = np.array([lats, lons, alts]).T
# Normally the satellite time stamps are not the same as the ASI.
# You may need to call Conjunction.interp_sat() to find the LLA coordinates
# at the ASI timestamps.
sat_time = asi.data.time

conjunction_obj = asilib.Conjunction(asi, (sat_time, sat_lla))

# Map the satellite track to the imager's azimuth and elevation coordinates and
# image pixels. NOTE: the mapping is not along the magnetic field lines! You need
# to install IRBEM and then use conjunction.lla_footprint() before
# calling conjunction_obj.map_azel.
sat_azel, sat_azel_pixels = conjunction_obj.map_azel()

# Calculate the auroral intensity near the satellite and mean intensity within a 10x10 km area.
nearest_pixel_intensity = conjunction_obj.intensity(box=None)
area_intensity = conjunction_obj.intensity(box=(10, 10))
area_mask = conjunction_obj.equal_area(box=(10,10))

# Need to change masked NaNs to 0s so we can plot the rectangular area contours.
area_mask[np.where(np.isnan(area_mask))] = 0

# Initiate the animation generator function.
gen = asi.animate_fisheye_gen(
    ax=ax[0], azel_contours=True, overwrite=True, cardinal_directions='NE'
)

for i, (time, image, _, im) in enumerate(gen):
    # Plot the entire satellite track, its current location, and a 20x20 km box
    # around its location.
    ax[0].plot(sat_azel_pixels[:, 0], sat_azel_pixels[:, 1], 'red')
    ax[0].scatter(sat_azel_pixels[i, 0], sat_azel_pixels[i, 1], c='red', marker='o', s=50)
    ax[0].contour(area_mask[i, :, :], levels=[0.99], colors=['yellow'])

    if 'vline1' in locals():
        vline1.remove()  # noqa: F821
        vline2.remove()  # noqa: F821
        text_obj.remove()  # noqa: F821
    else:
        # Plot the ASI intensity along the satellite path
        ax[1].plot(sat_time, nearest_pixel_intensity)
        ax[2].plot(sat_time, area_intensity)
    vline1 = ax[1].axvline(time, c='b')
    vline2 = ax[2].axvline(time, c='b')

    # Annotate the location_code and satellite info in the top-left corner.
    location_code_str = (
        f'THEMIS/{location_code} '
        f'LLA=({asi.meta["lat"]:.2f}, '
        f'{asi.meta["lon"]:.2f}, {asi.meta["alt"]:.2f})'
    )
    satellite_str = f'Satellite LLA=({sat_lla[i, 0]:.2f}, {sat_lla[i, 1]:.2f}, {sat_lla[i, 2]:.2f})'
    text_obj = ax[0].text(
        0,
        1,
        location_code_str + '\n' + satellite_str,
        va='top',
        transform=ax[0].transAxes,
        color='red',
    )
    ax[1].set(ylabel='ASI intensity\nnearest pixel [counts]')
    ax[2].set(xlabel='Time', ylabel='ASI intensity\n10x10 km area [counts]')

print(f'Animation saved in {asilib.config["ASI_DATA_DIR"] / "animations" / asi.animation_name}')

there does not seem to be a convenient way to include an extra subplot. Trying to arbitrarily insert ax[3] leads to indexing errors and the code for the fisheye animation generator shows that ax.plots is set to None by default. Is there a way for a user to insert custom data? I've been doing it by replacing the plotted area_intensity with my own data, but that requires using functions like np.tile in order to ensure that my data is in the correct form.

Is there a better way to do it?

For custom satellite data: I have been downsampling my satellite trajectory (LLA coords) and passing it to asilib.Conjunction, but I would like to know what the intended usage of interp_sat() is. Is it intended for use on position data where the original data is sampled at an extremely high rate? Of course, there are inaccuracies, but I want to know if asilib has an inbuilt method to handle this. However this is not a huge problem.

Relax dependency requirements?

I would like to show an example or two using asilib at the upcoming PyHC summer school. All the notebooks we present should run in the shared PyHC environment we're setting up. There is a rather narrow range of versions for the various PyHC package dependencies that will allow them to coexist in the same environment. Looking at the asilib requirements.txt, I see that many of asilib's dependencies are pinned to exact versions, and I'm 99% sure that's going to cause a conflict if we try to install asilib in the shared environment. Would it be possible to turn some of those "==" dependencies into lower bounds?

Great package, by the way! Have you considered using pyspedas to do some of the data downloads, to take advantage of it's ability to cache data files and only download them if needed?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.