Trying to install pyDARN on open suse 15.1 via:

pip3 install --user pydarn
pip3 install --user git+https://github.com/superdarn/pydarn
git clone https://github.com/superdarn/pydarn
cd pydarn
pip3 install --user .
python3 setup.py install --user 

All resulting in the same error with:

Traceback (most recent call last):
  File "setup.py", line 67, in <module>
    'h5py', 'deepdish', 'pathlib2']

Last line in the error is:

  File "/usr/lib64/python3.6/lib2to3/pgen2/grammar.py", line 108, in load
    d = pickle.load(f)
EOFError: Ran out of input

• how to test
• how to make/guidelines
• how to code review

BUG

When calculating the slant range in gate2slant(), the center=0 and center=1 cases are flipped.

Priority

• Minor - typo, incorrect naming, fails on specific uncommon situation
• Major - misrepresentation of data, failure in a common situation
• Urgent - failure in usage, large misrepresentation

Example of the bug

This bug was found by looking at the source code (see below), so I don't have any specific test code to demonstrate it.

Potential Bug Location

Compare to the equivalent calculation in RST:

    /* If not calculating center position of range-beam cell then calculate
     * position of near-left corner instead */
    if (center==0) {
        bm_edge=-pos->bmsep*0.5;
        range_edge=-0.5*rsep*20/3;
    }

Potential Solution(s)

Should be a simple swap:

if center:
        range_offset = 0.0
else:
        # 0.5 offset to the edge of the range gate
        range_offset = -0.5 * record['rsep']

In #72 we noticed plt.figure will override a user's plt.figure, now is the time to investigate what is the best method to do this.

As per this we should strive to use setup.cfg.

Also, we should use a MANIFEST.in file for including other files into the pydarn package as per this.

In summary, we should:

1. Use a setup.cfg file
2. Use a MANIFEST.in file and include_package_data=True

New Feature Nottification

Name: plot_grid

module: grid.py

package: plotting

Scope

This feature will plot grid files on a polar plot in magnetic coordinates.
Will not plot on a geographic map or geo-coordinates.

Description

We will use a grid file to plot data on the polar plot from magnetic coordinates.

pyDARN Checklist

• Does this fit within pyDARN's scope?
• Is this a minor change i.e., new small utility function?
• Is this a major change i.e., new plotting function?
• Do you need help developing it?
• Have you created a pyDARN GitHub project to show the tasks needed to be done?

BUG

When plotting range-time plots or summary plots using the slant=True option the major ticks are shown at 200km starting at ~180km. Minor ticks are set to 100km, assuming that they are starting from the nearest '100' which in this case is at 200 km, +20km from the major ticks.

Priority

• Minor - typo, incorrect naming, fails on specific uncommon situation
• Major - misrepresentation of data, failure in a common situation (technically, but they're not labelled and are pointing to the full rounded hundreds so it's not a horrific misrepresentation of the data)
• Urgent - failure in usage, large misrepresentation

Example of the bug

import matplotlib.pyplot as plt 
import pydarn

#Load in dmap file
file = "/Users/carley/Documents/data/20161123.1601.00.mcm.a.fitacf"
SDarn_read = pydarn.SuperDARNRead(file)
fitacf_data = SDarn_read.read_fitacf()
#Plot and show
a = pydarn.RTP.plot_range_time(fitacf_data, beam_num=4, slant=True)
plt.show()

Attempts

What have you tried already to fix it?

What have you tried to get around it?

Data Location

Any fitacf file should recreate this bug in range-time plots.

Potential Bug Location

Line 402 and 417 in rtp.py

        if slant:
            ax.yaxis.set_ticks(np.arange(ymin, ymax+1, yspacing))
        else:
            ax.yaxis.set_ticks(np.arange(ymin, ymax+1, (ymax)/5))

.......
        if slant:
            ax.yaxis.set_minor_locator(ticker.MultipleLocator(100))
        else:
            ax.yaxis.set_minor_locator(ticker.MultipleLocator(5))

Potential Solution(s)

Possibly need to set the base, or bottom tick for both to be the same?

Extra Notes

No issue with range gates for the plots.
While we're playing with ticks, it might be nice to have them both round off at the hundreds?

New Feature

Name: plot_fan

module: fan.py

package: plotting/

Scope

• Create new optional input for plot_fan
• Conversion of datetime to scan_index (don't think it's generalised enough to go in utils, or would be used anywhere else, so probably a few lines of code in plot_fan?)
• Return the scan_index value for user to note/if it's needed elsewhere - possible need to be in utils?
• Update documentation on fan plotting.

What will the feature do?

• Allow plotting fan plots from a given date and time, rather than requiring a scan number.
• Convert a datetime object into the relevant scan index for given data file to be used in the plotting.

What will the feature not do?

• Changing any code relating to scan_index and the underlying way that fan plots are plotted.
• Scan index will still be a parameter that you can input into fan plots.

Description

New optional input for fan plots where a datetime object can be inputted which is used to find the scan index number at that time to plot the scan at the given time.
Do not intend to change how plot_fan works (I.e using scan_index to read in correct data) unless required.
Make as similar to plot_grid datetime input functionality as possible.
Be aware scan_index default is 1 - handle properly if optional input of datetime is given.
Be aware of previous conventions used in other libraries (I.e RST gets the next scan from the time given) and write documentation accordingly.

Discussion subject: function of converting datetime to scan index required in a utils package, or just as small addition to plot_fan?

pyDARN Checklist

• Does this fit within pyDARN's scope?
• Is this a minor change i.e., new small utility function?
• Is this a major change i.e., new plotting function?
• Do you need help developing it?
• Have you created a pyDARN GitHub project to show the tasks needed to be done? (Added to fan plots project)

User Interface

    if datetime given as input
        convert datetime object into scan_index in file
        return scan_index
   
    input new scan_index into remaining plot_fan code

pyDARN 2.0

I am purposing a Major release due to the structural change with the move of the IO package.

Other new features:

• power plots
• fan plots
• slant ranges
• Hardware updating
• updated documentation

With old dat files being converted to RAWACF files and higher data products, some of their issues are coming to light. As well, this could occur in current radar files as radar OS's are changing.

One issue is erroneous dates in the file. A warning can be raised in the case of the dates being consistently increasing with gaps something.
Then an error for some dates that jump around too much.

This will help DDWG spot these errors and potentially prevent errors in plotting methods as well.

This is just a housekeeping issue post. Make sure we don't lose track of things.

As mentioned in #4, we need to handle char arrays differently when reading dmap arrays. For more details see "Outstanding Issue" in this comment: #4 (comment)

We can fix this by adding:
if array_type_fmt == 'c':
array_value = np.array([ord(x) if len(x) == 1 else 0 for x in array_value])

right here.

edit: This fix only handles reading in the char arrays. Not sure what happens when we write them out.

BUG

While accessing some of the map2 files I'm getting this error

Error: The following fields in record 462 are missing: {'vector.mlat', 'vector.vel.median', 'vector.mlon', 'vector.kvect', 'vector.vel.sd', 'vector.channel', 'vector.index', 'vector.stid'}
Traceback (most recent call last):
File "convert_map_to_csv.py", line 22, in read_map_files
_md = reader.read_map()
File "/home/shibaji/anaconda3/envs/collb/lib/python3.7/site-packages/pydarnio/dmap/superdarn.py", line 493, in read_map
self._read_darn_records(file_struct_list)
File "/home/shibaji/anaconda3/envs/collb/lib/python3.7/site-packages/pydarnio/dmap/superdarn.py", line 374, in _read_darn_records
self._read_darn_record(format_fields)
File "/home/shibaji/anaconda3/envs/collb/lib/python3.7/site-packages/pydarnio/dmap/superdarn.py", line 350, in _read_darn_record
SDarnUtilities.missing_field_check(format_fields, record, self.rec_num)
File "/home/shibaji/anaconda3/envs/collb/lib/python3.7/site-packages/pydarnio/dmap/superdarn.py", line 178, in missing_field_check
SuperDARNFieldMissingError(rec_num, missing_fields)
pydarnio.exceptions.superdarn_exceptions.SuperDARNFieldMissingError: Error: The following fields in record 462 are missing: {'vector.mlat', 'vector.vel.median', 'vector.mlon', 'vector.kvect', 'vector.vel.sd', 'vector.channel', 'vector.index', 'vector.stid'}

File - /sd-data/2012/map2/north/20120309.north.map2.bz2

With the announcement of pyDARN splitting off the io, I am getting feedback on how users want the io to stay in to make pyDARN one complete package.
As I don't disagree with either separate or keep together, I thought an idea to achieve the best of both worlds.

We can make pyDARNio a dependency of pyDARN via install in setup file or sub-repo.
I am leaning to the sub-repo idea.

This way, we can keep changes separate but have io together with pyDARN for those that prefer it that way.

I will leave this up as an agenda topic for the next meeting.

Re-look at why matplotlibs deprecation warnings are showing up in plotting methods.

I'm using the develop branch, specifically commit 9b5caa2 and I noticed several small issues. I might be able to help fix them, but figured they should be reported:

Summary of issues:

• default velocity colormap is reversed
• when you filter out frequency for for some modes, you get stripes in the RTI
• some documentation for RTP is wrong
• auto-scale for z goes wacky

Details

1. The default velocity colormap doesn't follow the redshift/blueshift convention. For example, I plotted up some Saskatoon radar data with this code:

   import pydarn
   from matplotlib import pyplot
   fitted_file = "20120614.0401.1200.sas.lmfit2"
   darn_read = pydarn.SDarnRead(fitted_file)
   data = darn_read.read_fitacf()
   filts = {'max_scalar_filter':{'tfreq': 12500}}
   fig = pyplot.figure(figsize=(10,8))
   ax1 = fig.add_subplot(111)
   pydarn.RTP.plot_range_time(data, parameter='v',
   beam_num=12, ax=ax1,
   filter_settings=filts,
   zmin=-1000, zmax=1000)
   pyplot.show()

You can download the lmfit2 file here: https://github.com/asreimer/lmfit2/blob/master/example/20120614.0401.1200.sas.lmfit2.gz

The plot I get is this:

But what I should have gotten is a plot where the colormap is reversed. See: https://github.com/asreimer/lmfit2/blob/master/example/no_filter.png or this RTI that was generated using the VT superdarn website:

Notice that red and blue are backwards.

2. If you look at the plot I generated using the code in 1, you'll see it has many white stripes. This is undesirable behaviour and will likely happen on all of the "twofsound" mode data.

3. The docstring for RTP says that one should use the plot_filter keywork, which is incorrect. One needs to use filter_settings instead.

4. The autoscaling for the parameter being plotted can go wacky. Running this code:

   filts = {'min_scalar_filter':{'tfreq': 12500}}
   fig = pyplot.figure(figsize=(10,8))
   ax1 = fig.add_subplot(111)
   pydarn.RTP.plot_range_time(data, parameter='p_l',
   beam_num=12, ax=ax1,
   filter_settings=filts)

   pyplot.show()

Gives this:

but using the zmin and zmax fixes the problem:

filts = {'min_scalar_filter':{'tfreq': 12500}}
fig = pyplot.figure(figsize=(10,8))
ax1 = fig.add_subplot(111)
pydarn.RTP.plot_range_time(data, parameter='p_l',
                           beam_num=12, ax=ax1,
                           filter_settings=filts,
                           zmin=1,zmax=30)

pyplot.show()

feature request for native gzip or bzip decompression while reading. Most of our users have their data stored in compressed formats, and RST is capable of reading in compressed formats.

Purposed fix for running into missing new radars is to have the raised exception attempt another git submodule pull to grab any new radars that are missing. Of course, this will come with the change to using radar.dat and the new hardware repo.

More details #59

This is a low priority feature request.

It might be nice to include some cursor control methods in the DmapRead class so that one can jump to different records in a file without reading out all of the other records in the file.

For example:

import pydarn
filename='20171231.0001.00.sas.fitacf' #any dmap file will do
dmap_handler = pydarn.DmapRead(filename)
print(dmap_handler)  # prints a 0
record = dmap_handler.read_record()
print(dmap_handler)  # prints something larger than 0.
# begin new methods
dmap_handler.rewind() # sets cursor to beginning of the file
dmap_handler.next() # jumps to next record in the file
dmap_handler.jump(record_number) # sets the cursor to an offset where record "record_number" resides.

I've used this functionality in davitpy before, but it isn't critical to typical use cases.

This is addressing issue #59 where the exception should include:

• hardware location with pyDARN structure
• please make a github issue if this happens on import

BUG

When plotting multiple radars with overlapping FOVs, vectors are missing in the resultant plot.

Priority

• [x ] Major - misrepresentation of data, failure in a common situation

Example of the bug

The following code, plotting a handful of northern hemisphere radars:

site_file_dict = {'bks':'sddata/20160306.1200.02.bks.fitacf.bz2',
                  'fhe':'sddata/20160306.1200.02.fhe.fitacf.bz2',
                  'cly':'sddata/20160306.1200.02.cly.fitacf.bz2',
                  'fhw':'sddata/20160306.1200.02.fhw.fitacf.bz2',
                  'kap':'sddata/20160306.1200.03.kap.fitacf.bz2',
                  'wal':'sddata/20160306.1201.00.wal.fitacf.bz2',
                  'sto':'sddata/20160306.1201.00.sto.fitacf.bz2'}

fanplot = {}

for radar_site in site_file_dict.keys():
    sdfpath = site_file_dict[radar_site]
    with bz2.open(sdfpath) as grid_file:
        fstream = grid_file.read()
        SDarn_read = pydarnio.SDarnRead(fstream, True)
        records = SDarn_read.read_fitacf()
    kwargs = {'boundary':True, 'zmax':200, 'zmin':-200, 'groundscatter':True, 'lowlat':40, 'scan_index':89}
    # only plot the colorbar once
    if radar_site == 'bks':
        fanplot[radar_site] = pydarn.Fan.plot_fan(records, colorbar=True, **kwargs)
    else:
        fanplot[radar_site] = pydarn.Fan.plot_fan(records, colorbar=False, **kwargs)

produces the following plot:

Hiding the overlapping radars (STO and KAP in this case) will show the missing vectors at CLY:

site_file_dict = {'bks':'sddata/20160306.1200.02.bks.fitacf.bz2',
                  'fhe':'sddata/20160306.1200.02.fhe.fitacf.bz2',
                  'cly':'sddata/20160306.1200.02.cly.fitacf.bz2',
                  'fhw':'sddata/20160306.1200.02.fhw.fitacf.bz2',
#                   'kap':'sddata/20160306.1200.03.kap.fitacf.bz2',
#                   'sto':'sddata/20160306.1201.00.sto.fitacf.bz2',
                  'wal':'sddata/20160306.1201.00.wal.fitacf.bz2'}

fanplot = {}

for radar_site in site_file_dict.keys():
    sdfpath = site_file_dict[radar_site]
    with bz2.open(sdfpath) as grid_file:
        fstream = grid_file.read()
        SDarn_read = pydarnio.SDarnRead(fstream, True)
        records = SDarn_read.read_fitacf()
    kwargs = {'boundary':True, 'zmax':200, 'zmin':-200, 'groundscatter':True, 'lowlat':40, 'scan_index':89}
    # only plot the colorbar once
    if radar_site == 'bks':
        fanplot[radar_site] = pydarn.Fan.plot_fan(records, colorbar=True, **kwargs)
    else:
        fanplot[radar_site] = pydarn.Fan.plot_fan(records, colorbar=False, **kwargs)

yields

Data Location

FITACFs from VT.

Potential Bug Location

Line 204 in pydarn/fan.py: Fan.plot_fan() method

Potential Solution(s)

In above lines, set the alpha of the RGBA value to 0.

Doing so produces the following result:

New Feature

Name: labels and dots to indicate where the radars are located

module: fan.py

package: /plotting

Scope

New option in plot_fov to include Radar abbreviation and the radar site dots will be part of the FOV plot.

pyDARN Checklist

• Does this fit within pyDARN's scope?
• Is this a minor change i.e., new small utility function?
• Is this a major change i.e., new plotting function?
• Do you need help developing it?
• Have you created a pyDARN GitHub project to show the tasks needed to be done?

User Interface

plot_fov(66, radar_label=True)

Documentation

Putting in a CODE_OF_CONNDUCT.md to help understand:

• standards
• policies
• environment
• enforcement
  within the pyDARN/DAWG community

Purpose

• update
• new
• remove
• typo/grammar change
• markdown improvement

Category

• code documentation
• wiki
• GitHub
• README
• installation guide
  • OS specifics
  • dependency
  • environment
• tutorials
  • plotting
  • io
  • utils/exceptions
• examples
• citing/licensing
• trouble shooting
  • installation
  • tutorials
• history
• guidelines
  • developer
  • communication

Details

Will be generated from a GitHub provided template to then be modified for the pyDARN/DAWG community

Documentation

Following the code of conduct template, I will list here additive documentation pyDARN will need.

• Communication guidelines: how each member of pyDARN/DAWG should communicate and behave
• updating pyDARN to reference DAWG charter

Purpose

• update
• new
• remove
• typo/grammar change
• markdown improvement

Category

• code documentation
• wiki
• README
• installation guide
  • OS specifics
  • dependency
  • environment
• tutorials
  • plotting
  • io
  • utils/exceptions
• examples
• citing/licensing
• trouble shooting
  • installation
  • tutorials
• history
• guidelines
  • developer
  • communication

Hi, I recently installed pydarn to work on SuperDARN data. I am able to read and do some plots shown on the tutorial for .fitacf files. My problem is I have other data with the old format, '.fit', which I need to read and create a data frame for further analysis (time series analysis). So I am wondering if it is possible to read .fit files using pydarn.

Thank you!

currently have to use

from pydarn.utils.conversions import dict2dmap

in io/borealis.py

When preparing to test PR #52 I encountered an unrelated error HardwareFileNotFoundError. This also happens on the develop branch, so I believe is related to recent changes there.. The error message doesn't help me fix it: "Hardware file for dcn radar was not found. Please insure the abbreviation is correct and there exists a hardware file for it by checking: https://github.com/vtsuperdarn/hdw.dat".

I have the latest version of RST running and have a dcn hardware file.

Suggested changes:

• Change error message to point the user to where pydarn is accessing the hardware files locally.
• Not stop the install if hardware files are missing. There's still things you can do without them, like load fit- and rawACF files. Instead, indicate that functionality will be extremely limited.

An extra check for map file reading sees if the station ids are for the correct hemisphere.

Purposed solution:
Using the hardware files and SuperDARN.Radars class you can determine which stid belongs to which hemisphere then cross-checking that will stid present in the map file and the hemisphere field in the map file.

Need to update the plotting documentation and put in tutorials for new plots:

• ACF docs
• Power docs
• update summary for slant range

BUG

FOV only shows the first 75 range gates which maybe not be correct for radars with a large FOV.

Priority

• Minor - typo, incorrect naming, fails on specific uncommon situation
• Major - misrepresentation of data, failure in a common situation
• Urgent - failure in usage, large misrepresentation

Example of the bug

Data Location

make_grid with FHW data or any known radar with more than 75 Rage gates.

Potential Bug Location

Please note this is not required.
If you can link to some code in pyDARN where you think the bug is coming from this will help the developer fix it sooner.

Potential Solution(s)

Change the default and get the range gate value from the hardware files.

import pydarn
Traceback (most recent call last):
File "", line 1, in
File "/home/k2/Documents/superdarn_github/borealis/pydarn/pydarn/init.py", line 38, in
from .io.borealis.borealis import BorealisRead
File "/home/k2/Documents/superdarn_github/borealis/pydarn/pydarn/io/borealis/borealis.py", line 49, in
from .borealis_site import BorealisSiteRead, BorealisSiteWrite
File "/home/k2/Documents/superdarn_github/borealis/pydarn/pydarn/io/borealis/borealis_site.py", line 47, in
from pathlib2 import Path
ModuleNotFoundError: No module named 'pathlib2'

Fan plots are coming along with @DannoPeters and @billetd help.

I just wanted to let everyone it is happening and almost ready for PR.

Documentation

Need to add documentation on GRID plotting

Purpose

• update
• new
• remove
• typo/grammar change
• markdown improvement

Category

• code documentation
• wiki
• README
• installation guide
  • OS specifics
  • dependency
  • environment
• tutorials
  • plotting
  • io
  • utils/exceptions
• examples
• citing/licensing
• trouble shooting
  • installation
  • tutorials
• history
• guidelines
  • developer
  • communication

Details

Will include how to use and how to process grid files from RST docs.

Update: @DannoPeters did some work around this last summer and will be continuing again this summer.

When trying to read a rawACF I get this error:

pydarn.exceptions.superdarn_exceptions.SuperDARNFieldMissingError: Error: The following fields in record 14 are missing: {'acfd'}

Example code (using python3.7)
in_fn = 'pydarn/raw_sas/20090101.1201.00.sas.rawacf'
sdf = pydarn.SDarnRead(in_fn)
rwr = sdf.read_rawacf()

New Feature

module: fan.py

package: plotting

Scope

Add geographical coordinates to:

• plot_fan
• plot_fov

Description

Currently plot_fov and plot_fan only plot in magnetic coordinates, they should also be able to provide geographic coordinates as well.

pyDARN Checklist

• Does this fit within pyDARN's scope?
• Is this a minor change i.e., new small utility function?
• Is this a major change i.e., new plotting function?
• Do you need help developing it?
• Have you created a pyDARN GitHub project to show the tasks needed to be done?

Update: I have been working on developer guidelines and I am hoping to have out for review by the next meeting which should be in the next 3 weeks (after CEDAR conference).

As mentioned in #24 the scaling gets messed up when using lmfit2 data:

import pydarn
import matplotlib.pyplot as plt 

fitacf_file = "20120614.0401.1200.sas.lmfit2"
darn_read = pydarn.SDarnRead(fitacf_file)
fitacf_data = darn_read.read_fitacf()

filts = {'min_scalar_filter':{'tfreq': 12500}}
fig = plt.figure(figsize=(10,8))
ax1 = fig.add_subplot(111)
pydarn.RTP.plot_range_time(fitacf_data, parameter='p_l',
                           beam_num=12, ax=ax1,
                           filter_settings=filts)

Documentation

title or suggestion of documentation

Purpose

• update
• new
• remove
• typo/grammar change
• markdown improvement

Category

• code documentation
• wiki
• README
• installation guide
  • OS specifics
  • dependency
  • environment
• tutorials
  • plotting
  • io
  • utils/exceptions
• examples
• citing/licensing
• trouble shooting
  • installation
  • tutorials
• history
• guidelines
  • developer
  • communication

Details

Adding workflow documentation similar to RST

• issues
• projects
• Release
• workflow

New Feature

Name: plot_map

module: map.py

package: /plottting

Scope

• plot map data
• plot in MLT magnetic polar coordinates
• option to change parameters to plot
• fov coverage
• information KP voltage diff

Description

This feature will focus on the vectors and the other parts of the plotting will come.

pyDARN Checklist

• Does this fit within pyDARN's scope?
• Is this a minor change i.e., new small utility function?
• Is this a major change i.e., new plotting function?
• Do you need help developing it?
• Have you created a pyDARN GitHub project to show the tasks needed to be done?

User Interface

plot_map(map_data)

It's not very critical, but should the version badge be (or have been) updated to match the 1.0 release? Guess it's too late now to get into Zenodo, but something to watch for next time for a release.

Feature request: make pydarn pip installable from git link.

This issue is addressing the potentially problem a user may rarely run into a missing hardware file.
Details are in #59 and discuss how one can fix it will be laid out in the pyDARN troubleshooting page.

As this is mentioned and shown how various installation methods may lead to errors example #59. Also, to someone not very familiar with github may not know how to switch branches easily.

I noticed that pydarn will incorrectly classify an input bz2 file as having an issue with the first record.
If it's not the intention of pydarn to handle compressed files, then it should produce an error message saying so. If it's the intention of pydarn to handle compressed files, it should uncompress it first.

import pydarn
in_fn = '/home/kevin/PycharmProjects/pydarn/20090101.1201.00.sas.rawacf.bz2'
sd = pydarn.SDarnRead(in_fn)
r = sd.read_rawacf()
Error: /home/kevin/PycharmProjects/pydarn/20090101.1201.00.sas.rawacf.bz2 contains an block size 643383601 > remaining bytes 13381653 at record = 0.
Traceback (most recent call last):
File "", line 1, in
File "/home/kevin/PycharmProjects/pydarn/pydarn/io/superdarn.py", line 421, in read_rawacf
self._read_darn_records(file_struct_list)
File "/home/kevin/PycharmProjects/pydarn/pydarn/io/superdarn.py", line 372, in _read_darn_records
self._read_darn_record(format_fields)
File "/home/kevin/PycharmProjects/pydarn/pydarn/io/superdarn.py", line 347, in _read_darn_record
record = self.read_record()
File "/home/kevin/PycharmProjects/pydarn/pydarn/io/dmap.py", line 449, in read_record
remaining_bytes, "remaining bytes")
File "/home/kevin/PycharmProjects/pydarn/pydarn/io/dmap.py", line 287, in bytes_check
self.rec_num)
pydarn.exceptions.dmap_exceptions.MismatchByteError: Error: /home/kevin/PycharmProjects/pydarn/20090101.1201.00.sas.rawacf.bz2 contains an block size 643383601 > remaining bytes 13381653 at record = 0.

As pointed out in #24 lmfit2 data plots all the data in the range gates, unlike the conventional SuperDARN FITACF DMap. When filtering applied striping is more prevalent in the plot:

import pydarn
import matplotlib.pyplot as plt 

fitacf_file = "20120614.0401.1200.sas.lmfit2"
darn_read = pydarn.SDarnRead(fitacf_file)
fitacf_data = darn_read.read_fitacf()

filts = {'min_scalar_filter':{'tfreq': 12500}}
fig = plt.figure(figsize=(10,8))
ax1 = fig.add_subplot(111)
pydarn.RTP.plot_range_time(fitacf_data, parameter='p_l',
                           beam_num=12, ax=ax1,
                           filter_settings=filts,
                           zmin=1,zmax=30)


plt.show()

As mentioned by @asreimer , @egthomas, and @billetd , users tend to plot velocity rang-time parameter plots most of the time, making the default colormap

Unsensible as it is not red-blue shifting colormap.

The suggestion is to change the default colormap to match for velocity to ensure conventions are met, and users are not confused. As well as my own suggestion to make the default parameter also velocity if it is the most commonly plotted parameter.

This is currently being under review in PR #25
@billetd we will also need to update the tutorials.

Side note: pyDARN does not follow exact conventions as DaVitpy, and should not be directly compared to. Unlike DaVitpy, pyDARN does not do parameter detection for colormaps and boundaries in plot_range_time to ensure future-proofing if parameter names change. This lower future maintenance of the code.

BUG

When using plot_fan or plot_fov we get an UserWarning from matplotlib using set_xtickslabels but not setting the ticks beforehand. Because this is with a polar plot some thought needs to be given.

Priority

• Minor - typo, incorrect naming, fails on specific uncommon situation
• Major - misrepresentation of data, failure in a common situation
• Urgent - failure in usage, large misrepresentation

Example of the bug

294: UserWarning: FixedFormatter should only be used together with FixedLocator
287: MatplotlibDeprecationWarning: Adding an axes using the same arguments as a previous axes currently reuses the earlier instance.  In a future version, a new instance will always be created and returned.  Meanwhile, this warning can be suppressed, and the future behavior ensured, bypassing a unique label to each axes instance.

Potential Bug Location

From @aburrell request on PR #19 to add altitude values to the list.

@DannoPeters can you list out some steps on how to do this or possible options?

the pydarn.DarnRead output doesn't have a user friendly object interface to let you know whether or not you successfully read in data.

import pydarn
fitacf_file = '20170120.0001.00.inv.fitacf'
sd_read = pydarn.DarnRead(fitacf_file)
print(sd_read)
<pydarn.io.superdarn.DarnRead object at 0x129385860>

Hey all, I'm largely a n00b here, so sorry if this has been covered elsewhere, though I couldn't find it anywhere. Has anyone looked at pydarn in the new Ubuntu20? I know it's not released yet...but will be soon.

Anywho, I was using a VM with that and think I didn't skimp on the software packages to be installed. But starting with the install instructions on the readthedocs I didn't have pip3 installed even though I have Python3.8.2. Anyone else run into that or is that maybe from using a pre-release of the OS?

EDIT: Also didn't come standard with git. These seem to both be needed to run:
pip3 install --user git+https://github.com/superdarn/pydarn@develop