nasa / podaacpy Goto Github PK

View Code? Open in Web Editor NEW

74.0 13.0 45.0 1.26 MB

A python utility library for interacting with NASA JPL's PO.DAAC

Home Page: http://podaacpy.readthedocs.org/en/latest/

License: Apache License 2.0

Python 100.00%

nasa nasa-data science oceanography metadata python podaac

podaacpy's Introduction

podaacpy

podaacpy has been retired please use https://github.com/podaac/data-subscriber

A python utility library for interacting with NASA JPL's PO.DAAC

Software DOI

If you are using Podaacpy in your research, please consider citing the software . This DOI represents all versions, and will always resolve to the latest one. If you wish to reference actual versions, then please find the appropriate DOI's over at Zenodo.

What is PO.DAAC?

The Physical Oceanography Distributed Active Archive Center (PO.DAAC) is an element of the
Earth Observing System Data and Information System (EOSDIS).
The EOSDIS provides science data to a wide community of users for NASA's Science Mission Directorate.

What does podaacpy offer?

The library provides a Python toolkit for interacting with all PO.DAAC Web Services v3.2.2 APIs, namely

PO.DAAC Web Services: services include
Dataset Metadata
- retrieves the metadata of a dataset
Granule Metadata
- retrieves the metadata of a granule
Search Dataset
- searches PO.DAAC's dataset catalog, over Level 2, Level 3, and Level 4 datasets
Search Granule
- does granule searching on PO.DAAC level 2 swath datasets (individual orbits of a satellite), and level 3 & 4 gridded datasets (time averaged to span the globe)
Image Granule -renders granules in the PO.DAAC's catalog to images such as jpeg and/or png
Extract Granule
- subsets a granule in PO.DAAC catalog and produces either netcdf3 or hdf4 files
Metadata Compliance Checker: an online tool and web
service designed to check and validate the contents of netCDF and HDF granules for the
Climate and Forecast (CF) and Attribute Convention for Dataset Discovery (ACDD) metadata conventions.
Level 2 Subsetting: allows users to subset and download popular PO.DAAC level 2 (swath) datasets.
PO.DAAC Drive: an HTTP based data access service. PO.DAAC Drive replicates much of the functionality of FTP while addressing many of its issues.

Additionally, Podaacpy provides the following ocean-related data services

NASA OceanColor Web:
File Search - locate publically available files within the NASA Ocean Data Processing System (ODPS)
Bulk data downloads via HTTP - mimic FTP bulk data downloads using the HTTP-based data distribution server.

Installation

From the cheeseshop

pip3 install podaacpy

or from conda

conda install -c conda-forge podaacpy

or from source

git clone https://github.com/nasa/podaacpy.git && cd podaacpy
python3 setup.py install

Quickstart

Check out the examples directory for our Jupyter notebook examples.

Tests

podaacpy uses the popular nose testing suite for unit tests.
You can run the podaacpy tests simply by running

nosetests

Additonally, click on the build sticker at the top of this readme to be directed to the most recent build on travis-ci.

Documentation

You can view the documentation online at

http://podaacpy.readthedocs.org/en/latest/

Alternatively, you can build the documentation manually as follows

cd docs && make html

Documentation is then available in docs/build/html/

Community, Support and Development

Please open a ticket in the issue tracker.
Please use labels to
classify your issue.

License

podaacpy is licensed permissively under the Apache License v2.0.
A copy of that license is distributed with this software.

Copyright and Export Classification

Copyright 2016-2019, by the California Institute of Technology. ALL RIGHTS RESERVED. 
United States Government Sponsorship acknowledged. Any commercial use must be 
negotiated with the Office of Technology Transfer at the California Institute 
of Technology.
This software may be subject to U.S. export control laws. By accepting this software, 
the user agrees to comply with all applicable U.S. export laws and regulations. 
User has the responsibility to obtain export licenses, or other export authority 
as may be required before exporting such information to foreign countries or 
providing access to foreign persons.

podaacpy's People

Contributors

Stargazers

Watchers

podaacpy's Issues

Add dependencies to setup.py

We need to add the dependencies to setup.py.

future dependency missing in podaacpy conda recipe

Smoke tests are failing in ocw Travis CI builds with podaacpy because the dependency future is missing from the recipe. This will also be a good opportunity to test the process of updating recipe files in conda-forge.

Add requires.io support

https://requires.io/

Incorrect URL instantiation in podaacp.py

Hi @Omkar20895 this URL should be initiated to HTTPS right?

podaacpy/podaac/podaac.py

Line 32 in 986d0aa

self.URL = 'http://podaac.jpl.nasa.gov/ws/'

class Podaac:

    def __init__(self):
        self.URL = 'http://podaac.jpl.nasa.gov/ws/'

load_image_granule() does not exist

The Image Granule web service is not returning any content upon querying.
I have opened a thread here [0] and hopefully it should resolve soon.
The podaac service is quite unstable and it turning out to be PITA for us :P

[0] https://podaac.jpl.nasa.gov/forum/viewtopic.php?f=6&t=384

Release of podaacpy version-1.7.0

Lots of new features have been addressed since the last release and I think this is a perfect time to release a new version.

@lewismc

Documentation build fails locally

I tried to "make html" in the docs folder, however I got the following error:

Theme error:
no theme named 'sphinx_rtd_theme' found (missing theme.conf?)
make: *** [html] Error 1

Upgrade Podaacpy to PO.DAAC Web Services v3.2.2

August 25, 2016
We are pleased to announce the new version (v3.2.2) of the PO.DAAC Web Services now available online at http://podaac.jpl.nasa.gov/ws

PO.DAAC Web Services have been modified to conform to an updated API in the new HiTIDE/L2SS. Also, there are significant changes in the Granule Subset web service. The Granule Subset web service is now asynchronous, which prevents web timeouts and allows the subsetting of multiple granules in a single request. A Granule Subset request is submitted, using http POST rather than GET, and a token is returned that can be used in the new Subset Status web service to determine the state of the Granule Subset request. There is also a new Dataset Variables web service.

For questions or to provide feedback, please visit the PO.DAAC Web Services Forum

Downloading subsetted dataset from PO.DAAC

This issue is raised to track the issue raised in the PO.DAAC forum here.

Add recipe to conda-forge

Hi @lewismc and @Omkar20895

In order to complete our mission to have OCW on conda forge, we need to ensure that all of its dependencies are also available on conda-forge. As of today, I believe podaacpy is the only one that is outstanding. Let me know if you would like me to push a podaacpy recipe to conda-forge, and if so whether or not you wish to be listed as maintainers. Additionally, the following changes need to be made in preparation for this undertaking:

In setup.py, remove coveralls and any other dependencies not needed for runtime. This should be done to reduce bloat and to make it easier to build the package from the recipe in case some dependencies are missing from the main anaconda or conda-forge channels.
zip_safe should be set to False.
Add a MANIFEST.in which includes the LICENSE file.

The latter two are needed since the maintainers of conda-forge want to make sure all packages on their channel include copies of the LICENSE file with each distribution in order to comply with the terms of the license. I can submit the PR for all of these if need be. Once done it would be ideal to also upgrade the package on PyPI to ensure the source distribution downloaded by conda contains the LICENSE file.

Increase the coverage of the test cases.

This is really important because coverage implies stability of the code base.

Tests failing in master

When I pull master branch and test it I get the following

lmcgibbn@LMC-032857 /usr/local/podaacpy(master) $ nosetests
..EEEEEEEEEEEEEEE
======================================================================
ERROR: test_extract_granule (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 56, in test_extract_granule
    result = podaac.extract_granule()
AttributeError: 'module' object has no attribute 'extract_granule'

======================================================================
ERROR: test_list_available_extract_granule_datasetIds (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 91, in test_list_available_extract_granule_datasetIds
    result = podaac.list_available_extract_granule_datasetIds()
AttributeError: 'module' object has no attribute 'list_available_extract_granule_datasetIds'

======================================================================
ERROR: test_list_available_extract_granule_datasetShortNames (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 96, in test_list_available_extract_granule_datasetShortNames
    result = podaac.list_available_extract_granule_datasetShortNames()
AttributeError: 'module' object has no attribute 'list_available_extract_granule_datasetShortNames'

======================================================================
ERROR: test_list_available_granule_search_datasetIds (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 61, in test_list_available_granule_search_datasetIds
    result = podaac.list_available_granule_search_datasetIds()
AttributeError: 'module' object has no attribute 'list_available_granule_search_datasetIds'

======================================================================
ERROR: test_list_available_granule_search_datasetShortNames (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 66, in test_list_available_granule_search_datasetShortNames
    result = podaac.list_available_granule_search_datasetShortNames()
AttributeError: 'module' object has no attribute 'list_available_granule_search_datasetShortNames'

======================================================================
ERROR: test_list_available_granule_search_level2_datasetIds (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 71, in test_list_available_granule_search_level2_datasetIds
    result = podaac.list_available_granule_search_level2_datasetIds()
AttributeError: 'module' object has no attribute 'list_available_granule_search_level2_datasetIds'

======================================================================
ERROR: test_list_available_granule_search_level2_datasetShortNames (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 76, in test_list_available_granule_search_level2_datasetShortNames
    result = podaac.list_available_granule_search_level2_datasetShortNames()
AttributeError: 'module' object has no attribute 'list_available_granule_search_level2_datasetShortNames'

======================================================================
ERROR: test_list_available_image_granule_datasetIds (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 81, in test_list_available_image_granule_datasetIds
    result = podaac.list_available_image_granule_datasetIds()
AttributeError: 'module' object has no attribute 'list_available_image_granule_datasetIds'

======================================================================
ERROR: test_list_available_image_granule_datasetShortNames (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 86, in test_list_available_image_granule_datasetShortNames
    result = podaac.list_available_image_granule_datasetShortNames()
AttributeError: 'module' object has no attribute 'list_available_image_granule_datasetShortNames'

======================================================================
ERROR: test_load_dataset_md (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 26, in test_load_dataset_md
    result = podaac.load_dataset_md()
AttributeError: 'module' object has no attribute 'load_dataset_md'

======================================================================
ERROR: test_load_granule_md (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 31, in test_load_granule_md
    result = podaac.load_granule_md()
AttributeError: 'module' object has no attribute 'load_granule_md'

======================================================================
ERROR: test_load_image_granule (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 51, in test_load_image_granule
    result = podaac.load_image_granule()
AttributeError: 'module' object has no attribute 'load_image_granule'

======================================================================
ERROR: test_load_last24hours_datacasting_granule_md (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 36, in test_load_last24hours_datacasting_granule_md
    result = podaac.load_last24hours_datacasting_granule_md()
AttributeError: 'module' object has no attribute 'load_last24hours_datacasting_granule_md'

======================================================================
ERROR: test_search_dataset (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 41, in test_search_dataset
    result = podaac.search_dataset()
AttributeError: 'module' object has no attribute 'search_dataset'

======================================================================
ERROR: test_search_granule (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 46, in test_search_granule
    result = podaac.search_granule()
AttributeError: 'module' object has no attribute 'search_granule'

----------------------------------------------------------------------
Ran 17 tests in 8.326s

FAILED (errors=15)
lmcgibbn@LMC-032857 /usr/local/podaacpy(master) $

Access Denied for Dataset URL

The OCW builds and podaac builds (try running a fresh one) are failing when urllib tries to access the dataset at the following url.

http://podaac-opendap.jpl.nasa.gov/opendap/allData/ghrsst/data/GDS2/L4/GLOB/CMC/CMC0.2deg/v2/1991/245/19910902120000-CMC-L4_GHRSST-SSTfnd-CMC0.2deg-GLOB-v02.0-fv02.0.nc

Fix documentation issues

http://podaacpy.readthedocs.io/en/latest/webservices.html

This is not correct and needs to be resolved.
Building and testing the documentation locally is also documented at https://github.com/lewismc/podaacpy#documentation

Version miss-match in condaforge and pypi

The condaforge hosts 1.6.0 version of podaacpy and pypi hosts 1.7.0 version of podaacpy.

Add PyPI username and password in Travis.yml

@lewismc Please add your username and password (encrypted) to .travis.yml file.

You'll need travis cli installed (instructions here)
Encrypted password can be generated using travis encrypt 'plain-text-password'

extract_l4_granule granule name split results in IndexError: list index out of range

The following code is buggy and results in the list index out of range.

granule_name = granule_name.split('\t')[3][:-1]
IndexError: list index out of range

PR coming up.

Verify that MCC works with OPeNDAP URLs

Let's write some more tests for the MCC which work with the following URLs

https://gpm1.gesdisc.eosdis.nasa.gov:443/opendap/GPM_L3/GPM_3IMERGM.03/2014/3B-MO.MS.MRG.3IMERG.20140312-S000000-E235959.03.V03D.HDF5

https://goldsmr4.gesdisc.eosdis.nasa.gov:443/opendap/MERRA2_MONTHLY/M2IMNXINT.5.12.4/1980/MERRA2_100.instM_2d_int_Nx.198001.nc4

http://opendap.jpl.nasa.gov:80/opendap/allData/smap/L3/RSS/V2/monthly/SCI/2015/RSS_smap_SSS_monthly_2015_04_v02.0.nc

Updating all the badges in the readme

The present travis-ci is pointing to lewismc/podaacpy but it should point to nasa/podaacpy

Add return types to all functions

Right now we do not have return types for any functions, documentation could be improved if these were provided
http://www.sphinx-doc.org/en/stable/domains.html#info-field-lists

Unwanted stdout of exceptions in travis-ci-build.

Travis build shows up some exceptions that are raised knowingly by me as a part of testing the exceptions in podaac module. Please take a look at the image below for reference:

A simple solution can be removing the --nocapture flag from travis nosetests command.

Tests failing in master

Hi @Omkar20895 the following tests are failing. Can you take a look. This is running off of a clean master branch.

lmcgibbn@LMC-032857 /usr/local/podaacpy(master) $ nosetests
.......EE......EEEEEEEEEEEEEEE
======================================================================
ERROR: podaac.nose_tests_podaac.podaac_test.test_list_available_granule_search_datasetIds
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/usr/local/podaacpy/podaac/nose_tests_podaac/podaac_test.py", line 81, in test_list_available_granule_search_datasetIds
    data = podaac.list_available_granule_search_datasetIds()
  File "/usr/local/podaacpy/podaac/podaac.py", line 458, in list_available_granule_search_datasetIds
    json = requests.get("http://podaac.jpl.nasa.gov/dmasSolr/solr/dataset/select/?q=*:*&fl=Dataset-PersistentId,Dataset-ShortName-Full&rows=2147483647&fq=DatasetPolicy-AccessType-Full:(OPEN+OR+PREVIEW+OR+SIMULATED+OR+REMOTE)+AND+DatasetPolicy-ViewOnline:Y&wt=json").json()
  File "/Library/Python/2.7/site-packages/requests/models.py", line 741, in json
    return json.loads(self.text, **kwargs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 365, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 383, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
-------------------- >> begin captured logging << --------------------
requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): podaac.jpl.nasa.gov
requests.packages.urllib3.connectionpool: DEBUG: "GET /dmasSolr/solr/dataset/select/?q=*:*&fl=Dataset-PersistentId,Dataset-ShortName-Full&rows=2147483647&fq=DatasetPolicy-AccessType-Full:(OPEN+OR+PREVIEW+OR+SIMULATED+OR+REMOTE)+AND+DatasetPolicy-ViewOnline:Y&wt=json HTTP/1.1" 503 323
--------------------- >> end captured logging << ---------------------

======================================================================
ERROR: podaac.nose_tests_podaac.podaac_test.test_list_available_granule_search_datasetShortNames
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/usr/local/podaacpy/podaac/nose_tests_podaac/podaac_test.py", line 89, in test_list_available_granule_search_datasetShortNames
    data = podaac.list_available_granule_search_datasetShortNames()
  File "/usr/local/podaacpy/podaac/podaac.py", line 488, in list_available_granule_search_datasetShortNames
    json = requests.get("http://podaac.jpl.nasa.gov/dmasSolr/solr/dataset/select/?q=*:*&fl=Dataset-PersistentId,Dataset-ShortName-Full&rows=2147483647&fq=DatasetPolicy-AccessType-Full:(OPEN+OR+PREVIEW+OR+SIMULATED+OR+REMOTE)+AND+DatasetPolicy-ViewOnline:Y&wt=json").json()
  File "/Library/Python/2.7/site-packages/requests/models.py", line 741, in json
    return json.loads(self.text, **kwargs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 365, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 383, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
-------------------- >> begin captured logging << --------------------
requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): podaac.jpl.nasa.gov
requests.packages.urllib3.connectionpool: DEBUG: "GET /dmasSolr/solr/dataset/select/?q=*:*&fl=Dataset-PersistentId,Dataset-ShortName-Full&rows=2147483647&fq=DatasetPolicy-AccessType-Full:(OPEN+OR+PREVIEW+OR+SIMULATED+OR+REMOTE)+AND+DatasetPolicy-ViewOnline:Y&wt=json HTTP/1.1" 503 323
--------------------- >> end captured logging << ---------------------

======================================================================
ERROR: test_extract_granule (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 56, in test_extract_granule
    result = podaac.extract_granule()
AttributeError: 'module' object has no attribute 'extract_granule'

======================================================================
ERROR: test_list_available_extract_granule_datasetIds (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 91, in test_list_available_extract_granule_datasetIds
    result = podaac.list_available_extract_granule_datasetIds()
AttributeError: 'module' object has no attribute 'list_available_extract_granule_datasetIds'

======================================================================
ERROR: test_list_available_extract_granule_datasetShortNames (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 96, in test_list_available_extract_granule_datasetShortNames
    result = podaac.list_available_extract_granule_datasetShortNames()
AttributeError: 'module' object has no attribute 'list_available_extract_granule_datasetShortNames'

======================================================================
ERROR: test_list_available_granule_search_datasetIds (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 61, in test_list_available_granule_search_datasetIds
    result = podaac.list_available_granule_search_datasetIds()
AttributeError: 'module' object has no attribute 'list_available_granule_search_datasetIds'

======================================================================
ERROR: test_list_available_granule_search_datasetShortNames (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 66, in test_list_available_granule_search_datasetShortNames
    result = podaac.list_available_granule_search_datasetShortNames()
AttributeError: 'module' object has no attribute 'list_available_granule_search_datasetShortNames'

======================================================================
ERROR: test_list_available_granule_search_level2_datasetIds (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 71, in test_list_available_granule_search_level2_datasetIds
    result = podaac.list_available_granule_search_level2_datasetIds()
AttributeError: 'module' object has no attribute 'list_available_granule_search_level2_datasetIds'

======================================================================
ERROR: test_list_available_granule_search_level2_datasetShortNames (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 76, in test_list_available_granule_search_level2_datasetShortNames
    result = podaac.list_available_granule_search_level2_datasetShortNames()
AttributeError: 'module' object has no attribute 'list_available_granule_search_level2_datasetShortNames'

======================================================================
ERROR: test_list_available_image_granule_datasetIds (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 81, in test_list_available_image_granule_datasetIds
    result = podaac.list_available_image_granule_datasetIds()
AttributeError: 'module' object has no attribute 'list_available_image_granule_datasetIds'

======================================================================
ERROR: test_list_available_image_granule_datasetShortNames (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 86, in test_list_available_image_granule_datasetShortNames
    result = podaac.list_available_image_granule_datasetShortNames()
AttributeError: 'module' object has no attribute 'list_available_image_granule_datasetShortNames'

======================================================================
ERROR: test_load_dataset_md (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 26, in test_load_dataset_md
    result = podaac.load_dataset_md()
AttributeError: 'module' object has no attribute 'load_dataset_md'

======================================================================
ERROR: test_load_granule_md (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 31, in test_load_granule_md
    result = podaac.load_granule_md()
AttributeError: 'module' object has no attribute 'load_granule_md'

======================================================================
ERROR: test_load_image_granule (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 51, in test_load_image_granule
    result = podaac.load_image_granule()
AttributeError: 'module' object has no attribute 'load_image_granule'

======================================================================
ERROR: test_load_last24hours_datacasting_granule_md (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 36, in test_load_last24hours_datacasting_granule_md
    result = podaac.load_last24hours_datacasting_granule_md()
AttributeError: 'module' object has no attribute 'load_last24hours_datacasting_granule_md'

======================================================================
ERROR: test_search_dataset (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 41, in test_search_dataset
    result = podaac.search_dataset()
AttributeError: 'module' object has no attribute 'search_dataset'

======================================================================
ERROR: test_search_granule (podaac.tests.test_podaac.TestPodaacDataWebservices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/podaacpy/podaac/tests/test_podaac.py", line 46, in test_search_granule
    result = podaac.search_granule()
AttributeError: 'module' object has no attribute 'search_granule'

----------------------------------------------------------------------
Ran 30 tests in 19.359s

FAILED (errors=17)

Implement L2SS REST Interface Specification V1

Having just spoken with @AaronPlave regarding the PO.DAAC L2SS REST Interface Specification V1, he provided me with documentation which I've made available on our wiki.
An example of this API in use can be seen at the HiTIDE application for Level2 subsetting.

Update documentation, README and ipynb to accomodate L2SS API

Once we've addressed #64 and #65 we need to update all of the documentation and ensure it all builds well.
The ipynb also needs to be updated so that people can take it for a run.
We should also update the .ipynb because the functions will all have changed.

PODAAC Solr search server down

The server @ http://podaac.jpl.nasa.gov/dmasSolr/solr/ is not up yet.
I have posted a thread in the forum @ https://podaac.jpl.nasa.gov/forum/viewtopic.php?f=6&t=367
We need to await the reply in the thread and if there is none then I think we need to find an alternative.

Granule search return HTTP Status - 500

I have opened an issue in podaac forum for the same. Here is the link.

Update Travis CI Build

Make nosetests more verbal by adding -v flag
Fix testpypi deployment steps.
Cache pip dependencies
Disable email notification

Unable to find [URL Template] for accessing preview image

I have opened an issue here, please refer to this for further information.

podaac.extract_l4_granule documentation incorrect

Hi @Omkar20895 we need to remove the following documentation as the parameter does not exist in the function signature

podaacpy/podaac/podaac.py

Lines 578 to 581 in 986d0aa

  :param short_name: the shorter name for a dataset. \ 

  Either short_name or dataset_id is required for a \ 

  granule search. Example: ASCATA-L2-25km 

  :type short_name: :mod:`string`

        :param short_name: the shorter name for a dataset. \
                Either short_name or dataset_id is required for a \
                granule search. Example: ASCATA-L2-25km
        :type short_name: :mod:`string`

Incorrect 'startIndex' parameter in data_part2 within podaac_utils.py

Hi @Omkar20895 I see the following code as an example

        data_part1 = requests.get(
            self.URL + 'search/dataset/?format=atom&itemsPerPage=400').text
        data_part2 = requests.get(
            self.URL + 'search/dataset?startIndex=400&itemsPerPage=400&format=atom').text

Should the startIndex data_part2 not start from 401 seeing as data_part1 has the count until 400? Or does the first count start from 0 and finish at 309?

Discrepancies in the documentation.

There are some discrepancies in the Retrieve granule images, the field in the arguments to the function is given as 'styles' rather than 'style' and also there is a new unknown parameter 'transparency=true' which is used, this parameter causes the function to fail with an error

Fix documentation issues

After the API is completed, documentation would be nice

reading sources... [100%] index
/usr/local/podaac-python/docs/source/index.rst:11: WARNING: toctree contains reference to nonexisting document u'podaac/overview'
/usr/local/podaac-python/docs/source/index.rst:11: WARNING: toctree contains reference to nonexisting document u'podaac/webservices'
/usr/local/podaac-python/docs/source/index.rst:11: WARNING: toctree contains reference to nonexisting document u'podaac/mdcc'
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] index
writing additional files... genindex search
copying static files... WARNING: logo file 'podaac_logo.png' does not exist

... also the theme should be changed to something more appealing!

Edit all license headers

Change license headers to the following

   Copyright 2016 California Institute of Technology.

   Licensed under the Apache License, Version 2.0 (the "License");
   you may not use this file except in compliance with the License.
   You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.

Tika.verify

http://podaac-uat.jpl.nasa.gov/mcc/

#yeahbuddy @lewismc

Automated PyPI deployments using Travis CI

Update .travis.yml to automatically deploy new releases to PyPI.

Add PyPi version and downloads badge

We also need to add a PyPi version badge which looks like this

Error in installing podaacpy

When I tried to install podaacpy from the cheeseshop I am getting this error.
I cross checked this with Ibrahim, he too got the same error.

webservices.html is not working again.

Please refer to this link.
When I build the documentation locally it works just fine and webservices.html works with the expected content in it.

@lewismc

Could not write the contents of a file downloaded using python requests into .nc file

try:    
     url = 'http://podaac.jpl.nasa.gov/ws/extract/granule/?datasetId=PODAAC-ASOP2-25X01&shortName=ASCATA-L2-25km&granuleName=ascat_20130719_230600_metopa_35024_eps_o_250_2200_ovw.l2.nc&bbox=45,0,180,90&format=netcdf'
    granule = requests.get(url)
    if granule.status_code == 404 or granule.status_code == 400 or granule.status_code == 503 or granule.status_code == 408: 
        granule.raise_for_status()

    file = open(granuleName, 'w+')
    file.write(granule.text.encode("utf-8"))

except requests.exceptions.HTTPError as e:
    print e

When I try to execute the above code it creates a local file with the name ascat_20130719_230600_metopa_35024_eps_o_250_2200_ovw.l2_subsetted_.nc

But when I try to open it using the panoply tool(which is used to view .nc files) I am getting an error 'There was an error opening the dataset java.lang.NegativeArraySizeException'.

Address RATS Report

Hi @Omkar20895 the report below is from a release auditing tool which I ran the source code over. If you have some time can you please take a look at some of the issues so we can discuss them?
Thanks

Entries in perl database: 33
Entries in ruby database: 46
Entries in python database: 62
Entries in c database: 334
Entries in php database: 55


Analyzing 20160804.090751.29944/podaacpy-master/podaac/podaac.py
Analyzing 20160804.090751.29944/podaacpy-master/podaac/podaac_data_source.py
Analyzing 20160804.090751.29944/podaacpy-master/podaac/podaac_utils.py
Analyzing 20160804.090751.29944/podaacpy-master/podaac/__init__.py
Analyzing 20160804.090751.29944/podaacpy-master/podaac/mcc.py
Analyzing 20160804.090751.29944/podaacpy-master/podaac/tests/mcc_test.py
Analyzing 20160804.090751.29944/podaacpy-master/podaac/tests/__init__.py
Analyzing 20160804.090751.29944/podaacpy-master/podaac/tests/podaac_test.py
Analyzing 20160804.090751.29944/podaacpy-master/docs/source/conf.py
Analyzing 20160804.090751.29944/podaacpy-master/setup.py
RATS results.


Severity: Medium
Issue: remove
A function call is not being made here, but a reference is being made to a name that is normally a vulnerable function. It could be being assigned as a pointer to function. 
File: 20160804.090751.29944/podaacpy-master/podaac/podaac_utils.py
Lines: 39 60 141 163 185 207
File: 20160804.090751.29944/podaacpy-master/podaac/tests/podaac_test.py
Lines: 116 118 136
Severity: Medium
Issue: open
A function call is not being made here, but a reference is being made to a name that is normally a vulnerable function. It could be being assigned as a pointer to function. 
File: 20160804.090751.29944/podaacpy-master/podaac/mcc.py
Lines: 87
File: 20160804.090751.29944/podaacpy-master/setup.py
Lines: 55 62
Severity: Medium
Issue: read
A function call is not being made here, but a reference is being made to a name that is normally a vulnerable function. It could be being assigned as a pointer to function. 
File: 20160804.090751.29944/podaacpy-master/setup.py
Lines: 55
If the above area is blank, please see the RATSweb Processing Notes
Inputs detected at the following points

20160804.090751.29944/podaacpy-master/setup.py: Line 55: function read

Double check to be sure that all input accepted from an external data source does not exceed the limits of the variable being used to hold it. Also make sure that the input cannot be used in such a manner as to alter your program's behaviour in an undesirable way.


Total lines analyzed: 1861
Total time 0.008923 seconds
208562 lines per second

Master build failing

Build is failing with log available at
https://travis-ci.org/lewismc/podaacpy/builds/159646604

Worker information

hostname: travis-worker-gce-org-prod4-11:215bafa3-55bc-494e-a5db-a371dfa31af8

version: v2.3.1-34-g4ab376b https://github.com/travis-ci/worker/tree/4ab376ba587e12cb465474238c78e88e6600aca2

instance: testing-gce-82ec08f2-3217-468f-b728-1f30214a63b2:travis-ci-python-precise-1471814219

startup: 21.906132483s
system_info

Build system information

Build language: python

Build group: stable

Build dist: precise

Build id: 159646604

Job id: 159646605

travis-build version: ef82e8245

Build image provisioning date and time

Sun Aug 21 21:35:17 UTC 2016

Operating System Details

Distributor ID: Ubuntu

Description:    Ubuntu 12.04.5 LTS

Release:    12.04

Codename:   precise

Linux Version

3.13.0-92-generic

Cookbooks Version

f77e708 https://github.com/travis-ci/travis-cookbooks/tree/f77e708

Git version

git version 1.8.5.6

bash version

GNU bash, version 4.2.25(1)-release (x86_64-pc-linux-gnu)

Copyright (C) 2011 Free Software Foundation, Inc.

License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law.

GCC version

gcc (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3

Copyright (C) 2011 Free Software Foundation, Inc.

This is free software; see the source for copying conditions.  There is NO

warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

LLVM version

clang version 3.4 (tags/RELEASE_34/final)

Target: x86_64-unknown-linux-gnu

Thread model: posix

Pre-installed Ruby versions

ruby-2.2.5

Pre-installed Node.js versions

v0.10.36

Pre-installed Go versions

1.4.2

mysql --version

mysql  Ver 14.14 Distrib 5.5.50, for debian-linux-gnu (x86_64) using readline 6.2

Pre-installed PostgreSQL versions

9.1.23

9.2.18

9.3.14

9.4.9

Redis version

redis-server 3.0.6

riak version

2.0.2

memcached version

1.4.13

MongoDB version

MongoDB 2.4.14

CouchDB version

couchdb 1.6.1

Neo4j version

1.9.4

Cassandra version

2.0.9

ElasticSearch version

1.4.0

Installed Sphinx versions

2.0.10

2.1.9

2.2.6

Default Sphinx version

2.2.6

Installed Firefox version

firefox 38.4.0esr

PhantomJS version

1.9.8

ant -version

Apache Ant(TM) version 1.8.2 compiled on December 3 2011

mvn -version

Apache Maven 3.2.5 (12a6b3acb947671f09b81f49094c53f426d8cea1; 2014-12-14T17:29:23+00:00)

Maven home: /usr/local/maven

Java version: 1.7.0_80, vendor: Oracle Corporation

Java home: /usr/lib/jvm/java-7-oracle/jre

Default locale: en, platform encoding: UTF-8

OS name: "linux", version: "3.13.0-92-generic", arch: "amd64", family: "unix"

fix.CVE-2015-7547

$ export DEBIAN_FRONTEND=noninteractive

Reading package lists...

Building dependency tree...

Reading state information...

libc6 is already the newest version.

0 upgraded, 0 newly installed, 0 to remove and 19 not upgraded.
git.checkout

6.04s$ git clone --depth=50 --branch=1.1.0 https://github.com/lewismc/podaacpy.git lewismc/podaacpy

Cloning into 'lewismc/podaacpy'...

remote: Counting objects: 468, done.

remote: Compressing objects: 100% (165/165), done.

remote: Total 468 (delta 313), reused 453 (delta 302), pack-reused 0

Receiving objects: 100% (468/468), 420.04 KiB | 0 bytes/s, done.

Resolving deltas: 100% (313/313), done.

Checking connectivity... done.

Note: checking out '5ed58c2b74aa3de25a8d3c5b8c984983d945bb66'.

You are in 'detached HEAD' state. You can look around, make experimental

changes and commit them, and you can discard any commits you make in this

state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may

do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

$ cd lewismc/podaacpy

$ git checkout -qf 5ed58c2b74aa3de25a8d3c5b8c984983d945bb66

0.01s$ source ~/virtualenv/python3.5/bin/activate

cache.1

Setting up build cache

$ export CASHER_DIR=$HOME/.casher

0.06s$ Installing caching utilities

0.00s

1.14sattempting to download cache archive

fetching 1.1.0/cache-linux-precise-e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855--python-3.5.tgz

fetching 1.1.0/cache--python-3.5.tgz

fetching master/cache-linux-precise-e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855--python-3.5.tgz

found cache

cache.pip

0.00s

1.39sadding /home/travis/.cache/pip to cache

$ python --version

Python 3.5.2

$ pip --version

pip 8.1.2 from /home/travis/virtualenv/python3.5.2/lib/python3.5/site-packages (python 3.5)
install.1

7.61s$ sudo apt-get update

Hit http://us-central1.gce.archive.ubuntu.com precise Release.gpg

Hit http://us-central1.gce.archive.ubuntu.com precise-updates Release.gpg

Hit http://us-central1.gce.archive.ubuntu.com precise-backports Release.gpg

Hit http://us-central1.gce.archive.ubuntu.com precise Release

Hit http://us-central1.gce.archive.ubuntu.com precise-updates Release

Hit http://us-central1.gce.archive.ubuntu.com precise-backports Release

Hit http://us-central1.gce.archive.ubuntu.com precise/main Sources

Hit http://us-central1.gce.archive.ubuntu.com precise/restricted Sources

Hit http://us-central1.gce.archive.ubuntu.com precise/universe Sources

Hit http://us-central1.gce.archive.ubuntu.com precise/multiverse Sources

Hit http://us-central1.gce.archive.ubuntu.com precise/main amd64 Packages

Hit http://us-central1.gce.archive.ubuntu.com precise/restricted amd64 Packages

Hit http://us-central1.gce.archive.ubuntu.com precise/universe amd64 Packages

Hit http://us-central1.gce.archive.ubuntu.com precise/multiverse amd64 Packages

Hit http://us-central1.gce.archive.ubuntu.com precise/main i386 Packages

Hit http://apt.postgresql.org precise-pgdg Release.gpg

Hit http://downloads-distro.mongodb.org dist Release.gpg

Hit http://us-central1.gce.archive.ubuntu.com precise/restricted i386 Packages

Hit http://us-central1.gce.archive.ubuntu.com precise/universe i386 Packages

Hit http://apt.postgresql.org precise-pgdg Release

Hit http://us-central1.gce.archive.ubuntu.com precise/multiverse i386 Packages

Hit http://us-central1.gce.archive.ubuntu.com precise/main TranslationIndex

Hit http://us-central1.gce.archive.ubuntu.com precise/multiverse TranslationIndex

Hit http://us-central1.gce.archive.ubuntu.com precise/restricted TranslationIndex

Hit http://us-central1.gce.archive.ubuntu.com precise/universe TranslationIndex

Hit http://us-central1.gce.archive.ubuntu.com precise-updates/main Sources

Hit http://us-central1.gce.archive.ubuntu.com precise-updates/restricted Sources

Hit http://us-central1.gce.archive.ubuntu.com precise-updates/universe Sources

Hit http://us-central1.gce.archive.ubuntu.com precise-updates/multiverse Sources

Hit http://us-central1.gce.archive.ubuntu.com precise-updates/main amd64 Packages

Hit http://us-central1.gce.archive.ubuntu.com precise-updates/restricted amd64 Packages

Hit http://us-central1.gce.archive.ubuntu.com precise-updates/universe amd64 Packages

Hit http://us-central1.gce.archive.ubuntu.com precise-updates/multiverse amd64 Packages

Hit http://downloads-distro.mongodb.org dist Release

Hit http://apt.postgresql.org precise-pgdg/main amd64 Packages

Hit http://us-central1.gce.archive.ubuntu.com precise-updates/main i386 Packages

Hit http://us-central1.gce.archive.ubuntu.com precise-updates/restricted i386 Packages

Hit http://us-central1.gce.archive.ubuntu.com precise-updates/universe i386 Packages

Hit http://us-central1.gce.archive.ubuntu.com precise-updates/multiverse i386 Packages

Hit http://us-central1.gce.archive.ubuntu.com precise-updates/main TranslationIndex

Hit http://us-central1.gce.archive.ubuntu.com precise-updates/multiverse TranslationIndex

Hit http://us-central1.gce.archive.ubuntu.com precise-updates/restricted TranslationIndex

Hit http://us-central1.gce.archive.ubuntu.com precise-updates/universe TranslationIndex

Hit http://us-central1.gce.archive.ubuntu.com precise-backports/main Sources

Hit http://us-central1.gce.archive.ubuntu.com precise-backports/restricted Sources

Hit http://us-central1.gce.archive.ubuntu.com precise-backports/universe Sources

Hit http://us-central1.gce.archive.ubuntu.com precise-backports/multiverse Sources

Hit http://us-central1.gce.archive.ubuntu.com precise-backports/main amd64 Packages

Hit http://us-central1.gce.archive.ubuntu.com precise-backports/restricted amd64 Packages

Hit http://apt.postgresql.org precise-pgdg/main i386 Packages

Hit http://downloads-distro.mongodb.org dist/10gen amd64 Packages

Hit http://us-central1.gce.archive.ubuntu.com precise-backports/universe amd64 Packages

Hit http://us-central1.gce.archive.ubuntu.com precise-backports/multiverse amd64 Packages

Hit http://us-central1.gce.archive.ubuntu.com precise-backports/main i386 Packages

Hit http://us-central1.gce.archive.ubuntu.com precise-backports/restricted i386 Packages

Hit http://us-central1.gce.archive.ubuntu.com precise-backports/universe i386 Packages

Hit http://us-central1.gce.archive.ubuntu.com precise-backports/multiverse i386 Packages

Hit http://us-central1.gce.archive.ubuntu.com precise-backports/main TranslationIndex

Hit http://us-central1.gce.archive.ubuntu.com precise-backports/multiverse TranslationIndex

Hit http://us-central1.gce.archive.ubuntu.com precise-backports/restricted TranslationIndex

Hit http://us-central1.gce.archive.ubuntu.com precise-backports/universe TranslationIndex

Hit http://us-central1.gce.archive.ubuntu.com precise/main Translation-en

Hit http://us-central1.gce.archive.ubuntu.com precise/multiverse Translation-en

Hit http://us-central1.gce.archive.ubuntu.com precise/restricted Translation-en

Hit http://us-central1.gce.archive.ubuntu.com precise/universe Translation-en

Hit http://us-central1.gce.archive.ubuntu.com precise-updates/main Translation-en

Hit http://ppa.launchpad.net precise Release.gpg

Hit http://security.ubuntu.com precise-security Release.gpg

Ign http://apt.postgresql.org precise-pgdg/main TranslationIndex

Hit http://downloads-distro.mongodb.org dist/10gen i386 Packages

Hit http://us-central1.gce.archive.ubuntu.com precise-updates/multiverse Translation-en

Hit http://us-central1.gce.archive.ubuntu.com precise-updates/restricted Translation-en

Hit http://us-central1.gce.archive.ubuntu.com precise-updates/universe Translation-en

Hit http://us-central1.gce.archive.ubuntu.com precise-backports/main Translation-en

Hit http://us-central1.gce.archive.ubuntu.com precise-backports/multiverse Translation-en

Hit http://us-central1.gce.archive.ubuntu.com precise-backports/restricted Translation-en

Hit http://us-central1.gce.archive.ubuntu.com precise-backports/universe Translation-en

Ign http://downloads-distro.mongodb.org dist/10gen TranslationIndex

Hit http://ppa.launchpad.net precise Release.gpg

Hit http://security.ubuntu.com precise-security Release

Ign http://apt.postgresql.org precise-pgdg/main Translation-en_US

Ign http://apt.postgresql.org precise-pgdg/main Translation-en

Hit http://ppa.launchpad.net precise Release.gpg

Hit http://security.ubuntu.com precise-security/main Sources

Hit http://ppa.launchpad.net precise Release.gpg

Hit http://security.ubuntu.com precise-security/restricted Sources

Hit http://ppa.launchpad.net precise Release.gpg

Hit http://security.ubuntu.com precise-security/universe Sources

Hit http://ppa.launchpad.net precise Release.gpg

Hit http://security.ubuntu.com precise-security/multiverse Sources

Ign http://downloads-distro.mongodb.org dist/10gen Translation-en_US

Hit https://packagecloud.io precise Release.gpg

Hit http://ppa.launchpad.net precise Release.gpg

Hit http://security.ubuntu.com precise-security/main amd64 Packages

Ign http://downloads-distro.mongodb.org dist/10gen Translation-en

Hit http://ppa.launchpad.net precise Release

Hit https://packagecloud.io precise Release

Hit http://security.ubuntu.com precise-security/restricted amd64 Packages

Hit http://ppa.launchpad.net precise Release

Hit http://security.ubuntu.com precise-security/universe amd64 Packages

Hit http://ppa.launchpad.net precise Release

Hit https://packagecloud.io precise/main amd64 Packages

Hit http://security.ubuntu.com precise-security/multiverse amd64 Packages

Hit http://ppa.launchpad.net precise Release

Hit http://security.ubuntu.com precise-security/main i386 Packages

Hit https://packagecloud.io precise/main i386 Packages

Get:1 https://packagecloud.io precise/main TranslationIndex [162 B]

Ign https://packagecloud.io precise/main TranslationIndex

Hit http://ppa.launchpad.net precise Release

Hit http://security.ubuntu.com precise-security/restricted i386 Packages

Hit http://ppa.launchpad.net precise Release

Hit http://security.ubuntu.com precise-security/universe i386 Packages

Hit http://ppa.launchpad.net precise Release

Hit http://security.ubuntu.com precise-security/multiverse i386 Packages

Hit http://ppa.launchpad.net precise/main amd64 Packages

Hit http://security.ubuntu.com precise-security/main TranslationIndex

Ign https://packagecloud.io precise/main Translation-en_US

Ign https://packagecloud.io precise/main Translation-en

Hit http://ppa.launchpad.net precise/main i386 Packages

Hit http://security.ubuntu.com precise-security/multiverse TranslationIndex

Hit http://ppa.launchpad.net precise/main TranslationIndex

Hit http://security.ubuntu.com precise-security/restricted TranslationIndex

Hit http://ppa.launchpad.net precise/main amd64 Packages

Hit http://security.ubuntu.com precise-security/universe TranslationIndex

Hit http://ppa.launchpad.net precise/main i386 Packages

Hit http://security.ubuntu.com precise-security/main Translation-en

Hit http://ppa.launchpad.net precise/main TranslationIndex

Hit http://ppa.launchpad.net precise/main Sources

Hit http://security.ubuntu.com precise-security/multiverse Translation-en

Hit http://ppa.launchpad.net precise/main amd64 Packages

Hit http://security.ubuntu.com precise-security/restricted Translation-en

Hit http://ppa.launchpad.net precise/main i386 Packages

Hit http://security.ubuntu.com precise-security/universe Translation-en

Hit http://ppa.launchpad.net precise/main TranslationIndex

Hit http://ppa.launchpad.net precise/main amd64 Packages

Hit http://ppa.launchpad.net precise/main i386 Packages

Hit http://ppa.launchpad.net precise/main TranslationIndex

Hit http://ppa.launchpad.net precise/main amd64 Packages

Hit http://ppa.launchpad.net precise/main i386 Packages

Hit http://ppa.launchpad.net precise/main TranslationIndex

Hit http://ppa.launchpad.net precise/main amd64 Packages

Hit http://ppa.launchpad.net precise/main i386 Packages

Hit http://ppa.launchpad.net precise/main TranslationIndex

Hit http://ppa.launchpad.net precise/main amd64 Packages

Hit http://ppa.launchpad.net precise/main i386 Packages

Hit http://ppa.launchpad.net precise/main TranslationIndex

Hit http://ppa.launchpad.net precise/main Translation-en

Hit http://ppa.launchpad.net precise/main Translation-en

Hit http://ppa.launchpad.net precise/main Translation-en

Hit http://ppa.launchpad.net precise/main Translation-en

Hit http://ppa.launchpad.net precise/main Translation-en

Hit http://ppa.launchpad.net precise/main Translation-en

Hit http://ppa.launchpad.net precise/main Translation-en

Reading package lists... Done

install.2

4.26s$ sudo apt-get -y install python-dev python-pip

Reading package lists... Done

Building dependency tree       

Reading state information... Done

python-dev is already the newest version.

The following NEW packages will be installed:

  python-pip python-setuptools

0 upgraded, 2 newly installed, 0 to remove and 19 not upgraded.

Need to get 536 kB of archives.

After this operation, 1,467 kB of additional disk space will be used.

Get:1 http://us-central1.gce.archive.ubuntu.com/ubuntu/ precise/main python-setuptools all 0.6.24-1ubuntu1 [441 kB]

Get:2 http://us-central1.gce.archive.ubuntu.com/ubuntu/ precise/universe python-pip all 1.0-1build1 [95.1 kB]

Fetched 536 kB in 0s (27.5 MB/s)

Selecting previously unselected package python-setuptools.

(Reading database ... 81332 files and directories currently installed.)

Unpacking python-setuptools (from .../python-setuptools_0.6.24-1ubuntu1_all.deb) ...

Selecting previously unselected package python-pip.

Unpacking python-pip (from .../python-pip_1.0-1build1_all.deb) ...

Processing triggers for man-db ...

Setting up python-setuptools (0.6.24-1ubuntu1) ...

Setting up python-pip (1.0-1build1) ...

install.3

8.45s$ python setup.py install

running install

running bdist_egg

running egg_info

creating podaacpy.egg-info

writing podaacpy.egg-info/PKG-INFO

writing dependency_links to podaacpy.egg-info/dependency_links.txt

writing requirements to podaacpy.egg-info/requires.txt

writing top-level names to podaacpy.egg-info/top_level.txt

writing namespace_packages to podaacpy.egg-info/namespace_packages.txt

writing manifest file 'podaacpy.egg-info/SOURCES.txt'

reading manifest file 'podaacpy.egg-info/SOURCES.txt'

writing manifest file 'podaacpy.egg-info/SOURCES.txt'

installing library code to build/bdist.linux-x86_64/egg

running install_lib

running build_py

creating build

creating build/lib

creating build/lib/podaac

copying podaac/podaac_utils.py -> build/lib/podaac

copying podaac/mcc.py -> build/lib/podaac

copying podaac/__init__.py -> build/lib/podaac

copying podaac/podaac.py -> build/lib/podaac

creating build/lib/podaac/tests

copying podaac/tests/__init__.py -> build/lib/podaac/tests

copying podaac/tests/mcc_test.py -> build/lib/podaac/tests

copying podaac/tests/podaac_test.py -> build/lib/podaac/tests

creating build/bdist.linux-x86_64

creating build/bdist.linux-x86_64/egg

creating build/bdist.linux-x86_64/egg/podaac

copying build/lib/podaac/podaac_utils.py -> build/bdist.linux-x86_64/egg/podaac

copying build/lib/podaac/mcc.py -> build/bdist.linux-x86_64/egg/podaac

creating build/bdist.linux-x86_64/egg/podaac/tests

copying build/lib/podaac/tests/__init__.py -> build/bdist.linux-x86_64/egg/podaac/tests

copying build/lib/podaac/tests/mcc_test.py -> build/bdist.linux-x86_64/egg/podaac/tests

copying build/lib/podaac/tests/podaac_test.py -> build/bdist.linux-x86_64/egg/podaac/tests

copying build/lib/podaac/__init__.py -> build/bdist.linux-x86_64/egg/podaac

copying build/lib/podaac/podaac.py -> build/bdist.linux-x86_64/egg/podaac

byte-compiling build/bdist.linux-x86_64/egg/podaac/podaac_utils.py to podaac_utils.cpython-35.pyc

byte-compiling build/bdist.linux-x86_64/egg/podaac/mcc.py to mcc.cpython-35.pyc

byte-compiling build/bdist.linux-x86_64/egg/podaac/tests/__init__.py to __init__.cpython-35.pyc

byte-compiling build/bdist.linux-x86_64/egg/podaac/tests/mcc_test.py to mcc_test.cpython-35.pyc

byte-compiling build/bdist.linux-x86_64/egg/podaac/tests/podaac_test.py to podaac_test.cpython-35.pyc

byte-compiling build/bdist.linux-x86_64/egg/podaac/__init__.py to __init__.cpython-35.pyc

byte-compiling build/bdist.linux-x86_64/egg/podaac/podaac.py to podaac.cpython-35.pyc

creating build/bdist.linux-x86_64/egg/EGG-INFO

copying podaacpy.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO

copying podaacpy.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO

copying podaacpy.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO

copying podaacpy.egg-info/namespace_packages.txt -> build/bdist.linux-x86_64/egg/EGG-INFO

copying podaacpy.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO

copying podaacpy.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO

copying podaacpy.egg-info/zip-safe -> build/bdist.linux-x86_64/egg/EGG-INFO

creating dist

creating 'dist/podaacpy-1.1.0-py3.5.egg' and adding 'build/bdist.linux-x86_64/egg' to it

removing 'build/bdist.linux-x86_64/egg' (and everything under it)

Processing podaacpy-1.1.0-py3.5.egg

Copying podaacpy-1.1.0-py3.5.egg to /home/travis/virtualenv/python3.5.2/lib/python3.5/site-packages

Adding podaacpy 1.1.0 to easy-install.pth file

Installed /home/travis/virtualenv/python3.5.2/lib/python3.5/site-packages/podaacpy-1.1.0-py3.5.egg

Processing dependencies for podaacpy==1.1.0

Searching for coveralls

Reading https://pypi.python.org/simple/coveralls/

Best match: coveralls 1.1

Downloading https://pypi.python.org/packages/12/50/5c1034eb92e5bc3d824a3745ca9162f2e4846c6ab5f96dccb5f84f77e98f/coveralls-1.1.tar.gz#md5=3b4431f7d38ceb3ff4e47cb8361f3333

Processing coveralls-1.1.tar.gz

Writing /tmp/easy_install-jgk6a9by/coveralls-1.1/setup.cfg

Running coveralls-1.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-jgk6a9by/coveralls-1.1/egg-dist-tmp-u2hauzie

zip_safe flag not set; analyzing archive contents...

Moving coveralls-1.1-py3.5.egg to /home/travis/virtualenv/python3.5.2/lib/python3.5/site-packages

Adding coveralls 1.1 to easy-install.pth file

Installing coveralls script to /home/travis/virtualenv/python3.5.2/bin

Installed /home/travis/virtualenv/python3.5.2/lib/python3.5/site-packages/coveralls-1.1-py3.5.egg

Searching for beautifulsoup4

Reading https://pypi.python.org/simple/beautifulsoup4/

Best match: beautifulsoup4 4.5.1

Downloading https://pypi.python.org/packages/86/ea/8e9fbce5c8405b9614f1fd304f7109d9169a3516a493ce4f7f77c39435b7/beautifulsoup4-4.5.1.tar.gz#md5=994abd90e691beaf7d42c00ffb2f3a67

Processing beautifulsoup4-4.5.1.tar.gz

Writing /tmp/easy_install-5ynrcabe/beautifulsoup4-4.5.1/setup.cfg

Running beautifulsoup4-4.5.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-5ynrcabe/beautifulsoup4-4.5.1/egg-dist-tmp-xzwbcsxr

zip_safe flag not set; analyzing archive contents...

Moving beautifulsoup4-4.5.1-py3.5.egg to /home/travis/virtualenv/python3.5.2/lib/python3.5/site-packages

Adding beautifulsoup4 4.5.1 to easy-install.pth file

Installed /home/travis/virtualenv/python3.5.2/lib/python3.5/site-packages/beautifulsoup4-4.5.1-py3.5.egg

Searching for requests

Reading https://pypi.python.org/simple/requests/

Best match: requests 2.11.1

Downloading https://pypi.python.org/packages/2e/ad/e627446492cc374c284e82381215dcd9a0a87c4f6e90e9789afefe6da0ad/requests-2.11.1.tar.gz#md5=ad5f9c47b5c5dfdb28363ad7546b0763

Processing requests-2.11.1.tar.gz

Writing /tmp/easy_install-s62os5el/requests-2.11.1/setup.cfg

Running requests-2.11.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-s62os5el/requests-2.11.1/egg-dist-tmp-obmb7n2x

warning: no files found matching 'test_requests.py'

creating /home/travis/virtualenv/python3.5.2/lib/python3.5/site-packages/requests-2.11.1-py3.5.egg

Extracting requests-2.11.1-py3.5.egg to /home/travis/virtualenv/python3.5.2/lib/python3.5/site-packages

Adding requests 2.11.1 to easy-install.pth file

Installed /home/travis/virtualenv/python3.5.2/lib/python3.5/site-packages/requests-2.11.1-py3.5.egg

Searching for coverage>=3.6

Reading https://pypi.python.org/simple/coverage/

Best match: coverage 4.2

Downloading https://pypi.python.org/packages/f5/47/2e39972c41eb988d551bb2da25c4ee16ea78492ec1b6ffdf09f50cd09b92/coverage-4.2.zip#md5=9841c9e1ddc862144fda3fd7e5ba5106

Processing coverage-4.2.zip

Writing /tmp/easy_install-chp_mnk0/coverage-4.2/setup.cfg

Running coverage-4.2/setup.py -q bdist_egg --dist-dir /tmp/easy_install-chp_mnk0/coverage-4.2/egg-dist-tmp-872pkh7f

warning: no previously-included files found matching 'ci/appveyor.token'

no previously-included directories found matching 'doc/_build'

no previously-included directories found matching 'tests/eggsrc/build'

no previously-included directories found matching 'tests/eggsrc/dist'

no previously-included directories found matching 'tests/eggsrc/*.egg-info'

warning: no previously-included files matching '*.py[co]' found anywhere in distribution

creating /home/travis/virtualenv/python3.5.2/lib/python3.5/site-packages/coverage-4.2-py3.5-linux-x86_64.egg

Extracting coverage-4.2-py3.5-linux-x86_64.egg to /home/travis/virtualenv/python3.5.2/lib/python3.5/site-packages

Adding coverage 4.2 to easy-install.pth file

Installing coverage-3.5 script to /home/travis/virtualenv/python3.5.2/bin

Installing coverage3 script to /home/travis/virtualenv/python3.5.2/bin

Installing coverage script to /home/travis/virtualenv/python3.5.2/bin

Installed /home/travis/virtualenv/python3.5.2/lib/python3.5/site-packages/coverage-4.2-py3.5-linux-x86_64.egg

Searching for docopt>=0.6.1

Reading https://pypi.python.org/simple/docopt/

Best match: docopt 0.6.2

Downloading https://pypi.python.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz#md5=4bc74561b37fad5d3e7d037f82a4c3b1

Processing docopt-0.6.2.tar.gz

Writing /tmp/easy_install-a6h9wzg3/docopt-0.6.2/setup.cfg

Running docopt-0.6.2/setup.py -q bdist_egg --dist-dir /tmp/easy_install-a6h9wzg3/docopt-0.6.2/egg-dist-tmp-q3vqiwxn

zip_safe flag not set; analyzing archive contents...

Moving docopt-0.6.2-py3.5.egg to /home/travis/virtualenv/python3.5.2/lib/python3.5/site-packages

Adding docopt 0.6.2 to easy-install.pth file

Installed /home/travis/virtualenv/python3.5.2/lib/python3.5/site-packages/docopt-0.6.2-py3.5.egg

Finished processing dependencies for podaacpy==1.1.0

install.4

0.49s$ pip install requests bs4 coveralls

Requirement already satisfied (use --upgrade to upgrade): requests in /home/travis/virtualenv/python3.5.2/lib/python3.5/site-packages/requests-2.11.1-py3.5.egg

Collecting bs4

Requirement already satisfied (use --upgrade to upgrade): coveralls in /home/travis/virtualenv/python3.5.2/lib/python3.5/site-packages/coveralls-1.1-py3.5.egg

Requirement already satisfied (use --upgrade to upgrade): beautifulsoup4 in /home/travis/virtualenv/python3.5.2/lib/python3.5/site-packages/beautifulsoup4-4.5.1-py3.5.egg (from bs4)

Requirement already satisfied (use --upgrade to upgrade): docopt>=0.6.1 in /home/travis/virtualenv/python3.5.2/lib/python3.5/site-packages/docopt-0.6.2-py3.5.egg (from coveralls)

Requirement already satisfied (use --upgrade to upgrade): coverage>=3.6 in /home/travis/virtualenv/python3.5.2/lib/python3.5/site-packages/coverage-4.2-py3.5-linux-x86_64.egg (from coveralls)

Installing collected packages: bs4

Successfully installed bs4-0.0.1

34.62s$ nosetests  --with-coverage --cover-package=podaac -v

test_check_local_file (podaac.tests.mcc_test.testMCC) ... ok

test_check_remote_file (podaac.tests.mcc_test.testMCC) ... ok

test_dataset_metadata (podaac.tests.podaac_test.test_podaac) ... ok

test_dataset_search (podaac.tests.podaac_test.test_podaac) ... ok

test_dataset_variable (podaac.tests.podaac_test.test_podaac) ... ok

test_extract_l4_granule (podaac.tests.podaac_test.test_podaac) ... ok

test_granule_metadata (podaac.tests.podaac_test.test_podaac) ... ok

test_granule_preview (podaac.tests.podaac_test.test_podaac) ... ok

test_granule_search (podaac.tests.podaac_test.test_podaac) ... ERROR

test_list_available_extract_granule_dataset_ids (podaac.tests.podaac_test.test_podaac) ... ok

test_list_available_extract_granule_dataset_short_names (podaac.tests.podaac_test.test_podaac) ... ok

test_list_available_granule_search_dataset_ids (podaac.tests.podaac_test.test_podaac) ... ok

test_list_available_granule_search_dataset_short_names (podaac.tests.podaac_test.test_podaac) ... ok

test_list_available_granule_search_level2_dataset_ids (podaac.tests.podaac_test.test_podaac) ... ok

test_list_available_granule_search_level2_dataset_short_names (podaac.tests.podaac_test.test_podaac) ... ok

test_load_last24hours_datacasting_granule_md (podaac.tests.podaac_test.test_podaac) ... ok

test_subset_status (podaac.tests.podaac_test.test_podaac) ... ok

======================================================================

ERROR: test_granule_search (podaac.tests.podaac_test.test_podaac)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/home/travis/build/lewismc/podaacpy/podaac/tests/podaac_test.py", line 113, in test_granule_search

    dataset_id=test_dataset_id, start_time=start_time, end_time=end_time, bbox=bbox, start_index=start_index, format=format)

  File "/home/travis/build/lewismc/podaacpy/podaac/podaac.py", line 396, in granule_search

    granules.raise_for_status()

  File "/home/travis/virtualenv/python3.5.2/lib/python3.5/site-packages/requests-2.11.1-py3.5.egg/requests/models.py", line 862, in raise_for_status

    raise HTTPError(http_error_msg, response=self)

requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http://podaac.jpl.nasa.gov/ws/search/granule/?datasetId=PODAAC-ASOP2-25X01&startTime=2013-01-01T01:30:00Z&endTime=2014-01-01T00:00:00Z&bbox=-45,-45,45,45&startIndex=1&sortBy=timeAsc&itemsPerPage=7&format=atom&pretty=True

-------------------- >> begin captured stdout << ---------------------

404 Client Error: Not Found for url: http://podaac.jpl.nasa.gov/ws/search/granule/?datasetId=PODAAC-ASOP2-25X01&startTime=2013-01-01T01:30:00Z&endTime=2014-01-01T00:00:00Z&bbox=-45,-45,45,45&startIndex=1&sortBy=timeAsc&itemsPerPage=7&format=atom&pretty=True

--------------------- >> end captured stdout << ----------------------

-------------------- >> begin captured logging << --------------------

requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): podaac.jpl.nasa.gov

requests.packages.urllib3.connectionpool: DEBUG: "GET /ws/search/granule/?datasetId=PODAAC-ASOP2-25X01&startTime=2013-01-01T01:30:00Z&endTime=2014-01-01T00:00:00Z&bbox=-45,-45,45,45&startIndex=1&sortBy=timeAsc&itemsPerPage=7&format=atom&pretty=True HTTP/1.1" 404 None

--------------------- >> end captured logging << ---------------------

Name                     Stmts   Miss  Cover

--------------------------------------------

podaac.py                    0      0   100%

podaac/mcc.py               30      3    90%

podaac/podaac.py           197     32    84%

podaac/podaac_utils.py      79      0   100%

--------------------------------------------

TOTAL                      306     35    89%

----------------------------------------------------------------------

Ran 17 tests in 34.276s

FAILED (errors=1)

The command "nosetests  --with-coverage --cover-package=podaac -v" exited with 1.
cache.2

store build cache

0.00s

3.05schange detected (content changed, file is created, or file is deleted):

/home/travis/.cache/pip/http/a/9/6/3/0/a96306593491a3f88cd22cf2931643c4029989f4dd25a28dac4da73e

changes detected, packing new archive

uploading archive

Done. Your build exited with 1.

Subset and download l3 and l4 subsetted datasets

I have been investigating how to subset and download l3 and l4 datasets, since I am new to this I have been facing some confusions.
I am discussing over this[0] thread in the podaac forum.
This should be hopefully solved soon.

[0] https://podaac.jpl.nasa.gov/forum/viewtopic.php?f=5&t=219

PODAAC site can't be reached.

There was some discrepancy in the podaac extract_granule web service.
I have opened a ticket here [0]

[0] https://podaac.jpl.nasa.gov/forum/viewtopic.php?f=6&t=387

byte-compiling build/bdist.macosx-10.6-intel/egg/podaac/mcc.py to mcc.cpython-35.pyc
  File "build/bdist.macosx-10.6-intel/egg/podaac/mcc.py", line 56
    print error
              ^
SyntaxError: Missing parentheses in call to 'print'

byte-compiling build/bdist.macosx-10.6-intel/egg/podaac/nose_tests_mcc/__init__.py to __init__.cpython-35.pyc
byte-compiling build/bdist.macosx-10.6-intel/egg/podaac/nose_tests_mcc/mcc_test.py to mcc_test.cpython-35.pyc
byte-compiling build/bdist.macosx-10.6-intel/egg/podaac/nose_tests_podaac/__init__.py to __init__.cpython-35.pyc
byte-compiling build/bdist.macosx-10.6-intel/egg/podaac/nose_tests_podaac/podaac_test.py to podaac_test.cpython-35.pyc
Sorry: TabError: inconsistent use of tabs and spaces in indentation (podaac_test.py, line 25)
byte-compiling build/bdist.macosx-10.6-intel/egg/podaac/podaac.py to podaac.cpython-35.pyc
  File "build/bdist.macosx-10.6-intel/egg/podaac/podaac.py", line 55
    print error
              ^
SyntaxError: Missing parentheses in call to 'print'

byte-compiling build/bdist.macosx-10.6-intel/egg/podaac/podaac_data_source.py to podaac_data_source.cpython-35.pyc
  File "build/bdist.macosx-10.6-intel/egg/podaac/podaac_data_source.py", line 57
    print error
              ^
SyntaxError: Missing parentheses in call to 'print'

byte-compiling build/bdist.macosx-10.6-intel/egg/podaac/podaac_utils.py to podaac_utils.cpython-35.pyc
byte-compiling build/bdist.macosx-10.6-intel/egg/podaac/tests/__init__.py to __init__.cpython-35.pyc
byte-compiling build/bdist.macosx-10.6-intel/egg/podaac/tests/mcc_test.py to mcc_test.cpython-35.pyc
byte-compiling build/bdist.macosx-10.6-intel/egg/podaac/tests/podaac_test.py to podaac_test.cpython-35.pyc
  File "build/bdist.macosx-10.6-intel/egg/podaac/tests/podaac_test.py", line 104
    print dataset_id
                   ^
SyntaxError: Missing parentheses in call to 'print'

	:param short_name: the shorter name for a dataset. \
	Either short_name or dataset_id is required for a \
	granule search. Example: ASCATA-L2-25km
	:type short_name: :mod:`string`