I run the Customize and Access NSIDC Data to download the data, but an error occured i

Can you share the value of response.ok and <code clas

Determine how many granules exist over this time and area of interest about nsidc-data-access-notebook HOT 6 CLOSED

nsidc commented on July 20, 2024

Determine how many granules exist over this time and area of interest

from nsidc-data-access-notebook.

Comments (6)

Fumeng-zh commented on July 20, 2024 2

Thanks for your answer, it can work now! Matt Fisher ***@***.***> 于2023年6月29日周四 18:50写道：

…

That value for granule_search_url looks right. When you click that link, you're getting an error because you haven't passed any search parameters. The code above is doing some extra work to pass those parameters to the server with requests.get(..., params=search_params, ...) and that is causing the request URL to be modified to add the key-value pairs in search_params as URL query parameters. Example: >>> response = requests.get('https://example.com', params={'foo': 'bar'}) >>> response.request.url 'https://example.com/?foo=bar' >>> response.ok True >>> response.status_code 200 >>> response.content b'<!doctype html>\n<html>\n<head>\n <title>Example Domain</title>\n\n <meta charset="utf-8" />\n <meta http-equiv="Content-type" content="text/html; charset=utf-8" />\n <meta name="viewport" content="width=device-width, initial-scale=1" />\n <style type="text/css">\n body {\n background-color: #f0f0f2;\n margin: 0;\n padding: 0;\n font-family: -apple-system, system-ui, BlinkMacSystemFont, "Segoe UI", "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif;\n \n }\n div {\n width: 600px;\n margin: 5em auto;\n padding: 2em;\n background-color: #fdfdff;\n border-radius: 0.5em;\n box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);\n }\n a:link, a:visited {\n color: #38488f;\n text-decoration: none;\n }\n @media (max-width: 700px) {\n div {\n margin: 0 auto;\n width: auto;\n }\n }\n </style> \n</head>\n\n<body>\n<div>\n <h1>Example Domain</h1>\n <p>This domain is for use in illustrative examples in documents. You may use this\n domain in literature without prior coordination or asking for permission.</p>\n <p><a href="https://www.iana.org/domains/example">More information...</a></p>\n</div>\n</body>\n</html>\n' Can you please provide: - response.request.url: The actual URL the code is querying - response.ok: Whether the query was successful - response.content: The raw content received from the server. If the content is not JSON (as in the example above), the json.loads step would fail. I think the issue may be with the data you entered; I was able to reproduce this issue by specifying an invalid bounding box. @asteiker <https://github.com/asteiker> I think we need a validity check before trying to access results['feed']: if 'feed' not in results: # An error occurred, handle! # should we check before json.loads() and print `response.content` instead of `results`? # That way if we receive invalid JSON, we can see what the response looks like instead # of failing with: # json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) print(f'An error occurred: {results["errors"]}') break # or raise? ¯\_(ツ)_/¯ — Reply to this email directly, view it on GitHub <#10 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BAAZTNWRTSNHZT5O3VHHSTTXNWW5FANCNFSM6AAAAAAZXEAS7Y> . You are receiving this because you authored the thread.Message ID: ***@***.***>

from nsidc-data-access-notebook.

andypbarrett commented on July 20, 2024 1

This search should be replaced with earthaccess. I have a candidate workflow here

I've added a class to handle Service capabilities as well. Maybe a plugin?

from nsidc-data-access-notebook.

MattF-NSIDC commented on July 20, 2024

Can you share the value of response.ok and results?

Possibly we need to add a check that the requests.get() was successful before proceeding.

from nsidc-data-access-notebook.

Fumeng-zh commented on July 20, 2024

The results is empty, is there somthing wrong with the granule_search_url = 'https://cmr.earthdata.nasa.gov/search/granules', because this link returns an error.

from nsidc-data-access-notebook.

MattF-NSIDC commented on July 20, 2024

That value for granule_search_url looks right. When you click that link, you're getting an error because you haven't passed any search parameters. The code above is doing some extra work to pass those parameters to the server with requests.get(..., params=search_params, ...) and that is causing the request URL to be modified to add the key-value pairs in search_params as URL query parameters. Example:

>>> response = requests.get('https://example.com', params={'foo': 'bar'})
>>> response.request.url
'https://example.com/?foo=bar'
>>> response.ok
True
>>> response.status_code
200
>>> response.content
b'<!doctype html>\n<html>\n<head>\n    <title>Example Domain</title>\n\n    <meta charset="utf-8" />\n    <meta http-equiv="Content-type" content="text/html; charset=utf-8" />\n    <meta name="viewport" content="width=device-width, initial-scale=1" />\n    <style type="text/css">\n    body {\n        background-color: #f0f0f2;\n        margin: 0;\n        padding: 0;\n        font-family: -apple-system, system-ui, BlinkMacSystemFont, "Segoe UI", "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif;\n        \n    }\n    div {\n        width: 600px;\n        margin: 5em auto;\n        padding: 2em;\n        background-color: #fdfdff;\n        border-radius: 0.5em;\n        box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);\n    }\n    a:link, a:visited {\n        color: #38488f;\n        text-decoration: none;\n    }\n    @media (max-width: 700px) {\n        div {\n            margin: 0 auto;\n            width: auto;\n        }\n    }\n    </style>    \n</head>\n\n<body>\n<div>\n    <h1>Example Domain</h1>\n    <p>This domain is for use in illustrative examples in documents. You may use this\n    domain in literature without prior coordination or asking for permission.</p>\n    <p><a href="https://www.iana.org/domains/example">More information...</a></p>\n</div>\n</body>\n</html>\n'

Can you please provide:

response.request.url: The actual URL the code is querying
response.ok: Whether the query was successful
response.content: The raw content received from the server. If the content is not JSON (as in the example above), the json.loads step would fail. Since that didn't happen, I think you got valid JSON, but the JSON contains an errors key instead of a feed key.

I think the issue may be with the data you entered; I was able to reproduce this issue by specifying an invalid bounding box.

@asteiker I think we need a validity check before trying to access results['feed']:

    # or `if 'errors' in results:`
    if 'feed' not in results:
        # An error occurred, handle!
        # should we check before json.loads() and print `response.content` instead of `results`?
        # That way if we receive invalid JSON, we can see what the response looks like instead
        # of failing with:
        #     json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
        # Maybe this isn't a realistic case to be concerned with.
        print(f'An error occurred: {results["errors"]}')
        print(f'Request URL: {response.request.url}')
        break # or raise? ¯\_(ツ)_/¯

With this new check, the code outputs a less cryptic message for my invalid input data:

An error occurred: ['[-180, -90, 180, 90] is not a valid URL encoded bounding_box']
Request URL: https://cmr.earthdata.nasa.gov/search/granules?short_name=ATL03&version=006&temporal=2021-01-01T00%3A00%3A00Z%2C2022-01-01T00%3A00%3A00Z&page_size=100&page_num=1&bounding_box=-180%2C+-90%2C+180%2C+90

from nsidc-data-access-notebook.

MattF-NSIDC commented on July 20, 2024

Great! Thanks for surfacing this issue :) I'm going to close this issue and open a new one for the error handling.

from nsidc-data-access-notebook.

Determine how many granules exist over this time and area of interest about nsidc-data-access-notebook HOT 6 CLOSED

Comments (6)

Related Issues (9)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent