I am trying to use the example Jupyter Notebook for PyNHD.

After commenting out the import of a colormap that isn't present:

If I change the import of pynhd to pynhd.pynhd, I get the following message:

What happened?

When creating a conda environment, with dependencies, pynhd and shapely<2.0, importing pynhd results in a shapely import error.

What did you expect to happen?

No response

Minimal Complete Verifiable Example

mamba create -n hyrtest pynhd "shapely<2.0"

In terminal with hyrtest activated:
python
import pynhd

MVCE confirmation

• Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue.
• Complete example — the example is self-contained, including all data and the text of any traceback.
• New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

❯ python
Python 3.11.0 | packaged by conda-forge | (main, Jan 14 2023, 12:27:40) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pynhd
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/rmcd/mambaforge/envs/hyrtest/lib/python3.11/site-packages/pynhd/__init__.py", line 9, in <module>
    from pynhd.core import AGRBase, GeoConnex, ScienceBase
  File "/home/rmcd/mambaforge/envs/hyrtest/lib/python3.11/site-packages/pynhd/core.py", line 15, in <module>
    from pygeoogc import ArcGISRESTful, ServiceURL
  File "/home/rmcd/mambaforge/envs/hyrtest/lib/python3.11/site-packages/pygeoogc/__init__.py", line 13, in <module>
    from pygeoogc.pygeoogc import WFS, WMS, ArcGISRESTful, ServiceURL
  File "/home/rmcd/mambaforge/envs/hyrtest/lib/python3.11/site-packages/pygeoogc/pygeoogc.py", line 11, in <module>
    from shapely import LineString, MultiPoint, MultiPolygon, Point, Polygon
ImportError: cannot import name 'LineString' from 'shapely' (/home/rmcd/mambaforge/envs/hyrtest/lib/python3.11/site-packages/shapely/__init__.py)

Anything else we need to know?

No response

Environment

What happened:

What you expected to happen:

Minimal Complete Verifiable Example:

# Put your MCVE code here

Anything else we need to know?: I am using Python 3.8 for which I get errors of failing to load DLL files. Do I need to use earlier version of Python?

Environment:

</details>

Is your feature request related to a problem? Please describe.
Add support for StreamStat following the suggestion in hyriver/pygeohydro#38

Describe the solution you'd like

NLDI and StreamStats are working together to revise the NLDI delineation tools so they will delineate from a click point not just from the catchment.
The data processing steps and quality assurance work, as well as the underlying data in StreamStats, typically mean that delineations from StreamStats will be more accurate than from the NHDPlus datasets being queried in NLDI. For example, South Carolina data is based on lidar data, we're currently working on 3-meter lidar data in Nebraska. Thus, depending on the use, you may want to include the option of using StreamStats as well as NLDI.

Describe alternatives you've considered
We need to figure out a way to implement StreamStat that can either complement NLDI and/or work with a similar API to NLDI.

Additional context
@USGSPMM3, I went through the documentation and it seems that it's designed with a specific workflow in mind. I was wondering if you can provide a common example. Also, can you explain the importance of rcode? I don't understand the reason behind rcode being mandatory when you can provide lon and lat.

I expected the flow_trace() function to return all of the upstream river reaches, but it is only returning one. I think this is because the NLDI API changed, and now requires distance as a parameter.
https://waterdata.usgs.gov/blog/nldi_update/#distance-is-now-a-required-query-parameter

import pynhd
pygeoapi = pynhd.PyGeoAPI()
lng, lat = -73.82705, 43.29139
trace = pygeoapi.flow_trace((lng, lat),  crs="epsg:4326", direction="up")
print(len(trace))

returns 1
(expected this watershed to contain dozens or even hundreds of river reaches).

Is your feature request related to a problem? Please describe.
No.

Describe the solution you'd like
Add support for a new NLDI service called pygeoapi_plugins

Describe alternatives you've considered
No alternative.

Additional context
The documentation for this service can be found here

Is your feature request related to a problem? Please describe.
No. InternetOfWaters has a web service called geoconnex that provides access to many hydro-linked datasets.

Describe the solution you'd like
Its Swagger UI can be found here

Describe alternatives you've considered
N/A

Additional context
N/A

What happened:
nhdplus_attrs keeps throwing ServiceError exception.

What you expected to happen:
The ScienceBase service which is the underlying service for this function has been down for a couple of days.

Minimal Complete Verifiable Example:

import pynhd as nhd

meta = nhd.nhdplus_attrs()

Anything else we need to know?:

Environment:

</details>

What happened:
Following an example notebook - dam_impact.ipynb fails at the cell (17) using navigate_byid(). All previous cells executed successfully and in sequence. Returns the following error -

---------------------------------------------------------------------------
InvalidInputValue                         Traceback (most recent call last)
c:\Users\keonm\Documents\GitHub\HyRiver-examples\Geospatial Hydrologic Data Using Web Services.ipynb Cell 20' in <cell line: 3>()
      [3](vscode-notebook-cell:/c%3A/Users/keonm/Documents/GitHub/HyRiver-examples/Geospatial%20Hydrologic%20Data%20Using%20Web%20Services.ipynb#ch0000019?line=2) for agency, fid in sites[["agency_cd", "site_no"]].itertuples(index=False, name=None):
      [4](vscode-notebook-cell:/c%3A/Users/keonm/Documents/GitHub/HyRiver-examples/Geospatial%20Hydrologic%20Data%20Using%20Web%20Services.ipynb#ch0000019?line=3)     try:
----> [5](vscode-notebook-cell:/c%3A/Users/keonm/Documents/GitHub/HyRiver-examples/Geospatial%20Hydrologic%20Data%20Using%20Web%20Services.ipynb#ch0000019?line=4)         flw_up[fid] = nldi.navigate_byid(
      [6](vscode-notebook-cell:/c%3A/Users/keonm/Documents/GitHub/HyRiver-examples/Geospatial%20Hydrologic%20Data%20Using%20Web%20Services.ipynb#ch0000019?line=5)             fsource="nwissite",
      [7](vscode-notebook-cell:/c%3A/Users/keonm/Documents/GitHub/HyRiver-examples/Geospatial%20Hydrologic%20Data%20Using%20Web%20Services.ipynb#ch0000019?line=6)             fid=f"{agency}-{fid}",
      [8](vscode-notebook-cell:/c%3A/Users/keonm/Documents/GitHub/HyRiver-examples/Geospatial%20Hydrologic%20Data%20Using%20Web%20Services.ipynb#ch0000019?line=7)             navigation="upstreamTributaries",
      [9](vscode-notebook-cell:/c%3A/Users/keonm/Documents/GitHub/HyRiver-examples/Geospatial%20Hydrologic%20Data%20Using%20Web%20Services.ipynb#ch0000019?line=8)             source="flowlines",
     [10](vscode-notebook-cell:/c%3A/Users/keonm/Documents/GitHub/HyRiver-examples/Geospatial%20Hydrologic%20Data%20Using%20Web%20Services.ipynb#ch0000019?line=9)             distance=10)
     [11](vscode-notebook-cell:/c%3A/Users/keonm/Documents/GitHub/HyRiver-examples/Geospatial%20Hydrologic%20Data%20Using%20Web%20Services.ipynb#ch0000019?line=10)     except ZeroMatched:
     [12](vscode-notebook-cell:/c%3A/Users/keonm/Documents/GitHub/HyRiver-examples/Geospatial%20Hydrologic%20Data%20Using%20Web%20Services.ipynb#ch0000019?line=11)         noflw.append(fid)

File ~\miniconda3\envs\pygeo-hyriver\lib\site-packages\pynhd\pynhd.py:945, in NLDI.navigate_byid(self, fsource, fid, navigation, source, distance, trim_start)
    [942](file:///~/miniconda3/envs/pygeo-hyriver/lib/site-packages/pynhd/pynhd.py?line=941)     raise ZeroMatched
    [944](file:///~/miniconda3/envs/pygeo-hyriver/lib/site-packages/pynhd/pynhd.py?line=943) if navigation not in valid_navigations.keys():
--> [945](file:///~/miniconda3/envs/pygeo-hyriver/lib/site-packages/pynhd/pynhd.py?line=944)     raise InvalidInputValue("navigation", list(valid_navigations.keys()))
    [947](file:///~/miniconda3/envs/pygeo-hyriver/lib/site-packages/pynhd/pynhd.py?line=946) url = valid_navigations[navigation]
    [949](file:///~/miniconda3/envs/pygeo-hyriver/lib/site-packages/pynhd/pynhd.py?line=948) r_json = self._get_url(url)

InvalidInputValue: Given navigation is invalid. Valid options are:
description
type

What you expected to happen: Cell executes + calls NLDI web service using function

Environment: Created conda environment using repo's .yml. Installed Jupyter Lab & running in VSCode.

Is your feature request related to a problem?

Hi Taher, I have several projects that are limited to shapely < 2.0. Would it be possible to also provide documentation prior to the latest release of hyriver packages? Thanks for considering - Rich

Describe the solution you'd like

No response

Describe alternatives you've considered

No response

Additional context

No response

What happened?

I was playing with the pynhd Quick Start examples but encountered with input error, "x, y, z, and time must be same size". It seems to be caused by crs transform.

What did you expect to happen?

No response

Minimal Complete Verifiable Example

from pynhd import NLDI, WaterData, NHDPlusHR
import pynhd as nhd
nldi = NLDI()
station_id = "01031500"
basin = nldi.get_basins(station_id)

MVCE confirmation

• Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue.
• Complete example — the example is self-contained, including all data and the text of any traceback.
• New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

ProjError                                 Traceback (most recent call last)
Cell In[5], line 3
      1 nldi = NLDI()
      2 station_id = "01031500"
----> 3 basin = nldi.get_basins(station_id)

File ~\anaconda3\envs\flow_Ml\lib\site-packages\pynhd\pynhd.py:900, in NLDI.get_basins(self, feature_ids, fsource, split_catchment, simplified)
    898 urls = (("linked-data", fsource, fid, f"basin?{query}") for fid in feature_ids)
    899 index, resp = self._get_urls(urls, True)
--> 900 basins = geoutils.json2geodf(resp, 4269, 4326)  # type: ignore
    901 basins.index = pd.Index([feature_ids[i] for i in index], name="identifier")
    902 basins = basins[~basins.geometry.isnull()].copy()

File ~\anaconda3\envs\flow_Ml\lib\site-packages\pygeoutils\pygeoutils.py:120, in json2geodf(content, in_crs, crs)
    118     geodf = geodf.set_crs(in_crs)
    119     if in_crs != crs:
--> 120         geodf = geodf.to_crs(crs)
    121 geodf = cast("gpd.GeoDataFrame", geodf)
    122 return geodf

File ~\anaconda3\envs\flow_Ml\lib\site-packages\geopandas\geodataframe.py:1364, in GeoDataFrame.to_crs(self, crs, epsg, inplace)
   1362 else:
   1363     df = self.copy()
-> 1364 geom = df.geometry.to_crs(crs=crs, epsg=epsg)
   1365 df.geometry = geom
   1366 if not inplace:

File ~\anaconda3\envs\flow_Ml\lib\site-packages\geopandas\geoseries.py:1124, in GeoSeries.to_crs(self, crs, epsg)
   1047 def to_crs(self, crs=None, epsg=None):
   1048     """Returns a ``GeoSeries`` with all geometries transformed to a new
   1049     coordinate reference system.
   1050 
   (...)
   1121 
   1122     """
   1123     return GeoSeries(
-> 1124         self.values.to_crs(crs=crs, epsg=epsg), index=self.index, name=self.name
   1125     )

File ~\anaconda3\envs\flow_Ml\lib\site-packages\geopandas\array.py:779, in GeometryArray.to_crs(self, crs, epsg)
    775     return self
    777 transformer = Transformer.from_crs(self.crs, crs, always_xy=True)
--> 779 new_data = vectorized.transform(self.data, transformer.transform)
    780 return GeometryArray(new_data, crs=crs)

File ~\anaconda3\envs\flow_Ml\lib\site-packages\geopandas\_vectorized.py:1114, in transform(data, func)
   1111 result[~has_z] = set_coordinates(data[~has_z].copy(), np.array(new_coords_z).T)
   1113 coords_z = get_coordinates(data[has_z], include_z=True)
-> 1114 new_coords_z = func(coords_z[:, 0], coords_z[:, 1], coords_z[:, 2])
   1115 result[has_z] = set_coordinates(data[has_z].copy(), np.array(new_coords_z).T)
   1117 return result

File ~\anaconda3\envs\flow_Ml\lib\site-packages\pyproj\transformer.py:430, in Transformer.transform(self, xx, yy, zz, tt, radians, errcheck, direction)
    428     intime = None
    429 # call pj_transform.  inx,iny,inz buffers modified in place.
--> 430 self._transformer._transform(
    431     inx,
    432     iny,
    433     inz=inz,
    434     intime=intime,
    435     direction=direction,
    436     radians=radians,
    437     errcheck=errcheck,
    438 )
    439 # if inputs were lists, tuples or floats, convert back.
    440 outx = _convertback(xisfloat, xislist, xistuple, inx)

File pyproj/_transformer.pyx:459, in pyproj._transformer._Transformer._transform()

ProjError: x, y, z, and time must be same size

Anything else we need to know?

No response

Environment

Two methods provided by the WaterData class are largely equivalent: byid and getfeature_byid. getfeature_byid appears to be inherited from pygeoogc.WFS and is lower-level compared to byid. byidis a lot more convenient b/c it returns a GeoDataFrame.

But byid doesn't have a docstring, so it's not obvious how it's used and how it's different from getfeature_byid. Assuming I'm getting this right (FYI, I settled on using byid b/c it's a lot more convenient), I suggest at least adding a docstring to byid. I can submit a PR later, once you confirm I am getting this right.

What happened: I am trying to use pynhd but it gives the error: "ImportError: DLL load failed while importing lib: The specified module could not be found." How to solve this?

pynhd.show_versions()

ImportError Traceback (most recent call last)
in
----> 1 pynhd.show_versions()

~\anaconda3\lib\site-packages\pynhd\print_versions.py in show_versions(file)
168 for (modname, ver_f) in deps:
169 try:
--> 170 mod = _get_mod(modname)
171 except ModuleNotFoundError:
172 deps_blob.append((modname, None))

~\anaconda3\lib\site-packages\pynhd\print_versions.py in get_mod(modname)
94 return sys.modules[modname]
95 try:
---> 96 return importlib.import_module(modname)
97 except ModuleNotFoundError:
98 return importlib.import_module(modname.replace("-", ""))

~\anaconda3\lib\importlib_init_.py in import_module(name, package)
125 break
126 level += 1
--> 127 return _bootstrap._gcd_import(name[level:], package, level)
128
129

~\anaconda3\lib\importlib_bootstrap.py in _gcd_import(name, package, level)

~\anaconda3\lib\importlib_bootstrap.py in find_and_load(name, import)

~\anaconda3\lib\importlib_bootstrap.py in find_and_load_unlocked(name, import)

~\anaconda3\lib\importlib_bootstrap.py in _load_unlocked(spec)

~\anaconda3\lib\importlib_bootstrap_external.py in exec_module(self, module)

~\anaconda3\lib\importlib_bootstrap.py in _call_with_frames_removed(f, *args, **kwds)

~\anaconda3\lib\site-packages\pygeos_init_.py in
32 # end delvewheel patch
33
---> 34 from .lib import GEOSException # NOQA
35 from .lib import Geometry # NOQA
36 from .lib import geos_version, geos_version_string # NOQA

ImportError: DLL load failed while importing lib: The specified module could not be found.

=========================
What you expected to happen:

Minimal Complete Verifiable Example:

# Put your MCVE code here

Anything else we need to know?:

Environment: I am using Python 3.7.12 with anaconda.

Currently we can get WBD data using either pynhd.WaterData or pygeoogc.ArcGISRESTful().restful.wbd. The former accesses the USGS "waterlabs" GeoServer (which presumably is more experimental and less publicly supported) and the latter apparently accesses the official USGS TNM source. The former provides access to HUC08 and HUC12 layers, while the latter apparently only provides access to a HUC12 layer -- at least that's what I see in your pygeoogc example.

Which one would you recommend to a new user of the hydrodata ecosystem? I think that choice is somewhat confusing to users unfamiliar with web services. Personally, I like having the option to access a HUC08 layer directly, but at the same time the projection inconsistencies in the waterlabs Geoserver layers are not user friendly.

For WaterHackWeek I'm using pynhd.WaterData exclusively. FYI, here's the current state of my WHW tutorial notebook that uses the hydrodata suite to query for HUC layers. It's almost done, but it'll continue to change for the next couple of days.

What happened:
I'm just trying to run the example notebook "River Elevation and Cross-Section"

I got an error at cell 13 when trying to query the flowlines by COMID

I get a JSONDecodeError

Minimal Complete Verifiable Example:
This simple code produces the same error:

nhd = NHD("flowline_mr")
main_nhd = nhd.byids('COMID',['1722317'])

Environment:

INSTALLED VERSIONS
------------------
commit: None
python: 3.9.13 | packaged by conda-forge | (main, May 27 2022, 16:50:36) [MSC v.1929 64 bit (AMD64)]
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 141 Stepping 1, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None
LOCALE: English_United States.1252
libhdf5: 1.12.1
libnetcdf: 4.8.1

aiodns: 3.0.0
aiohttp: 3.8.1
aiohttp-client-cache: 0.7.1
aiosqlite: 0.17.0
async-retriever: 0.3.3
bottleneck: 1.3.4
brotli: installed
cchardet: 2.1.7
click: 8.1.3
cytoolz: 0.11.2
dask: 2022.6.1
defusedxml: 0.7.1
folium: 0.12.1.post1
geopandas: 0.11.0
lxml: 4.9.0
matplotlib: 3.4.3
netCDF4: 1.5.8
networkx: 2.8.4
numpy: 1.23.0
owslib: 0.25.0
pandas: 1.4.3
py3dep: 0.0
pyarrow: 6.0.0
pydantic: 1.9.1
pydaymet: 0.13.2
pygeohydro: 0.13.2
pygeoogc: 0.13.2
pygeos: 0.12.0
pygeoutils: 0.13.2
pynhd: 0.13.2
pyproj: 3.3.1
pytest: None
pytest-cov: None
rasterio: 1.2.10
requests: 2.28.0
requests-cache: 0.9.4
richdem: None
rioxarray: 0.11.1
scipy: 1.8.1
shapely: 1.8.2
tables: None
ujson: 5.3.0
urllib3: 1.26.9
xarray: 2022.3.0
xdist: None
yaml: 6.0

What happened?

I've pulled huc02 and huc4 from the USGS WBD using pynhd's WaterData() function.

I am trying to pull gages ('gagesii') within those boundary polygons (using WaterData('gagesii').by_geom()).
I got an error when attempting to pull gages for huc2. Gages returned successfully for the huc04 polygons.

import geopandas as gpd
import pynhd

polygon1_huc2 = gpd.read_file('tmp_huc2_polygon_1.gpkg')

print(f'Is polygon1_huc2 geometry valid:  {polygon1_huc2.is_valid}')

WaterData('gagesii').bygeom(polygon1_huc2.reset_index().geometry[0])

----

Examples gpkgs: tmp_huc_polygons.zip

What did you expect to happen?

I tested this with a huc 4 polygon and I did not run into this error. This is the expected output.

import geopandas as gpd

polygon1_huc4 = gpd.read_file('tmp_huc4_polygon_1.gpkg')

print(f'Is polygon1_huc4 geometry valid:  {polygon1_huc4.is_valid}')

WaterData('gagesii').bygeom(polygon1_huc4.reset_index().geometry[0])

Minimal Complete Verifiable Example

import geopandas as gpd
import pynhd

polygon1_huc4 = gpd.read_file('tmp_huc4_polygon_1.gpkg')
print(f'Is polygon1_huc4 geometry valid:  {polygon1_huc4.is_valid}')
gages_polygon1_huc4 = WaterData('gagesii').bygeom(polygon1_huc4.reset_index().geometry[0])
print(gages_polygon1_huc4.head(3))

print('----'*60)

polygon2_huc4 = gpd.read_file('tmp_huc4_polygon_2.gpkg')
print(f'Is polygon1_huc4 geometry valid:  {polygon2_huc4.is_valid}')
gages_polygon2_huc4 = WaterData('gagesii').bygeom(polygon2_huc4.reset_index().geometry[0])
print(gages_polygon2_huc4.head(3))

print('----'*60)

polygon1_huc2 = gpd.read_file('tmp_huc2_polygon_1.gpkg')
print(f'Is polygon1_huc2 geometry valid:  {polygon1_huc2.is_valid}')
gages_polygon1_huc2 = WaterData('gagesii').bygeom(polygon1_huc2.reset_index().geometry[0])
print(gages_polygon1_huc2.head(3))

print('----'*60)

polygon2_huc2 = gpd.read_file('tmp_huc2_polygon_2.gpkg')
print(f'Is polygon2_huc2 geometry valid:  {polygon2_huc2.is_valid}')
gages_polygon2_huc2 = WaterData('gagesii').bygeom(polygon2_huc2.reset_index().geometry[0])
print(gages_polygon1_huc2.head(3))

MVCE confirmation

• Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue.
• Complete example — the example is self-contained, including all data and the text of any traceback.
• New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

No response

Anything else we need to know?

I've attached 4 example polygons (2x HUC2, 2x HUC4) in the attached zip file (first text box) to test this bug. After downloading those example gpkg files, they can be read in and tested using the code chunk in the MVCE box.
2 polygons per huc level are provided to test and better demonstrate the error.

Environment

I'm using PythonWin with Python version 3.10 on a windows PC. I was trying out the quick start code for the pynhd package. Once I hit the code: gcx = GeoConnex("gages"). I get a NameError: name GeoConnex is not defined.

I expected my code to assign the value GeoConnex("gages") to the variable gcx.

import pynhd
gcx = GeoConnex("gages")

Thanks for this library! I am on a Windows 10 and Python 3.8.10

I was running through the example contained in the phynhd readme file for USGS station ID 01482100. I get an error when I run:

wd_cat = WaterData("catchmentsp")

catchments = wd_cat.byid("featureid", comids)

It seems like the URL provided by WaterData() is dead. Do you know if this is a temporary problem, or a permanent change? I get the following error:

`---------------------------------------------------------------------------
ClientResponseError Traceback (most recent call last)
~\Anaconda3\envs\pangeo\lib\site-packages\async_retriever\async_retriever.py in _retrieve(uid, url, session, read_type, s_kwds, r_kwds)
59 try:
---> 60 response.raise_for_status()
61 resp = await getattr(response, read_type)(**r_kwds)

~\Anaconda3\envs\pangeo\lib\site-packages\aiohttp\client_reqrep.py in raise_for_status(self)
999 self.release()
-> 1000 raise ClientResponseError(
1001 self.request_info,

ClientResponseError: 502, message='Bad Gateway', url=URL('https://labs.waterdata.usgs.gov/geoserver/wmadata/ows')

The above exception was the direct cause of the following exception:

ServiceError Traceback (most recent call last)
in
----> 1 catchments = wd_cat.byid("featureid", comids)

~\Anaconda3\envs\pangeo\lib\site-packages\pynhd\pynhd.py in byid(self, featurename, featureids)
233 def byid(self, featurename: str, featureids: Union[List[str], str]) -> gpd.GeoDataFrame:
234 """Get features based on IDs."""
--> 235 resp = self.wfs.getfeature_byid(featurename, featureids)
236 return self._to_geodf(resp)
237

~\Anaconda3\envs\pangeo\lib\site-packages\pygeoogc\pygeoogc.py in getfeature_byid(self, featurename, featureids)
502
503 if len(featureids) > 200:
--> 504 return self.getfeature_byfilter(f"{featurename} IN ({fid_list})", method="POST")
505
506 return self.getfeature_byfilter(f"{featurename} IN ({fid_list})")

~\Anaconda3\envs\pangeo\lib\site-packages\pygeoogc\pygeoogc.py in getfeature_byfilter(self, cql_filter, method)
550 elif method == "POST":
551 headers = {"content-type": "application/x-www-form-urlencoded"}
--> 552 resp = ar.retrieve(
553 [self.url], self.read_method, [{"data": payload, "headers": headers}], "POST"
554 )

~\Anaconda3\envs\pangeo\lib\site-packages\async_retriever\async_retriever.py in retrieve(urls, read, request_kwds, request_method, max_workers, cache_name, family)
190 )
191
--> 192 return [r for _, r in sorted(tlz.concat(results))]
193
194

~\Anaconda3\envs\pangeo\lib\site-packages\async_retriever\async_retriever.py in (.0)
184 chunked_reqs = tlz.partition_all(max_workers, inp.url_kwds)
185 results = (
--> 186 loop.run_until_complete(
187 async_session(c, inp.read, inp.r_kwds, inp.request_method, inp.cache_name, inp.family),
188 )

~\Anaconda3\envs\pangeo\lib\site-packages\nest_asyncio.py in run_until_complete(self, future)
68 raise RuntimeError(
69 'Event loop stopped before Future completed.')
---> 70 return f.result()
71
72 def _run_once(self):

~\Anaconda3\envs\pangeo\lib\asyncio\futures.py in result(self)
176 self.__log_traceback = False
177 if self._exception is not None:
--> 178 raise self._exception
179 return self._result
180

~\Anaconda3\envs\pangeo\lib\asyncio\tasks.py in __step(failed resolving arguments)
278 # We use the send method directly, because coroutines
279 # don't have __iter__ and __next__ methods.
--> 280 result = coro.send(None)
281 else:
282 result = coro.throw(exc)

~\Anaconda3\envs\pangeo\lib\site-packages\async_retriever\async_retriever.py in async_session(url_kwds, read, r_kwds, request_method, cache_name, family)
115 request_func = getattr(session, request_method.lower())
116 tasks = (_retrieve(uid, u, request_func, read, kwds, r_kwds) for uid, u, kwds in url_kwds)
--> 117 return await asyncio.gather(*tasks)
118
119

~\Anaconda3\envs\pangeo\lib\asyncio\tasks.py in __wakeup(self, future)
347 def __wakeup(self, future):
348 try:
--> 349 future.result()
350 except BaseException as exc:
351 # This may also be a cancellation.

~\Anaconda3\envs\pangeo\lib\asyncio\tasks.py in __step(failed resolving arguments)
278 # We use the send method directly, because coroutines
279 # don't have __iter__ and __next__ methods.
--> 280 result = coro.send(None)
281 else:
282 result = coro.throw(exc)

~\Anaconda3\envs\pangeo\lib\site-packages\async_retriever\async_retriever.py in _retrieve(uid, url, session, read_type, s_kwds, r_kwds)
61 resp = await getattr(response, read_type)(**r_kwds)
62 except (ClientResponseError, ContentTypeError) as ex:
---> 63 raise ServiceError(await response.text()) from ex
64 else:
65 return uid, resp

ServiceError:

502 Bad Gateway

What happened: Find None values in the return dataframe of using nldi.get_basins() function

What you expected to happen: If nothing for this USGS station, just return the str of the usgs id

Minimal Complete Verifiable Example:

from pynhd import NLDI

nldi = NLDI()
basin = nldi.get_basins(['04253294','04253296'])

Anything else we need to know?: To be more specific, the USGS_id which return none are ''04253294" and "04253296"

Environment:

INSTALLED VERSIONS
------------------
commit: None
python: 3.9.9 | packaged by conda-forge | (main, Dec 20 2021, 02:36:06) [MSC v.1929 64 bit (AMD64)]
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 85 Stepping 7, GenuineIntel
byteorder: little
LC_ALL: None
LANG: en
LOCALE: English_United States.1252

aiohttp-client-cache>=0.5.1: None
aiohttp>=3.8.1: None
aiosqlite: 0.17.0
async-retriever: 0.3.1
async-retriever>=0.3.1: None
cytoolz: 0.11.2
dask: 2021.12.0
defusedxml: 0.7.1
geopandas>=0.7: None
netCDF4: 1.5.8
networkx: 2.6.3
numpy>=1.17: None
owslib: 0.25.0
pandas>=1.0: None
pip: 21.3.1
py3dep: None
pyarrow: 6.0.0
pydantic: 1.9.0
pydaymet: None
pygeohydro: None
pygeoogc: 0.12.2
pygeoogc>=0.12: None
pygeos: 0.12.0
pygeoutils: 0.12.2
pygeoutils>=0.12: None
pynhd: 0.3.1
pyproj>=2.2: None
pytest: None
rasterio>=1.2: None
requests: 2.26.0
requests-cache>=0.8: None
rioxarray>=0.8: None
scipy: 1.7.3
setuptools: 59.8.0
shapely>=1.6: None
ujson: 5.1.0
urllib3: 1.26.7
ward: None
xarray>=0.18: None
yaml: 6.0

What happened?

The Following code returns an InputValueError in the async_retriever package.

# USGS gage 01482100 Delaware River at Del Mem Bridge at Wilmington De
gage_id = "01482100"
nldi = NLDI()
del_basins = nldi.get_basins(gage_id)
huc12_basins = WaterData(layer="huc12").bygeom(del_basins.geometry[0])

What did you expect to happen?

No response

Minimal Complete Verifiable Example

No response

MVCE confirmation

• Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue.
• Complete example — the example is self-contained, including all data and the text of any traceback.
• New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

---------------------------------------------------------------------------
InputValueError                           Traceback (most recent call last)
Cell In[10], line 1
----> 1 huc12_basins = WaterData(layer="huc12").bygeom(del_basins.geometry[0])

File ~/mambaforge/envs/gdptools-examples/lib/python3.10/site-packages/pynhd/pynhd.py:457, in WaterData.bygeom(self, geometry, geo_crs, xy, predicate, sort_attr)
    415 def bygeom(
    416     self,
    417     geometry: Polygon | MultiPolygon,
   (...)
    421     sort_attr: str | None = None,
    422 ) -> gpd.GeoDataFrame:
    423     """Get features within a geometry.
    424 
    425     Parameters
   (...)
    455         The requested features in the given geometry.
    456     """
--> 457     resp = self.wfs.getfeature_bygeom(
    458         geometry, geo_crs, always_xy=not xy, predicate=predicate, sort_attr=sort_attr
    459     )
    460     resp = cast("list[dict[str, Any]]", resp)
    461     return self._to_geodf(resp)

File ~/mambaforge/envs/gdptools-examples/lib/python3.10/site-packages/pygeoogc/pygeoogc.py:646, in WFS.getfeature_bygeom(self, geometry, geo_crs, always_xy, predicate, sort_attr)
    643 if predicate.upper() not in valid_predicates:
    644     raise InputValueError("predicate", valid_predicates)
--> 646 return self.getfeature_byfilter(
    647     f"{predicate.upper()}({geom_name}, {g_wkt})", method="POST", sort_attr=sort_attr
    648 )

File ~/mambaforge/envs/gdptools-examples/lib/python3.10/site-packages/pygeoogc/pygeoogc.py:735, in WFS.getfeature_byfilter(self, cql_filter, method, sort_attr)
    733 else:
    734     headers = {"content-type": "application/x-www-form-urlencoded"}
--> 735     resp = ar.retrieve_text([self.url], [{"data": payload, "headers": headers}], "POST")
    736 try:
    737     nfeatures = int(resp[0].split(self.nfeat_key)[-1].split(" ")[0].strip('"'))

File ~/mambaforge/envs/gdptools-examples/lib/python3.10/site-packages/async_retriever/async_retriever.py:498, in retrieve_text(urls, request_kwds, request_method, max_workers, cache_name, timeout, expire_after, ssl, disable, raise_status)
    437 def retrieve_text(
    438     urls: Sequence[StrOrURL],
    439     request_kwds: Sequence[dict[str, Any]] | None = None,
   (...)
    447     raise_status: bool = True,
    448 ) -> list[str]:
    449     r"""Send async requests and get the response as ``text``.
    450 
    451     Parameters
   (...)
    496     '01646500'
    497     """
--> 498     return retrieve(
    499         urls,
    500         "text",
    501         request_kwds,
    502         request_method,
    503         max_workers,
    504         cache_name,
    505         timeout,
    506         expire_after,
    507         ssl,
    508         disable,
    509         raise_status,
    510     )

File ~/mambaforge/envs/gdptools-examples/lib/python3.10/site-packages/async_retriever/async_retriever.py:394, in retrieve(urls, read_method, request_kwds, request_method, max_workers, cache_name, timeout, expire_after, ssl, disable, raise_status)
    330 def retrieve(
    331     urls: Sequence[StrOrURL],
    332     read_method: Literal["text", "json", "binary"],
   (...)
    341     raise_status: bool = True,
    342 ) -> RESPONSE:
    343     r"""Send async requests.
    344 
    345     Parameters
   (...)
    392     '01646500'
    393     """
--> 394     inp = BaseRetriever(
    395         urls,
    396         read_method=read_method,
    397         request_kwds=request_kwds,
    398         request_method=request_method,
    399         cache_name=cache_name,
    400     )
    402     if not disable:
    403         disable = os.getenv("HYRIVER_CACHE_DISABLE", "false").lower() == "true"

File ~/mambaforge/envs/gdptools-examples/lib/python3.10/site-packages/async_retriever/_utils.py:175, in BaseRetriever.__init__(self, urls, file_paths, read_method, request_kwds, request_method, cache_name)
    170     self.read_method = "read" if read_method == "binary" else read_method
    171     self.r_kwds = (
    172         {"content_type": None, "loads": json.loads} if read_method == "json" else {}
    173     )
--> 175 self.url_kwds = self.generate_requests(urls, request_kwds, self.file_paths)
    177 self.cache_name = create_cachefile(cache_name)

File ~/mambaforge/envs/gdptools-examples/lib/python3.10/site-packages/async_retriever/_utils.py:208, in BaseRetriever.generate_requests(urls, request_kwds, file_paths)
    206 if not_found:
    207     invalids = ", ".join(not_found)
--> 208     raise InputValueError(f"request_kwds ({invalids})", list(session_kwds))
    210 return zip(url_id, urls, request_kwds)

InputValueError: Given request_kwds (data, headers) is invalid. Valid options are:
self
method
str_or_url
expire_after
kwargs

Anything else we need to know?

I tried with versions 0.15.0 and 0.15.1.

Wondering if it was related to Margaux's issue I also tried:

huc12_basins = WaterData(layer="huc12").bygeom(del_basins.reset_index().geometry[0].simplify(1e-3))

Environment

Conda environment:

name: "gdptools-examples"
channels:
  - numba
  - bokeh
  - plotly
  - conda-forge
  - nodefaults
dependencies:
  - python>=3.9,<=3.10
  - pip
  - pydantic<=1.10.9
  - gdptools==0.2.3
  - netcdf4<=1.6.0
  - owslib
  - jupyterlab
  - jupytext
  # HyRiver packages
  - pynhd==0.15.0
  - pygeoogc==0.15.0
  - pygeohydro==0.15.0
  - bottleneck
  - numba
  - pyogrio
  - cartopy
  - hvplot
  - geoviews
  - holoviews
  - bokeh
  - datashader
  - networkx
  - flask
  - selenium
  - geckodriver
  - firefox
  - plotly
  - plotly-geo
  - nodejs

What happened?

Hi,

I am trying to use "nldi.get_basins" to extract the upstream boundary of a gauge. However, it returns an error. I tried the example, and it also failed. (see the attachment)

All the best,
Zewei

What did you expect to happen?

No response

Minimal Complete Verifiable Example

No response

MVCE confirmation

• Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue.
• Complete example — the example is self-contained, including all data and the text of any traceback.
• New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

No response

Anything else we need to know?

No response

Environment

Is your feature request related to a problem? Please describe.
A new dataset has been recently published that updates some of the NHDPlus attributes from two sources E2NHD and NWM. It could be useful to add support for this dataset.

Describe the solution you'd like
It could be good fit to include as a flag in prepare_nhdplus function.

Describe alternatives you've considered
It could be a separate function as well.

Additional context
The idea for this feature came from DOI-USGS/nhdplusTools#227.