500 server error with wildcard facets about esgf-pyclient HOT 7 CLOSED

Hi @pochedls , thanks for raising this. It looks like the underlying search interface has been modified and where we used a '*' we now need 'null'. You can patch your local copy with:

Lines 200-201 of: pyesgf/search/context.py

        if not ignore_facet_check:
            query_dict['facets'] = 'null'

I'll run the tests to check this doesn't have other unwanted consequences.

from esgf-pyclient.

Thanks for taking a look @agstephens. I'm wondering if it would be possible to remove facets from the API request if it isn't specified. For example,

from pyesgf.search import SearchConnection
conn = SearchConnection('https://esgf-node.llnl.gov/esg-search/', distrib=True)
ctx = conn.new_context(variable='tas', time_frequency='mon')

would query using the following URL:

https://esgf-node.llnl.gov/esg-search/search?format=application%2Fsolr%2Bjson&limit=0&distrib=false&type=Dataset&variable=tas&time_frequency=mon

from esgf-pyclient.

Hi I am also having this issue. I believe it would be helpful to not require a facets=null inclusion, and just have that be the default instead of returning a 500 error whenever facets is not defined in the constructor (examples posted on readthedocs do not work, for example). Thanks!

from esgf-pyclient.

@alaniwi: please report here on the issue identified with the Index Node functionality (i.e. the server-side issue).

from esgf-pyclient.

@soay do you have a comment on this?

from esgf-pyclient.

First of all, we have not changed anything, search is still working with facets=*. However, it is not a good practice to use the facets parameter with '*' as Solr tries to load all facets and values. For CMIP6 we have new facets that are unique for each dataset (e.g. pid, citation) so the query was running into a timeout. That's the reason for the 500.

I've modified esg-search some months ago not to return the pid and citation, so facets=* queries should work now.

In any way, it's still bad practice to use this. This is not a Solr feature, it's just the bad esg-search design that allows it. Given the re-structuration of ESGF I'm not sure it it's worth to invest time to change esg-search ...?

from esgf-pyclient.

@soay thanks for clarifying 😄

I'm closing this.

from esgf-pyclient.