hmm about search-engine-parser HOT 8 CLOSED

@ArshCypherZ which search engine are you using, as I'm getting the same with DuckDuckGo, Google and Bing searches.

  File "/usr/local/lib/python3.8/dist-packages/search_engine_parser/core/base.py", line 288, in search
    return self.get_results(soup, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/search_engine_parser/core/base.py", line 247, in get_results
    raise NoResultsOrTrafficError(
search_engine_parser.core.exceptions.NoResultsOrTrafficError: The result parsing was unsuccessful. It is either your query could not be found or it was flagged as unusual traffic

The error above happens with all 3 search engines.

I find it interesting that it's only recently happened certainly with DuckDuckGo, I've not had any luck with Google or Bing, it just kept happening. I know the term being searched for will return results as I have manually run the search just to check that it has results.

Desktop (please complete the following information):
OS: Ubuntu on a VPS
Python Version: 3.8.10
Search-engine-parser version: git master

from search-engine-parser.

@ArshCypherZ which search engine are you using, as I'm getting the same with DuckDuckGo, Google and Bing searches.

  File "/usr/local/lib/python3.8/dist-packages/search_engine_parser/core/base.py", line 288, in search
    return self.get_results(soup, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/search_engine_parser/core/base.py", line 247, in get_results
    raise NoResultsOrTrafficError(
search_engine_parser.core.exceptions.NoResultsOrTrafficError: The result parsing was unsuccessful. It is either your query could not be found or it was flagged as unusual traffic

The error above happens with all 3 search engines.

google

from search-engine-parser.

After further testing today, it appears in my case, to be certain queries that the engine returns the error with DDG, which makes the behaviour intermittent. I am returning the results from page 1 & page 2 in separate queries, as I couldn't see how to get around 100 results out of the DDG engine, so two queries one for page 1 & one for page 2. If I'm doing something wrong I'm open to advice. I am also sleeping the code for 60s between queries, then for 3 minutes between individual search queries.

I'm just getting frustrated at the occurrence of the NoResultsOrTrafficError. I will have to catch it and somehow flag that the error has taken place, so that I can skip those errors.

from search-engine-parser.

Another update having explored further, catching the exception and doing 2 searches one after the other with a 60s between queries, the First Query fails and generates the exception reporting to the console ENGINE FAILURE: DuckDuckGo the second query runs without issue. Now I am very confused.

Any and all help welcomed.

from search-engine-parser.

Having looked at individual manual searches for the search term, with some of the completely failed double search, if that search was a quoted search, eg."search term" as opposed to just search term, then the search returns nothing. Are searches conducted in the engine as quoted searches or non-quoted searches? As that might explain the errors, but not all of them.

from search-engine-parser.

if u find any solution, tell me too

from search-engine-parser.

The only solution I've found at the moment is catching the errors and skipping that search term. I hope there is a better solution than that which will allow the search to actually happen.

from search-engine-parser.

I have a fix for Yahoo -> #170 if somebody needs

from search-engine-parser.