@foo-git Downgraded from cloudscraper-1.2.71 to 1.2.69 worked for me! Thanks for the quick help :D

from rewe-discounts.

I just tested the script on my machine, and I also get the same error. Finally I can reproduce it and hopefully fix it :)
I'll try using some other scraping/request methods to access the page. I'll let you know when I've succeeded.

from rewe-discounts.

Using requests with an user agent instead of cloudscraper gives an error message hinting to an IP block. But using Firefox I can still access the url. So it's not an IP ban.

Access denied

You do not have access to mobile-api.rewe.de.

The site owner may have set restrictions that prevent you from accessing the site.

• Ray ID: redacted
• Timestamp: 2023-06-09 06:00:49 UTC
• Your IP address: redacted
• Requested URL: mobile-api.rewe.de/api/v3/all-offers?marketCode=562286
• Error reference number: 1020
• Server ID: FL_100F55
• User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36

from rewe-discounts.

Using a browser-based request method like playwright works, I can access the API. This however is severly overblown dependency-wise. I'll try to to find a smaller working solution.

from rewe-discounts.

@Nerbry72: I updated the cloudscraper dependency to 1.2.69 and (by coincidence?) it fixed the 403 error, I can access the url as usual. Please check if your package is also up-to-date.

from rewe-discounts.

This raises just a few eyebrows and hairs on my neck. Relevant xkcd.

from rewe-discounts.

Just to let you know, I just wanted to downgrade the cloudscaper dependency within my Homeassistant Integration too, but randomly it started working again.
1.2.69 should somewhen stop working, since 1.2.71 introduced newer ssl context proto version methods.

from rewe-discounts.

Thanks for your valuable feedback.
So right now I'm not sure if pinning the version to 1.2.69 helped to solve the issue, or if changes on Cloudflare/Rewe affected the API access. I have a feeling that this issue will pop up again in the future 😕

from rewe-discounts.

Unfortunately it does not work for me, although I have tried the above suggestions. Can somebody give me some tips?

from rewe-discounts.

(Hopefully) good news everyone: Based on the contribution of #16 (which seems to have been deleted, but comment got saved in #9 (comment)), I added a second code path which only gets used, if the traditional single-query method fails.

You can find the updated file in the additional_retrieval_method branch.

The output of the markdown file changed slightly, as the category names are not capitalised anymore.

Please test and feedback, otherwise I'll merge it into master in the next few weeks. Earlier with positive feedback 🚀

from rewe-discounts.