Unable to fetch data from websites with valid schema due to User Agent about cookbook HOT 4 CLOSED

@tshoyo thanks for digging into the problem and proposing a solution. A pull request would be much appreciated! 🙏

from cookbook.

So, as I was adding in the header fix, the URL I was using was able to successfully fetch the html, but it kept throwing a type conversion error. I realized that the website uses a div element rather than an article element to host the actual recipe data. This would mean the html parser would need to be improved upon for this use case.

However, I feel that using regex to parse html is not maintainable and would like to pursue a different route before messing with the current parser. It would be much better to utilize a DOM tree library to traverse the html, but I understand if there is a desire to keep dependencies to a minimum. This begs the question, is there any particular reason for fetching the recipe data server-side?
Javascript has a built in DOM parser with the addition of JQuery to make traversal that much more trivial. We could just request the target URL with $.ajax() and then search the html for recipe data.

This will increase the effectiveness of the parser without adding an additional library dependency. Worst comes to worst, we can fall back to the server-side parser if the Javascript one fails. Let me know what your thoughts are @mrzapp!

from cookbook.

@tshoyo You're right, regex parsing is a huge pain, but trying to fetch the HTML with JSON alone would most certainly throw a CORS error. We could fetch the HTML using PHP and then parse it using JavaScript.

There is also the built-in XMLParser in PHP, which should work. I would have no problem including a library though, since it's such a big part of this app.

I don't have a lot of time to work on this, so I can't afford to be picky on how problems are solved. As long as we're not making the code base unreadable or screwing up the UI, I'll be happy to include any improvements you can contribute in upcoming releases :)

from cookbook.

Lol ah I am yet to learn my lesson about CORS after all the times it's bitten me in the butt. I will see if I can make the XML parser work out, otherwise, I might try a library that uses JQuery-like syntax.

from cookbook.