ludbek / webpreview Goto Github PK

Pining exact version is bad practice because if webpreview is used in some project it will always use for example requests==2.10.0 even if there are improvement/security updates in this package.
Also if project has other dependency which requires newer version of this library, it will be impossible to be used together.
Instead requirements.txt should contain range from minimum required version to maximum (if it is incompatible with newer version of dependency) or just minimum version requirement (if there are no incompatibilities known with newer versions).
Related discussion:
pypa/setuptools#894

Hi, this is a cool project! But I have issues with commandline tool.

System: MacOS arm64, python version 3.8.11

I did pip install webpreview but there is no command webpreview.

I even tried python -m webpreview https://example.com and it says: 'webpreview' is a package and cannot be directly executed.
Did I miss anything?

The latest version of webpreview has been named webpreview-1.0.3dev-r0.tar.gz. The dev-r0 part makes it impossible to upload to pypi as it is considered immature package.

Does anyone know how to prevent sdist from appending dev-r0 to the package name?

Running webpreview in default configuration yields this error

webpreview/previews.py:51: GuessedAtParserWarning: No parser was explicitly specified, so I'm using the best available HTML parser for this system ("html.parser"). This usually isn't a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently.

The code that caused this warning is on line 51 of the file /home/nelson/src/linkblog/pinboard-to-static/venv/lib/python3.9/site-packages/webpreview/previews.py. To get rid of this warning, pass the additional argument 'features="html.parser"' to the BeautifulSoup constructor.

Presumably a change in BeautifulSoup since the last webpreview release. It works anyway, just annoying. Adding the suggested features argument does make the warning go away but raises the question of whether other parsers should be configurable.

Thanks for building this awesome pkg. In the readMe you claim it supports passing headers to web_preview function, but it doesn't.

Update: I just found out the header feature is supported in 1.3.1, but only for python3, is th7?ere any plan to support this in python2.7?

Thank you for all of this great code. It works great.
Now I just need to figure out how to make this work in an asyncio environment.

Not sure what the link was, I am fairly sure it had a few test URLs, like Amazon and Google, but one might have died.

I use this in a django application and would just like it to return None on title, description, image, within a certain time frame.

Hi tweet's preview image and the title is not working
I am using the following peace of code:
title, description, image = web_preview("https://twitter.com/realDonaldTrump/status/1290011569657610240?s=20", parser="html.parser")
print(title, description, image)

Looking forward to hearing from you about this issue.
Osama

Hi,

Thanks for your work as it is very useful. Why do you make a second request if the first one works?

try:
            res = requests.get(url, timeout=timeout, headers=headers)
        except (ConnectionError, HTTPError, Timeout, TooManyRedirects):
            raise URLUnreachable("The URL does not exist.")
        except MissingSchema: # if no schema add http as default
            url = "http://" + url

        # throw URLUnreachable exception for just incase
        try:
            res = requests.get(url, timeout=timeout, headers=headers)
        except (ConnectionError, HTTPError, Timeout, TooManyRedirects):
            raise URLUnreachable("The URL is unreachable.")

Also, you can reduce the rate of failure of the first block if you check for schema before any request is made (with a regex). Which would therefore allow you to merge the 2 blocks in one...

Greetings!

First of all, thanks for the good work! :)

I'm having some trouble with extracting the data from a few URLs, namely the ones that have relative paths in the returned image value. For instance, when using https://understand.ai as the URL, I get 'images/banner.jpg' in the image return.

It would it be nice if this lib could return the image URL along with the domain, or even if the web_preview() method had an optional parameter where the user would choose whether he/she always wanted the absolute URL or the URL exactly as it's set in the original website, like:

title, description, image = web_preview("aurl.com", absolute_url=True)

Hope I made myself clear :)

Cheers!