Comments (3)
Hi rsuhada,
How do you access that get_clean_html() method?
The Document class usually attempts to guess the document encoding... But it might have failed for you.
Then you need to specify it manually, with subclassing or using the encoding field.
I'm not sure how to implement encoding guessing transparently for those who need it, and omit it for ones who know their document encoding...
Any ideas?
from python-readability.
Hi buriy,
Here is what I do:
res = requests.get(url)
article = Document(res.text)
article_clean_html = article.get_clean_html()
with codecs.open("test_clean.html", encoding="utf-8", mode="w") as f:
f.write(article_clean_html)
When I open the test_clean.html, I see buggy unicode characters.
All is good, if I include the charest info into the html:
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
The encoding can be taken from res.encoding returned by the requests.get().
Alternatively - is this actually the correct way to use (and saving the output) the readability package?
Thank you!
from python-readability.
Ok, got you now.
You can do:
with codecs.open("test_clean.html", encoding="utf-8", mode="w") as f:
f.write('<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />')
f.write(article_clean_html)
But I agree that could be an enhancement that a lib could do if it adds a tag to the HTML.
from python-readability.
Related Issues (20)
- Inlining images?
- re._pattern_type has been removed in favor of re.Pattern in Python 3.7
- No chance for GitHub commit page? HOT 1
- RuntimeWarning and Correct invocation on the shell command line (not python script) HOT 1
- Missing <p>-text
- REGEXES["divToPElementsRe"] logical error HOT 3
- Error when using positive_keywords (or negative_keywords) argument with python >= 3.7 HOT 1
- .text may guess the encoding incorrectly HOT 4
- Does not handle github pages
- <p> wrongly inserted before <i> or <b>
- Problems with thecyberwire.com
- Circular import error for pip install readability-lxml HOT 4
- isProbablyReaderable HOT 3
- Issue with utf8 and HTML entities HOT 2
- Last two commands in the "usage" section are incorrect
- `test_many_repeated_spaces` fails on darwin python 3.8, 3.9 & 3.10
- Consider switching from lxml's clean_html for enhanced security (and possibly performance) HOT 5
- Summary is fooled by a modal popup
- Readability of MSN articles
- "comment" in unlikely candidates mutilates formatted code blocks HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from python-readability.