argosopentech / translate-html Goto Github PK

View Code? Open in Web Editor NEW

41.0 3.0 8.0 27 KB

Translate HTML using Argos Translate

License: MIT License

Python 100.00%

translate-html's Introduction

translate-html

Translate HTML using Beautiful Soup and Argos Translate

Install

pip install translatehtml

Example

import argostranslate.package, argostranslate.translate
import translatehtml

from_code = "es"
to_code = "en"

html_doc = """<div><h1>Perro</h1></div>"""

# Download and install Argos Translate package
available_packages = argostranslate.package.get_available_packages()
available_package = list(
    filter(
        lambda x: x.from_code == from_code and x.to_code == to_code, available_packages
    )
)[0]
download_path = available_package.download()
argostranslate.package.install_from_path(download_path)

# Translate
installed_languages = argostranslate.translate.get_installed_languages()
from_lang = list(filter(lambda x: x.code == from_code, installed_languages))[0]
to_lang = list(filter(lambda x: x.code == to_code, installed_languages))[0]

translation = from_lang.get_translation(to_lang)

translated_soup = translatehtml.translate_html(translation, html_doc)

print(translated_soup)

translate-html's People

Contributors

Stargazers

Watchers

Forkers

pierotofy renauddahou lceviri adu42 metin-mustafa codecivil misuzu sunnyxorange

translate-html's Issues

Doesn't work with beautifulsoup4==4.12.2

When running the example with latest beautifulsoup4 it doesn't translate:

$ pip freeze | grep -i beautifulsoup4
beautifulsoup4==4.9.3

>>> print(translated_soup)
<div><h1>Dog</h1></div>

$ pip install -U beautifulsoup4
$ pip freeze | grep -i beautifulsoup4
beautifulsoup4==4.12.2

>>> print(translated_soup)
<div><h1>Perro</h1></div>

translate-html should parse whitespace in HTML style

When an HTML string contains whitespace like tabs or newlines inside a tag, translate-html returns separate translations for the lines (or tab separated text). This is not consistent with the usual HTML parsing. For examples, please see LibreTranslate/LibreTranslate#288

All parts employed in the translation work correctly, e.g. BeautifulSoup respects whitespace (and should do so) when returning the tag tree and the translation function of the single text strings correctly assumes that new lines mean new content, not knowing anymore that the string came from an HTML tag.

So, somewhere on the way the HTML code should be "minified" in order to be consistent with a browser's parsing of the code - at the cost of breaking the visual formatting of the HTML code by translation. I am not sure where this should be done best but I would suggest to do it in the translate_html function. I will create a corresponding PR.

Error while translating, no problem without html tags

I just spotted something, for the translation of the word Talla (ES) to FR (for example) there are no problems without html tags, but with it does not work.

https://libretranslate.com/?source=es&target=fr&q=Talla (OK)
https://libretranslate.com/?source=es&target=fr&q=%253Cdiv%253ETalla%253C%2Fdiv%253E (BAD)

do you think the problem comes from argos or from translate-html? @PJ-Finlay

Translated comments shown in final document

The contents of the comments get translated correctly, but when reconstructing the BeautifulSoup object their tags are lost, causing the final translated document to show the contents of the comments when opened with a web browser.

The issue, as far as I can work out, is that neither itag_of_soup nor soup_of_itag differentiate between a bs4.element.NavigableString and a bs4.element.Comment (which inherits from the former).

So, itag_of_soup returns an str object regardless of whether its processing a NavigableString or a Comment. When soup_of_itag is called, it checks if the object passed to it is an instance of str and if so constructs a NavigableString, which for the case of comments results in losing the  characters in the final document.

Here's an example:

import argostranslate.translate
import translatehtml

# Original "file"
content = """
<html>
    <head>
        <title>Test</title>
    </head>
    <body>
        <!-- This should not be seen in a browser -->
        <h1>Welcome to Test!</h1>
    </body>
</html>
"""

# Define languages for translation from English to Hindi
en = argostranslate.translate.get_language_from_code("en")
hi = argostranslate.translate.get_language_from_code("hi")
ut = en.get_translation(hi)

# Translate the file with translate_html
content = translatehtml.translate_html(ut, content)

# Write the translated file
with open("test.html", "wt") as fp:
    fp.write(str(content))


Original file


Translation

doctype is broken after translating

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/1999/REC-html401-19991224/strict.dtd">

is broken in

html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/1999/REC-html401-19991224/strict.dtd"

it is an example for this doctype but in general as soon as there is a complex doctype it breaks everything

Translation issues (duplication and missing words) for german source with html encoded <

First of all: thanks for your work!

I stumbled into the following issues:

using the example with from_code = "de" and to_code = "en":

html_doc = """<"""

translates to:

<p>&lt;= &lt;= &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt; &lt;</p>

expected translation: <

and

html_doc = """< Fehler"""

translates to:
<<=

expected translation: < error

<img src="..." alt"some text" />