argosopentech / libretranslate-init Goto Github PK

View Code? Open in Web Editor NEW

40.0 3.0 15.0 25 KB

Shell scripts to install LibreTranslate

License: MIT License

Shell 100.00%

libretranslate-init's Introduction

LibreTranslate-init

Shell scripts to install LibreTranslate

Uses WSGI with Gunicorn and Nginx.

Install

# Add libretranslate user
useradd libretranslate
mkdir /home/libretranslate
chown libretranslate:libretranslate /home/libretranslate
usermod -aG sudo libretranslate
passwd -d libretranslate
su libretranslate

# Download LibreTranslate-init
git clone https://github.com/argosopentech/LibreTranslate-init.git ~/LibreTranslate-init

# Download dependencies and run LibreTranslate on port 5000
~/LibreTranslate-init/setup.sh

# Set server_name to your domain name in ~/LibreTranslate-init/nginx

# Run LibreTranslate WSGI with nginx and systemd
~/LibreTranslate-init/run.sh

# Check LibreTranslate status
sudo systemctl status libretranslate

# Enable https
~/LibreTranslate-init/get-cert.sh

Tutorials

Configuration

Find the setting you would like to change in the LibreTranslate settings
Add the setting to the environment of your libretranslate.service SystemD file

libretranslate-init's People

Contributors

Stargazers

Watchers

Forkers

pj-finlay mooseyboots watchcharts rhclayto ahmad962002 nready-nlp franktastik june17 johnhasmie afifudinmtop lcsouzamenezes vikrammudaliar hben35 danielmnb1 korkinmask

libretranslate-init's Issues

Can not connect

I installed it according to the instructions. When I try to call Libretranslate locally, the page is not found. Where can the error be? Niginx and Libretranslate are running

Make swap space permanent after restart

Currently the "Add swap space" commands on the readme only add swap space temporarily and it doesn't persist after a restart.

Issue in setup.sh : no env directory

In the setup.sh script, after that it cloned the libretranslate repo in the folder ~/Libretranslate, it calls this command :
virtualenv ~/Libretranslate/env
But there is no env folder in the Libretranslate repo.
Is this a new update of Libretranslate and this repo is not updated yet ?
Best regards

How can i run an update

it's possible to run an update with the script?

New threads created by by Gunicorn and Nginx but not deleted

https://community.libretranslate.com/t/python-library-of-libretranslate-run-with-gunicorn-and-nginx-not-freeing-up-threads/221

Setup Error

This line:
~/LibreTranslate/env/bin/pip install -e ~/LibreTranslate/ --no-cache-dir

results in the following error on current git version (it worked the last year fine):

ERROR: File "setup.py" not found. Directory cannot be installed in editable mode: /home/libretranslate/LibreTranslate
(A "pyproject.toml" file was found, but editable mode currently requires a setup.py based build.)

Create a EnvironmentFile for configuring LibreTranslate

SystemD supports the EnvironmentFile field to set environment variables. The could be useful for setting LibreTranslate arguments to configure LibreTranslate.

PermissionError: [Errno 13] Permission denied: 'db'

I sometimes see this error but it doesn't seem to cause issues.

Traceback (most recent call last):
  File "/home/libretranslate/LibreTranslate/env/bin/libretranslate", line 8, in <module>
    sys.exit(main())
  File "/home/libretranslate/LibreTranslate/env/lib/python3.10/site-packages/libretranslate/main.py", line 192, in main
    app = create_app(args)
  File "/home/libretranslate/LibreTranslate/env/lib/python3.10/site-packages/libretranslate/app.py", line 1069, in create_app
    Session(app)
  File "/home/libretranslate/LibreTranslate/env/lib/python3.10/site-packages/flask_session/__init__.py", line 54, in __init__
    self.init_app(app)
  File "/home/libretranslate/LibreTranslate/env/lib/python3.10/site-packages/flask_session/__init__.py", line 61, in init_app
    app.session_interface = self._get_interface(app)
  File "/home/libretranslate/LibreTranslate/env/lib/python3.10/site-packages/flask_session/__init__.py", line 90, in _get_interface
    session_interface = FileSystemSessionInterface(
  File "/home/libretranslate/LibreTranslate/env/lib/python3.10/site-packages/flask_session/sessions.py", line 322, in __init__
    self.cache = FileSystemCache(cache_dir, threshold=threshold, mode=mode)
  File "/home/libretranslate/LibreTranslate/env/lib/python3.10/site-packages/cachelib/file.py", line 65, in __init__
    os.makedirs(self._path)
  File "/usr/lib/python3.10/os.py", line 215, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/lib/python3.10/os.py", line 225, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: 'db'

Request token to make request

Hello,

I have successfully configured the environment and it works correctly, but any user can make requests.

I would like to configure the API so that it can only be done using a token, but I can't find a way to do it.

How would it be done with this configuration?

Thanks.

Error during setup

~/LibreTranslate-init/setup.sh results in error

ERROR: translatehtml 1.5.2 has requirement beautifulsoup4==4.9.3, but you'll have beautifulsoup4 4.11.1 which is incompatible.
Installing collected packages: pytz, six, backports.zoneinfo, tzdata, pytz-deprecation-shim, tzlocal, APScheduler, typing-extensions, pygments, commonmark, rich, wrapt, deprecated, pyparsing, packaging, limits, MarkupSafe, Werkzeug, zipp, importlib-metadata, Jinja2, itsdangerous, click, Flask, Flask-Limiter, appdirs, nvidia-cublas-cu11, nvidia-cudnn-cu11, nvidia-cuda-runtime-cu11, nvidia-cuda-nvrtc-cu11, torch, numpy, protobuf, idna, certifi, charset-normalizer, urllib3, requests, tqdm, stanza, PyYAML, ctranslate2, sentencepiece, argostranslate, soupsieve, beautifulsoup4, translatehtml, lxml, argos-translate-files, expiringdict, flask-swagger-ui, flask-swagger, morfessor, polyglot, pycld2, pyicu, async-timeout, redis, waitress, libretranslate
  Running setup.py develop for libretranslate
Successfully installed APScheduler-3.9.1 Flask-2.2.2 Flask-Limiter-2.6.3 Jinja2-3.1.2 MarkupSafe-2.1.1 PyYAML-6.0 Werkzeug-2.2.2 appdirs-1.4.4 argos-translate-files-1.1.0 argostranslate-1.7.0 async-timeout-4.0.2 backports.zoneinfo-0.2.1 beautifulsoup4-4.11.1 certifi-2022.9.24 charset-normalizer-2.1.1 click-8.1.3 commonmark-0.9.1 ctranslate2-2.17.0 deprecated-1.2.13 expiringdict-1.2.2 flask-swagger-0.2.14 flask-swagger-ui-4.11.1 idna-3.4 importlib-metadata-5.0.0 itsdangerous-2.1.2 libretranslate limits-2.7.1 lxml-4.8.0 morfessor-2.0.6 numpy-1.23.4 nvidia-cublas-cu11-11.10.3.66 nvidia-cuda-nvrtc-cu11-11.7.99 nvidia-cuda-runtime-cu11-11.7.99 nvidia-cudnn-cu11-8.5.0.96 packaging-21.3 polyglot-16.7.4 protobuf-4.21.9 pycld2-0.41 pygments-2.13.0 pyicu-2.10.2 pyparsing-3.0.9 pytz-2022.6 pytz-deprecation-shim-0.1.0.post0 redis-4.3.4 requests-2.28.1 rich-12.6.0 sentencepiece-0.1.96 six-1.16.0 soupsieve-2.3.2.post1 stanza-1.1.1 torch-1.13.0 tqdm-4.64.1 translatehtml-1.5.2 typing-extensions-4.4.0 tzdata-2022.6 tzlocal-4.2 urllib3-1.26.12 waitress-2.1.2 wrapt-1.14.1 zipp-3.10.0
Traceback (most recent call last):
  File "/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 583, in _build_master
    ws.require(__requires__)
  File "/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 900, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 791, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (beautifulsoup4 4.11.1 (/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages), Requirement.parse('beautifulsoup4==4.9.3'), {'translatehtml'})

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/libretranslate/LibreTranslate/env/bin/libretranslate", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3252, in <module>
    def _initialize_master_working_set():
  File "/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3235, in _call_aside
    f(*args, **kwargs)
  File "/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3264, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 585, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 598, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 786, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'beautifulsoup4==4.9.3' distribution was not found and is required by translatehtml
$

second attempt results in

Installing collected packages: libretranslate
  Attempting uninstall: libretranslate
    Found existing installation: libretranslate 1.3.1
    Uninstalling libretranslate-1.3.1:
      Successfully uninstalled libretranslate-1.3.1
  Running setup.py develop for libretranslate
Successfully installed libretranslate
Traceback (most recent call last):
  File "/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 583, in _build_master
    ws.require(__requires__)
  File "/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 900, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 791, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (beautifulsoup4 4.11.1 (/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages), Requirement.parse('beautifulsoup4==4.9.3'), {'translatehtml'})

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/libretranslate/LibreTranslate/env/bin/libretranslate", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3252, in <module>
    def _initialize_master_working_set():
  File "/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3235, in _call_aside
    f(*args, **kwargs)
  File "/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3264, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 585, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 598, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 786, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'beautifulsoup4==4.9.3' distribution was not found and is required by translatehtml
$

error with beautifulsoup4==4.9.3

Hello everyone, I have tried to install with freetranslate-init following all instructions; but fails in the installation when trying to use the beautifulsoup4===4.9.3 library. the system tells me that I have the beautifulsoup4 4.11.1. so unlock it and install the specific version (4.9.3) but it still doesn't work and show me errors.

What can I do?

Successfully built expiringdict flask-swagger-ui flask-swagger polyglot pycld2 pyicu
__ERROR: translatehtml 1.5.2 has requirement beautifulsoup4==4.9.3, but you'll have beautifulsoup4 4.11.1 which is incompatible.__
Installing collected packages: backports.zoneinfo, tzdata, pytz-deprecation-shim, tzlocal, pytz, six, APScheduler, wrapt, deprecated, typing-extensions, pyparsing, packaging, limits, click, MarkupSafe, Jinja2, Werkzeug, zipp, importlib-metadata, itsdangerous, Flask, pygments, commonmark, rich, Flask-Limiter, appdirs, soupsieve, beautifulsoup4, lxml, certifi, charset-normalizer, urllib3, idna, requests, numpy, tqdm, torch, protobuf, stanza, PyYAML, ctranslate2, sentencepiece, argostranslate, translatehtml, argos-translate-files, expiringdict, flask-swagger-ui, flask-swagger, morfessor, polyglot, pycld2, pyicu, waitress, libretranslate
  Running setup.py develop for libretranslate
Successfully installed APScheduler-3.9.1 Flask-2.1.2 Flask-Limiter-2.4.5.1 Jinja2-3.1.2 MarkupSafe-2.1.1 PyYAML-6.0 Werkzeug-2.1.2 appdirs-1.4.4 argos-translate-files-1.1.0 argostranslate-1.7.0 backports.zoneinfo-0.2.1 beautifulsoup4-4.11.1 certifi-2022.6.15 charset-normalizer-2.0.12 click-8.1.3 commonmark-0.9.1 ctranslate2-2.17.0 deprecated-1.2.13 expiringdict-1.2.1 flask-swagger-0.2.14 flask-swagger-ui-3.36.0 idna-3.3 importlib-metadata-4.12.0 itsdangerous-2.1.2 libretranslate limits-2.7.0 lxml-4.8.0 morfessor-2.0.6 numpy-1.23.1 packaging-21.3 polyglot-16.7.4 protobuf-4.21.4 pycld2-0.41 pygments-2.12.0 pyicu-2.9 pyparsing-3.0.9 pytz-2022.1 pytz-deprecation-shim-0.1.0.post0 requests-2.28.0 rich-12.5.1 sentencepiece-0.1.96 six-1.16.0 soupsieve-2.3.2.post1 stanza-1.1.1 torch-1.12.0 tqdm-4.64.0 translatehtml-1.5.2 typing-extensions-4.3.0 tzdata-2022.1 tzlocal-4.2 urllib3-1.26.11 waitress-2.1.2 wrapt-1.14.1 zipp-3.8.1
Traceback (most recent call last):
  File "/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 583, in _build_master
    ws.require(__requires__)
  File "/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 900, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 791, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (beautifulsoup4 4.11.1 (/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages), Requirement.parse('beautifulsoup4==4.9.3'), {'translatehtml'})

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/libretranslate/LibreTranslate/env/bin/libretranslate", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3252, in <module>
    def _initialize_master_working_set():
  File "/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3235, in _call_aside
    f(*args, **kwargs)
  File "/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3264, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 585, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 598, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/home/libretranslate/LibreTranslate/env/lib/python3.8/site-packages/pkg_resources/__init__.py", line 786, in resolve
    raise DistributionNotFound(req, requirers)
__pkg_resources.DistributionNotFound: The 'beautifulsoup4==4.9.3' distribution was not found and is required by translatehtml__

Illegal instruction - trap invalid opcode

After installation, I tried the command ~/LibreTranslate/env/bin/libretranslate, which produced the response Illegal instruction.

dmesg shows [2449591.634917] traps: libretranslate[2150063] trap invalid opcode ip:7f7b4a60f13c sp:7fff66433488 error:0 in libctranslate2-e7e12a4c.so[7f7b4a596000+2d04000].

Any ideas where to start troubleshooting?

Ubuntu 20.04.

How to use params

I'd like to use params for libretranslate, but I do not know where to add them. If I add them to the end of this line like that, it fails to start:

ExecStart=/home/libretranslate/LibreTranslate/env/bin/gunicorn --workers 3 --bind unix:libretranslate.sock -m 007 wsgi:app --api-keys

I also tried (and failed) with that:

ExecStart=/home/libretranslate/LibreTranslate/env/bin/gunicorn --workers 3 --bind unix:libretranslate.sock -m 007 wsgi:app --env LT_API_KEYS

/home/libretranslate/LibreTranslate/env/bin/libretranslate: not found

OS : ubuntu 20.4 on virtual machine

The following command ~/LibreTranslate/env/bin/libretranslate in setup.sh line 26 return this error /home/libretranslate/LibreTranslate/env/bin/libretranslate: not found

$ cd /home
$ ls
libretranslate	ubuntu
$ cd libretranslate
$ ls
Desktop    Downloads	   LibreTranslate-init	Pictures  Templates
Documents  LibreTranslate  Music		Public	  Videos
$ ~/LibreTranslate/env/bin/libretranslate
sh: 8: /home/libretranslate/LibreTranslate/env/bin/libretranslate: not found
$ cd LibreTranslate    
$ ls
app		    ISSUE_TEMPLATE.md	     README.md	       VERSION
docker-compose.yml  libretranslate.egg-info  requirements.txt  wsgi.py
Dockerfile	    LICENSE		     setup.cfg
env		    main.py		     setup.py
install_models.py   manage.py		     tests
$ cd env
$ ls
bin  lib  pyvenv.cfg
$ cd bin
$ ls
activate       activate_this.py  easy_install-3.8  pip-3.8  python3.8
activate.csh   activate.xsh	 gunicorn	   pip3.8   wheel
activate.fish  easy_install	 pip		   python   wheel3
activate.ps1   easy_install3	 pip3		   python3  wheel-3.8

Version Conflict Error

Thanks for offering this tool. I am trying to set this one on Ubuntu, and followed the instructions as outlined. However, I am seeing the following "VersionConflict" error. Any suggestion to address this issue? After following these steps, how can I map the libretranslate server to a domain of interest?

Cloning into '/home/libretranslate/LibreTranslate'...
remote: Enumerating objects: 833, done.
remote: Counting objects: 100% (169/169), done.
remote: Compressing objects: 100% (96/96), done.
remote: Total 833 (delta 103), reused 84 (delta 73), pack-reused 664
Receiving objects: 100% (833/833), 495.42 KiB | 3.93 MiB/s, done.
Resolving deltas: 100% (462/462), done.
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 584, in _build_master
ws.require(requires)
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 901, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 792, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (virtualenv 20.0.17 (/usr/lib/python3/dist-packages), Requirement.parse('virtualenv==16.7.8'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/bin/virtualenv", line 6, in
from pkg_resources import load_entry_point
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 3254, in
def _initialize_master_working_set():
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 3237, in _call_aside
f(*args, **kwargs)
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 3266, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 586, in _build_master
return cls._build_from_requirements(requires)
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 599, in _build_from_requirements
dists = ws.resolve(reqs, Environment())
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 787, in resolve
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'virtualenv==16.7.8' distribution was not found and is required by the application
/home/libretranslate/LibreTranslate-init/setup.sh: 22: /home/libretranslate/LibreTranslate/env/bin/pip: not found
/home/libretranslate/LibreTranslate-init/setup.sh: 25: /home/libretranslate/LibreTranslate/env/bin/pip: not found
/home/libretranslate/LibreTranslate-init/setup.sh: 26: /home/libretranslate/LibreTranslate/env/bin/libretranslate: not found

Support for Fedora

In the Readme: we can change to usermod -aG wheel libretranslate and we need to modify the other files