Comments (17)
There should be no need for libreoffice packages etc. at all on a unoconvert-client-only host. It's horrible to install (and update) the whole of libreoffice as a dependency for what's probably a very simple TCP thingy.
I'm not fluent with Python, at all, unfortunately.
from unoserver.
That's not possible, sorry. The communication is handled by the uno library that is a part of Libreoffice.
from unoserver.
Hi there, it is possible and there are a few ways to do it.
One way (I did something similar recently) is to use python's xmlrpc to add an rpc server to converter.py. The server would need to wrap Converter.convert so that it can be called from another process via rpc.
An xmlrpc client would also be needed, but you only have one function that needs to be wrapped so would be quick to do
The xmlrpc client would be installable in virtualenvs. Users would need to take care of starting the server/converter before using it via the xmlrpc client.
See e.g. https://docs.python.org/3/library/xmlrpc.server.html#module-xmlrpc.server
from unoserver.
I've added in #32 rough changes that would be needed to add the rpc server. The client will also be quite simple
from unoserver.
Yes, sure we could build a completely different rpcserver and do this, but that's not what this module does, and you could do that on your side as well. The convert server in your case would still have the exact same problem, it needs to have access to the uno library. Yes, you would now get a "client client" that does not, but that doesn't really solve the issue itself. You still need to install unoserver so it has access to LibreOffice.
from unoserver.
Thanks & yes agreed that this library would still need to be installed into system python env along with libreoffice+uno, there is no way around that.
The problem I was trying to solve is that other projects need to use this library, but at the moment they can only do so by either: i) running converter.py as a separate process (taking care to use system python not virtualenv python), or ii) installing the project into system python env. Both have their downsides especially the latter option you risk breaking your system python or being unable to satisfy the project's dependencies.
Also as an improvement to suggested structure, it would make more sense to run the rpc server in the same process that starts soffice (rpc server would still run from system python env). All the conversion code could be moved back into a "unoserver+rpc" module and the converter client could be very lightweight, using only RPC for conversion via both command line and API. This would also allow the converter to be installed in virtualenvs. Thanks!
from unoserver.
It would make it possible to make a Python library to make conversions, so that's a benefit.
from unoserver.
Having unoserver start a server, f ex with xml-rpc, and letting unoconvert use that instead is something we are fine with maintaining. It doesn't look like I will have time to implement it at the moment, but we are happy to accept contributions.
I would like to see the protocol used to somehow be versioned, so that we give an explicit error message if we end up using the wrong versions, and possibly even support different versions.
from unoserver.
2.0a is out now with an XML-RPC server
from unoserver.
2.0a is out now with an XML-RPC server
A great improvement! Will start testing ASAP and move into production rather quickly, I assume...
Can't wait to drop libreoffice from application servers. Excellent, that the RPC-version already seems capable of shipping the payload too - we've been using NFS, but it provides the converter servers too much access (due to bad application design, which is out of our hands).
from unoserver.
Headsup! I noticed the --daemon argument doesn't work in 2.0b1. If you need that I'll release a 2.0b2.
from unoserver.
Daemons are kind of redundant these days, with systemd it's much better to run service processes in the foreground.
I managed to get the server bit running on a dedicated conversion server for a test (moodle) environment, ie. installed 2.0b1 over what was there previously and it still just magically works.
But then, I hit sort of a brick wall. Sorry for the stupid question, but how might one actually run the light client?
Seems its a library at this stage, needing a bit of a wrapper script to work - this, unfortunately, is beyond my pretty much nonexistent python skills. Getting it to run with the RHEL8.8 system python3 3.6 would rock.
We now have a situation where transferring the data to convert and the resulting file via the XML-RPC mechanism (instead of NFS access currently used elsewhere) would be pure gold too...
from unoserver.
The unoconvert client script acts as that wrapper, so you would still use it the same way, by starting unoserver, and then using unconvert to convert files.
from unoserver.
Ah. Thanks for that golden tip. Running setup install with the system python and a custom prefix actually works, I did get the whole thing running and a remote conversion working.
Had a bit of head-banging with haproxy on the conversion server, as I didn't first realize one now needs to specify --uno-port too... Didn't help that I use ports starting at 2001 for the workers! 2002.. doh. :D
Just some final tweaks, packaging, distribution, testing, documentation and we're golden.
Big thanks from Aalto University, we'll be very early adopters for 2.0.
from unoserver.
Great stuff! Thanks for that, I'll release a 2.0 final shortly.
from unoserver.
Happy to report that after some stress- and other testing we went into production (with 2.0b1) on our main Moodle instance, so now there are quite a lot of (end user) beta testers involved. :D
In case you'd like to add some docs/tips/examples, I've written a wrapper script which emulates unoconv 0.7, which Moodle directly supports - hopefully, can get rid of it via native support some day, but it works. Other than that, the most interesting bits are probably the converter service multi-instance systemd unit (ie. 'systemctl enable unoserver@2001 unoserver@2002') and maybe some haproxy config tips. (Perhaps the non-interactive installation scripts for everything too, but they're a bit too specific for our environment to share...)
[Unit]
Description=Unoserver document conversion service
Documentation=https://github.com/unoconv/unoserver
Wants=network-online.target
After=network-online.target
StartLimitIntervalSec=600
StartLimitBurst=5
[Service]
User=apache
Group=apache
ExecStart=/opt/bin/unoserver-wrapper %i
TimeoutStartSec=60
TimeoutStopSec=15
RestartSec=10s
Restart=always
[Install]
WantedBy=multi-user.target
/opt/bin/unoserver-wrapper:
#!/bin/sh
/opt/bin/unoserver --interface 0.0.0.0 --port $1 --uno-port $(($1+100))
haproxy config bits (from a keepalived+haproxy balancer pair, shared with other related use):
defaults
mode tcp
log global
retries 3
timeout queue 45s
timeout connect 5s
timeout client 5m
timeout server 5m
timeout check 10s
maxconn 1000
balance roundrobin
log-format %ci:%cp\ %fi:%fp\ %bi:%bp\ %si:%sp\ %b/%s\ %U/%B\ %t
listen unoserver_prod
bind 1.2.3.4:2000 name unoserver
default-server inter 20s fastinter 5s downinter 10s
server unosrvp1_2001 1.2.3.5:2001 maxconn 1 check
server unosrvp1_2002 1.2.3.5:2002 maxconn 1 check
server unosrvp2_2001 1.2.3.6:2001 maxconn 1 check
server unosrvp2_2002 1.2.3.6:2002 maxconn 1 check
(In production, we actually use 2 converter servers with 6 libreoffice 7.6.2's running on each.)
last but not least, the unoconv-remote translator script for Moodle (uses a static document formats list generated with unoconv 0.7):
#!/bin/ksh
typeset -i EC=0
CONFIGFILE=/opt/etc/unoconv-remote.env
[[ -s $CONFIGFILE ]] && . $CONFIGFILE
## config defaults
PYTHONPATH=/opt/unoserver-client/lib
DEBUG=${DEBUG:-0}
LOG=${LOG:-/var/log/unoconv-remote/unoconv-remote.log}
UNOCONVERT=${UNOCONVERT:-/opt/unoserver-client/bin/unoconvert}
UNOCVERS=${UNOCVERS:-unoconv 0.7}
FORMATS=${FORMATS:-$(cat /var/lib/unoconv-remote/formats.txt)}
SERVER=${SERVER:-1.2.3.4}
PORT=${PORT:-2000}
## globals
HOST=${HOST:-$(uname -n)}
STARTTS=$(date +%FT%T)
ENDTS=$STARTTS
MISCHEAD="============== $HOST"
CONVHEAD="++++++++++++++ $HOST"
ENDCHEAD="-------------- $HOST"
export PYTHONPATH
function logmsg {
echo "$@" >>$LOG 2>&1
}
function logparams {
typeset -i _i=1
for _p in "$@"; do
print "\$$_i: '$_p'" >>$LOG 2>&1
_i+=1
done
}
case $1 in
--version)
logmsg "$MISCHEAD $STARTTS, version check"
(( $DEBUG > 0 )) && logparams $@
print "$UNOCVERS"
;;
--show)
logmsg "$MISCHEAD $STARTTS, show formats"
(( $DEBUG > 0 )) && logparams $@
print "$FORMATS" 1>&2
;;
-f)
logmsg "$CONVHEAD $STARTTS, conversion start"
(( $DEBUG > 0 )) && logparams $@
logmsg "Input: '$5'"
logmsg "Output: '$4' [$2] via $SERVER:$PORT"
$UNOCONVERT \
--host "$SERVER" \
--port "$PORT" \
--convert-to "$2" \
"$5" - \
2>>"$LOG" > "$4"
EC=$?
ENDTS=$(date +%FT%T)
if (( EC )) && [[ -f "$4" ]]; then
logmsg "WARNING - $ENDTS: non-zero exit for '$5', removing destination file '$4'"
rm "$4"
fi
logmsg "$ENDCHEAD $ENDTS, conversion end in $SECONDS s, exit code $EC"
;;
*)
logmsg "$MISCHEAD $STARTTS, unknown 1st option"
logparams $@
;;
esac
exit $EC
from unoserver.
Glad to hear it!
It would be good to have some place to store configuration tips.
from unoserver.
Related Issues (20)
- Issue with unoconverter initialization: Binary URP bridge disposed during call on Windows 10 HOT 1
- Error code -11 when converting docx to pdf via unoconvert HOT 7
- Details about concurrent executions of unoconv in the README could be useful HOT 5
- Unoserver PV6 Interface Support HOT 3
- Convert a specific PDF page to a PNG HOT 1
- Unoserver Hangs randomly HOT 3
- unoserver sometimes generates errored PDF files without crashing HOT 3
- Support for embedded images in HTML exports HOT 2
- Conversion errors when running in Docker containers HOT 1
- unoserver crash randomly HOT 4
- Setting PNG export resolution? HOT 4
- Setting page number for PNG export? HOT 1
- How to start service properly? HOT 5
- unable to start UnoServer from python HOT 1
- How to apply filter to export data from xslx to csv keeping utf-8 HOT 1
- Ability to set --infilter HOT 2
- DeprecationWarning Is this slated to be fixed? HOT 2
- Troubleshooting unoconvert Network Connection Error on Ubuntu 22.04 with Python 3.10 and LibreOffice 7.3.7.2 in a Closed Network Environment HOT 7
- Why --uno-interface defaulting to --interface value instead of 127.0.0.1 HOT 2
- The value of --host-location ignored for outfile HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from unoserver.