overleaf / clsi Goto Github PK

View Code? Open in Web Editor NEW

197.0 197.0 56.0 4.33 MB

A web api for compiling LaTeX documents in the cloud

License: GNU Affero General Public License v3.0

C 28.36% TeX 11.97% Shell 0.25% Dockerfile 0.09% Makefile 0.39% JavaScript 58.94%

clsi's Introduction

An open-source online real-time collaborative LaTeX editor.

Wiki • Server Pro • Contributing • Mailing List • Authors • License

Figure 1: A screenshot of a project being edited in Overleaf Community Edition.

Community Edition

Overleaf is an open-source online real-time collaborative LaTeX editor. We run a hosted version at www.overleaf.com, but you can also run your own local version, and contribute to the development of Overleaf.

Enterprise

If you want help installing and maintaining Overleaf in your lab or workplace, we offer an officially supported version called Overleaf Server Pro. It also includes more features for security (SSO with LDAP or SAML), administration and collaboration (e.g. tracked changes). Find out more!

Keeping up to date

Installation

We have detailed installation instructions in the Overleaf Toolkit.

Upgrading

If you are upgrading from a previous version of Overleaf, please see the Release Notes section on the Wiki for all of the versions between your current version and the version you are upgrading to.

Overleaf Docker Image

This repo contains two dockerfiles, Dockerfile-base, which builds the sharelatex/sharelatex-base image, and Dockerfile which builds the sharelatex/sharelatex (or "community") image.

The Base image generally contains the basic dependencies like wget and aspell, plus texlive. We split this out because it's a pretty heavy set of dependencies, and it's nice to not have to rebuild all of that every time.

The sharelatex/sharelatex image extends the base image and adds the actual Overleaf code and services.

Use make build-base and make build-community from server-ce/ to build these images.

We use the Phusion base-image (which is extended by our base image) to provide us with a VM-like container in which to run the Overleaf services. Baseimage uses the runit service manager to manage services, and we add our init-scripts from the server-ce/runit folder.

Contributing

Please see the CONTRIBUTING file for information on contributing to the development of Overleaf.

Authors

The Overleaf Team

License

The code in this repository is released under the GNU AFFERO GENERAL PUBLIC LICENSE, version 3. A copy can be found in the LICENSE file.

clsi's People

Contributors

Stargazers

Watchers

clsi's Issues

DockerRunner performance improvement

In the following substitute project with project x user for jailed compiles (disablePerUserCompiles in the web config).

The clsi app creates a new container for each unique command and project. A command could be a compile, word count, sync from pdf, sync from code request.

The first two commands scale OK, as there is only one container per project for the word count and one container per project and doc for the compile request. But the sync commands create a container for each unique lookup - unique per line x column x project (x doc for sync from code).

For the following examples I am using a VPS with dedicated threads of a fairly new server CPU (latest Xeon E5 clocked at 2.4Ghz). The VPS is running CentOS 7, Docker version 19.03.5, build 633a0ea and containerd 1.2.10 b34a5c8af56e510852c35414db4c1f4fa6172339, overlay2 storage driver on xfs. The source of the docker image is hosted on Github.

Creating and starting a new container has a time penalty of about 1000ms.

# time docker run sharelatex/texlive:2017.1-full true

real	0m0.982s

Starting an existing container has a time penalty of about 600ms.

# docker run --name project-xxx sharelatex/texlive:2017.1-full true
# time docker start project-xxx
project-xxx

real	0m0.612s

Another method to run commands in containers is to split the creation of the container and the command execution into two stages.
The first stage creates an idle container [1]. And the second stage uses this idle container as a jailed environment to run commands.

Creating an idle container has a time penalty of about 700ms.

# time docker run -d sharelatex/texlive:2017.1-full sleep 180000
18b229d06cb0d1484b71874bd27c6f5a09a70710b411eadba38d0bf2f8546fbf

real	0m0.645s

Running a command in the idle container has a time penalty of about 300ms.

# time docker exec 076c085fabe08d7a6c6adc5ef3a717135b3ead71af8e9e2ba401ec565a3eb87d true

real	0m0.305s

The second method still has a delay of about 1000ms for the first command per project. But every following command runs at a must faster pace: recompile from 600ms down to 300ms, sync from pdf/code from 1000ms down to 300ms.

Resource usage of idle containers:

metric	usage	comment
RAM	5.25MB	200 container consume about 1050MB
CPU time	650ms	average of the sum of all processes after an hour
PIDs	11	10 PIDs for `containerd` + 1 PID for `cat`/`sleep`

I am aware of your request distribution/scaling via a load agent, haproxy + a cookie, cookie storage in redis per project. For the proposed method the load agent may have to incorporate the memory (and PID [2]) usage as well. It really depends on the resource capacity of your worker nodes.

What do you think about this proposal?

cc @briangough @emcsween @henryoswald @mans0954 @mmazour @ShaneKilkelly

[1]: Using sleep X with a high X - it is prone to race conditions with a low timeout: once the container is started, there is no way to extend the delay in order to preserve to container for another command. X=(DockerRunner.MAX_CONTAINER_AGE=60*60)*50=180000 should give the cleanup task enough tries to cycle the container before it times out. Another option here is to use cat and a fake stdin/tty for the container, but this uses more resources - jailed stdin file handler and polling by cat.
[2]: The PID limit can be bumped up to 4 million via echo X > /proc/sys/kernel/pid_max .

Can't compile due to clsi error

Hello,
My sharelatex installation can't compile latex files.
I'm getting this error in the log :
server error Error: Missing where attribute in the options parameter passed to findOrCreate. Please note that the API has changed, and is now options (an object with where and defaults keys), queryOptions (transaction etc.) at module.exports.Model.findOrCreate (/home/sharelatex/sharelatex/clsi/node_modules/sequelize/lib/model.js:1100:13) at Object.module.exports.ProjectPersistenceManager.markProjectAsJustAccessed (/home/sharelatex/sharelatex/clsi/app/js/ProjectPersistenceManager.js:20:25) at /home/sharelatex/sharelatex/clsi/app/js/CompileController.js:28:42 at Object.module.exports.RequestParser.parse (/home/sharelatex/sharelatex/clsi/app/js/RequestParser.js:50:14) at module.exports.CompileController.compile (/home/sharelatex/sharelatex/clsi/app/js/CompileController.js:23:28) at Layer.handle [as handle_request] (/home/sharelatex/sharelatex/clsi/node_modules/express/lib/router/layer.js:76:5) at next (/home/sharelatex/sharelatex/clsi/node_modules/express/lib/router/route.js:100:13) at /home/sharelatex/sharelatex/clsi/node_modules/body-parser/lib/read.js:107:5 at IncomingMessage.onEnd (/home/sharelatex/sharelatex/clsi/node_modules/body-parser/node_modules/raw-body/index.js:136:7) at IncomingMessage.g (events.js:180:16) at IncomingMessage.EventEmitter.emit (events.js:92:17) at _stream_readable.js:920:16 at process._tickCallback (node.js:415:13)
at /home/sharelatex/sharelatex/clsi/app/js/ProjectPersistenceManager.js:20:25 (https://github.com/sharelatex/clsi-sharelatex/blob/master/app/coffee/ProjectPersistenceManager.coffee#L11) there's indeed no where option passed.
I'm not sure what to put there though.
Is this an actual problem from the code or something else ?

LuaTeX/ConTeXt Support

Any chance of adding the LuaTeX (and therefore ConTeXt) support? LuaTeX is just an extension of pdfTeX, so perhaps it wouldn't be too big of a lift. I'd be happy to try to contribute to try to make it happen if pointed in the right direction.

Can't compile files containing images

Hi,

I'm trying to install Sharelatex in production environment on a CentOS server. It works well so far.
But I can't compile my .tex files when they contain images. I only got a wonderful "Server Error Sorry, something went wrong and your project could not be compiled. Please try again in a few moments." and that's all. I suppose it's a problem with sequelize but I don't know how to solve it.

Here's the clsi logs :

Executing (a4ccea57-981f-405b-9246-701e749bf7ac): BEGIN TRANSACTION;
Executing (a4ccea57-981f-405b-9246-701e749bf7ac): -- SQLite is not able to choose the isolation level REPEATABLE READ.
Executing (a4ccea57-981f-405b-9246-701e749bf7ac): -- SQLite does not support SET autocommit.
Executing (a4ccea57-981f-405b-9246-701e749bf7ac): SELECT `id`, `project_id`, `lastAccessed`, `createdAt`, `updatedAt` FROM `Projects` AS `Project` LIMIT 1;
Executing (a4ccea57-981f-405b-9246-701e749bf7ac): COMMIT;
Executing (default): UPDATE `Projects` SET `id`=1,`project_id`=NULL,`lastAccessed`='2014-10-19 11:48:32.606 +00:00',`createdAt`='2014-10-08 12:58:18.000
+00:00',`updatedAt`='2014-10-19 11:48:32.000 +00:00' WHERE `id`=1
{"name":"clsi","hostname":"asi-latex","pid":30689,"level":30,"project_id":"544241406e7676e677cb92f2","msg":"starting 
compile","time":"2014-10-19T11:48:32.636Z","v":0}
Executing (default): SELECT `id`, `url`, `project_id`, `lastModified`, `createdAt`, `updatedAt` FROM `UrlCaches` AS `UrlCache` WHERE 
`UrlCache`.`url`='http://localhost:3009/project/544241406e7676e677cb92f2/file/544241406e7676e677cb92f5' AND 
`UrlCache`.`project_id`='544241406e7676e677cb92f2' LIMIT 1;
{"name":"clsi","hostname":"asi-latex","pid":30689,"level":30,"url":"http://localhost:3009/project/544241406e7676e677cb92f2/
file/544241406e7676e677cb92f5","lastModified":"2014-10-18T10:30:24.000Z","msg":"downloading URL","time":"2014-10-19T11:48:32.771Z","v":0}
Executing (846916ea-9267-447e-b63b-bc876fcb71a9): BEGIN TRANSACTION;
Executing (846916ea-9267-447e-b63b-bc876fcb71a9): -- SQLite is not able to choose the isolation level REPEATABLE READ.
Executing (846916ea-9267-447e-b63b-bc876fcb71a9): -- SQLite does not support SET autocommit.
Executing (846916ea-9267-447e-b63b-bc876fcb71a9): SELECT `id`, `url`, `project_id`, `lastModified`, `createdAt`, `updatedAt` FROM `UrlCaches` AS
 `UrlCache` LIMIT 1;
Executing (846916ea-9267-447e-b63b-bc876fcb71a9): COMMIT;
{"name":"clsi","hostname":"asi-latex","pid":30689,"level":50,"err":{"message":"Object [object SequelizeInstance],false has no method 
'updateAttributes'","name":"TypeError","stack":"TypeError: Object [object SequelizeInstance],false has no method 'updateAttributes'\n    at Object.<
anonymous> (/var/www/sharelatex/clsi/app/js/UrlCache.js:192:27)\n    at tryCatch1 (/var/www/sharelatex/clsi/node_modules/sequelize/node_modules/sequelize-
bluebird/js/main/util.js:43:21)\n    at Promise$_callHandler [as _callHandler] (/var/www/sharelatex/clsi/node_modules/sequelize/node_modules/sequelize-
bluebird/js/main/promise.js:639:13)\n    at Promise$_settlePromiseFromHandler [as _settlePromiseFromHandler] (/var/www/sharelatex/clsi/node_modules/
sequelize/node_modules/sequelize-bluebird/js/main/promise.js:653:18
)\n    at Promise$_settlePromiseAt [as _settlePromiseAt] (/var/www/sharelatex/clsi/
node_modules/sequelize/node_modules/sequelize-bluebird/js/main/promise.js:817:14)\n    at Promise$_settlePromises [as _settlePromises] (/var/www/sharelatex
/clsi/node_modules/sequelize/node_modules/sequelize-bluebird/js/main/promise.js:951:14)\n    at Async$_consumeFunctionBuffer [as _consumeFunctionBuffer] (/
var/www/sharelatex/clsi/node_modules/sequelize/node_modules/sequelize-bluebird/js/main/async.js:75:12)\n    at Async$consumeFunctionBuffer (/var/www/
sharelatex/clsi/node_modules/sequelize/node_modules/sequelize-bluebird/js/main/async.js:38:14)\n    at process._tickCallback (node.
js:419:13)"},"project_id":"544241406e7676e677cb92f2","msg":"error running compile","time":"2014-10-19T11:48:32.913Z","v":0}

Any ideas ?

Internal Server Error - top level object should have a compile attribute

I have installed CLSI on Ubuntu 18.04 LTS server and the service starts fine.
However, I am receiving "Internal Server Error" message when I run the following command:

curl -d '{
    "compile": {
        "options": {
            "compiler": "pdflatex",
            "timeout": 40 
        },
        "rootResourcePath": "main.tex", 
        "resources": [{
            "path": "main.tex",
            "content": "\\documentclass{article}\n\\begin{document}\nHello World\n\\end{document}"
        }]
    }
}' http://127.0.0.1:3013/project/5dcd0ccc7f44c800012d6ebf/compile

On the CLSI console, I see the following error message:

{"name":"clsi","hostname":"vpsxxxx","pid":16513,"level":50,"err":"top level object should have a compile attribute","url":"/project/5dcd0ccc7f44c800012d6ebf/compile","msg":"server error","time":"2019-11-18T18:11:33.458Z","v":0}
{"name":"clsi","hostname":"vpsxxxx","pid":16513,"level":50,"req":{"url":"/project/5dcd0ccc7f44c800012d6ebf/compile","method":"POST","remote-addr":"127.0.0.1","user-agent":"curl/7.58.0","content-length":"328"},"res":{"statusCode":500},"response-time":15222,"msg":"POST /project/5dcd0ccc7f44c800012d6ebf/compile","time":"2019-11-18T18:11:33.469Z","v":0}

Steps to Reproduce

Run the following CURL command.

curl -d '{
"compile": {
"options": {
"compiler": "pdflatex",
"timeout": 40
},
"rootResourcePath": "main.tex",
"resources": [{
"path": "main.tex",
"content": "\documentclass{article}\n\begin{document}\nHello World\n\end{document}"
}]
}
}' http://127.0.0.1:3013/project/5dcd0ccc7f44c800012d6ebf/compile

Expected Behaviour

A response in the following format.

{
"compile": {
"status": "success",
"outputFiles": [{
"type": "pdf",
"url": "http://localhost:3013/project/5dcd0ccc7f44c800012d6ebf/output/output.pdf"
}, {
"type": "log",
"url": "http://localhost:3013/project/5dcd0ccc7f44c800012d6ebf/output/output.log"
}]
}
}

Observed Behaviour

Internal Server Error

Context

Unable to use the CLSI service.

Technical Info

URL: N/A.
Browser Name and version:
Operating System and version (desktop or mobile): Ubuntu 18.04 desktop

sqlite error: "SQLITE_BUSY: database is locked"

Hi, great job with ShareLatex, it's a very interesting project.
In the last day I start experiencing an issue with the compiler module. Here's the the error stack:
[2016-02-04T17:21:34.331Z] ERROR: clsi/87485 on Mac.local: error downloading file for resources (project_id=566f5b422cd1e813de233c40, modified=2016-02-04T17:21:33.470Z) SequelizeTimeoutError: SQLITE_BUSY: database is locked at module.exports.Query.formatError (/Users/luca/lsharelatex/clsi/node_modules/sequelize/lib/dialects/sqlite/query.js:236:16) at Statement.<anonymous> (/Users/luca/lsharelatex/clsi/node_modules/sequelize/lib/dialects/sqlite/query.js:47:31) at Statement.replacement (/Users/luca/lsharelatex/clsi/node_modules/sqlite3/lib/trace.js:20:31) -- path: /Users/luca/lsharelatex/data/compiles/566f5b422cd1e813de233c40/ce0test1m1fig1.png -- resource_url: http://localhost:3009/project/566f5b422cd1e813de233c40/file/56b3889d70364cdb55f76abd

Near a week ago it worked, and in that moment I also built a docker image that it is currently working. But now, I have this problem both: in my development environment and also in the docker builds that I made.

I haven't make any change on this component, so I think also you could be experiencing the same issue.

Compiling on Overleaf, but using favorite Desktop IDE?

Is it possible to send latex files to the Overleaf server and get the compiled PDF back?

This is particularly useful when someone wants to use their favorite desktop IDE, but doesn't wanna install Latex locally.

[email protected] install fails on [email protected]

Had to change to v3.0.8 (latest) to install on my machine with node v0.12

smoketest periodically fails

First and foremost, thank you for your work.

We are trying to deploy sharelatex on our server for an academic purpose. Maybe it is not the correct way, but we start our instance of sharelatex by running 'grunt run' via systemd. The web app works correctly and we can create and compile a project. Nevertheless, we can notice by looking into the logs that a smoketest is run and fails periodically:

INFO: clsi/10700 on archlinux: running smoke tests
Executing (default): SELECT * FROM `Projects` WHERE `Projects`.`project_id`='smoketest' LIMIT 1;
Executing (default): UPDATE `Projects` SET `id`=1,`project_id`='smoketest',`lastAccessed`='2014-06-11 12:10:24',`createdAt`='2014-06-10 11:58:46',`updatedAt`='2014-06-11 12:10:24' WHERE `id`=1
INFO: clsi/10700 on archlinux: starting compile (project_id=smoketest)
INFO: clsi/10700 on archlinux: written files to disk (project_id=smoketest, time_taken=20)
INFO: clsi/10700 on archlinux: starting compile (directory=/usr/share/webapps/sharelatex/compiles/smoketest, compiler=pdflatex, timeout=60000, mainFile=main.tex)
INFO: clsi/10700 on archlinux: running command (project_id=smoketest, directory=/usr/share/webapps/sharelatex/compiles/smoketest)
command: [
"latexmk",
"-cd",
"-f",
"-jobname=output",
"-auxdir=/usr/share/webapps/sharelatex/compiles/smoketest",
"-outdir=/usr/share/webapps/sharelatex/compiles/smoketest",
"-pdf",
"-e",
"$pdflatex='pdflatex -synctex=1 -interaction=batchmode %O %S'",
"/usr/share/webapps/sharelatex/compiles/smoketest/main.tex"
]
WARN: clsi/10700 on archlinux: timeouts and sandboxing are not enabled with CommandRunner
Latexmk: This is Latexmk, John Collins, 10 Nov 2013, version: 4.39.
*** Report bugs etc to John Collins <collins at phys.psu.edu>. ***
Latexmk: Changing directory to '/usr/share/webapps/sharelatex/compiles/smoketest/'
Rule 'pdflatex': File changes, etc:
   Non-existent destination files:
      '/usr/share/webapps/sharelatex/compiles/smoketest/output.pdf'
------------
Run number 1 of rule 'pdflatex'
------------
------------
Running 'pdflatex -synctex=1 -interaction=batchmode  -recorder -output-directory="/usr/share/webapps/sharelatex/compiles/smoketest" --jobname=output  "main.tex"'
------------
Latexmk: applying rule 'pdflatex'...
This is pdfTeX, Version 3.1415926-2.5-1.40.14 (TeX Live 2013/Arch Linux)
restricted \write18 enabled.
entering extended mode
SyncTeX written on /usr/share/webapps/sharelatex/compiles/smoketest/output.synctex.gz.'output.fls' and '/usr/share/webapps/sharelatex/compiles/smoketest/output.fls' are identical (not copied) at /usr/sbin/latexmk line 6454.
Latexmk: Log file says output to 'output.pdf'
Latexmk: ===For rule 'pdflatex', actual output 'output.pdf'
======appears not to match expected output '/usr/share/webapps/sharelatex/compiles/smoketest/output.pdf'
Latexmk: Undoing directory change
Latexmk: All targets (/usr/share/webapps/sharelatex/compiles/smoketest/output.pdf) are up-to-date
INFO: clsi/10700 on archlinux: done compile (project_id=smoketest, time_taken=1288)
INFO: clsi/10700 on archlinux: http request (response-time=1418, req.remote-addr=127.0.0.1, req.content-length=133)
POST /project/smoketest/compile HTTP/1.1

As result, the grunt process which starts the clsi app grows unexpectedly (it fills the memory/swap) as well as the sharelatex dedicated log file (more than 10MB for a day).

Include latexmk with the CLSI

Many people have TexLive 2011 installed (Ubuntu 12.04 default), which doesn't work due to latexmk not understanding some newer options. If we include the latest latexmk script with the CLSI and reference it locally, then everyone should have the same consistent experience.

See for example overleaf/overleaf#27 for the problem with an old version of latexmk.

qpdf not installed

I was facing the following error and I fixed it by installing the package qpdf.
I'm running sharelatex on a debian 8 and this program is neither installed by default nor installed by texlive-full package. It might be good to add qpdf as a dependency in the wiki page https://github.com/sharelatex/sharelatex/wiki/Dependencies

[2015-05-27T19:48:52.975Z]  WARN: clsi/15135 on sharelatex: qpdf failed
    Error: spawn qpdf ENOENT
        at exports._errnoException (util.js:746:11)
        at Process.ChildProcess._handle.onexit (child_process.js:1053:32)
        at child_process.js:1144:20
        at process._tickCallback (node.js:355:11)
    --
    args: [
      "--linearize",
      "/home/sharelatex/sharelatex/data/compiles/5565cae1f83ff2115679dafd/output.pdf",
      "/home/sharelatex/sharelatex/data/compiles/5565cae1f83ff2115679dafd/.cache/clsi/14d96eb9c21/output.pdf.opt"
    ]
[2015-05-27T19:48:52.978Z]  WARN: clsi/15135 on sharelatex: qpdf returned error (code=-2)
    args: [
      "--linearize",
      "/home/sharelatex/sharelatex/data/compiles/5565cae1f83ff2115679dafd/output.pdf",
      "/home/sharelatex/sharelatex/data/compiles/5565cae1f83ff2115679dafd/.cache/clsi/14d96eb9c21/output.pdf.opt"
    ]

Accented characters in file/folder names cause compile to file

I think this is a problem in latexmk, but it may be something we are messing up internally in the CLSI. It will hopefully be a quick fix once the problem is tracked down.

Can't compile tex files containing images

I am developing an android LaTeX editor. Whenever i try to compile tex file containing images, it shows "Internal Server Error"

CLSI log shows:

ERROR: clsi/4510 : server error
SyntaxError: Unexpected string
at Object.parse (native)
at parse (/home/name/clsi-sharelatex/node_modules/body-parser/lib/types/json.js:84:17)
at /home/name/clsi-sharelatex/node_modules/body-parser/lib/read.js:102:18
at IncomingMessage.onEnd (/home/name/clsi-sharelatex/node_modules/body-parser/node_modules/raw-body/index.js:136:7)
at IncomingMessage.g (events.js:180:16)
at IncomingMessage.EventEmitter.emit (events.js:92:17)
at _stream_readable.js:920:16
at process._tickCallback (node.js:415:13)
Thu, 16 Apr 2015 14:49:08 GMT express deprecated res.send(status): Use res.sendStatus(status) instead at app.js:136:16

Can anyone help me to resolve this problem?

Smoke tests not compiled for Docker image in ho-dockerise branch

In commit b5a7eab, compiling of smoke tests was removed from compile:all in package.json. This should either be restored, or a call to compile:smoke_tests made in the Dockerfile.

juldntivv

clsi fails to run on CentOS 7

When I open a document in Sharelatex clsi fails with 3 errors(see log).

Everything else in Sharelatex seems to run perfectly fine.

Both mongodb & texlive is up to date. The rest of the dependencies are from CentOS 7 default repository.

Full log with changed domain & user name:
http://pastebin.com/L4zKdjE6

Hack: pandoc

Our (@gh2k @Hugh-OBrien @ShaneKilkelly) hackathon project.

Basic idea is to use pandoc to compile other file formats. So far we have gotten markdown and word files to compile to pdf. Hoping to also be able to compile most formats to html view more-web-like viewing on RHS of editor.

Work took place on branch pandoc-hack

Timeout received while compiling document

Hi, I've been trying to compile a document in my instance of sharelatex but I receive a "Server Error" in the browser and a timeout on the server.

Server Error Sorry, something went wrong and your project could not be compiled. Please try again in a few moments.

[2015-06-09T21:43:45.655Z] ERROR: clsi/28304 on sharelatex: error downloading url
    Error: ETIMEDOUT
        at null._onTimeout (/home/sharelatex/sharelatex/clsi/node_modules/request/index.js:599:15)
        at Timer.listOnTimeout [as ontimeout] (timers.js:112:15)
    --
    url: http://localhost:3009/project/5577211e5fcc401c4dc09ea2/file/557721215fcc401c4dc09ea4
    --
    filePath: /home/sharelatex/sharelatex/data/cache/5577211e5fcc401c4dc09ea2:0136d84ba919fd155df44bea75ab062b
[2015-06-09T21:43:45.656Z] ERROR: clsi/28304 on sharelatex: error running compile (project_id=5577211e5fcc401c4dc09ea2)
    Error: ETIMEDOUT
        at null._onTimeout (/home/sharelatex/sharelatex/clsi/node_modules/request/index.js:599:15)
        at Timer.listOnTimeout [as ontimeout] (timers.js:112:15)
[2015-06-09T21:43:45.661Z]  INFO: clsi/28304 on sharelatex: http request (response-time=61355, req.remote-addr=127.0.0.1, req.content-length=124480)
    POST /project/5577211e5fcc401c4dc09ea2/compile HTTP/1.1
[2015-06-09T21:43:45.663Z] ERROR: clsi/28304 on sharelatex: error downloading url
    Error: socket hang up
        at createHangUpError (http.js:1473:15)
        at Socket.socketCloseListener (http.js:1523:23)
        at Socket.emit (events.js:117:20)
        at TCP.close (net.js:466:12)

[2015-06-09T22:33:50.329Z] ERROR: web-sharelatex/28298 on sharelatex: CLSI returned failure code (project_id=5577211e5fcc401c4dc09ea2)
    Error: CLSI returned non-success code: 500
      at Request._callback (/home/sharelatex/sharelatex/web/app/js/Features/Compile/ClsiManager.js:88:19)
      at Request.self.callback (/home/sharelatex/sharelatex/web/node_modules/request/request.js:121:22)
      at Request.emit (events.js:98:17)
      at Request.<anonymous> (/home/sharelatex/sharelatex/web/node_modules/request/request.js:978:14)
      at Request.emit (events.js:117:20)
      at IncomingMessage.<anonymous> (/home/sharelatex/sharelatex/web/node_modules/request/request.js:929:12)
      at IncomingMessage.emit (events.js:117:20)
      at _stream_readable.js:944:16
      at process._tickDomainCallback (node.js:492:13)

[2015-06-09T22:33:50.330Z] ERROR: web-sharelatex/28298 on sharelatex: error passed to top level next middlewear (url=/project/5577211e5fcc401c4dc09ea2/compile, method=POST)
    Error: CLSI returned non-success code: 500
      at Request._callback (/home/sharelatex/sharelatex/web/app/js/Features/Compile/ClsiManager.js:88:19)
      at Request.self.callback (/home/sharelatex/sharelatex/web/node_modules/request/request.js:121:22)
      at Request.emit (events.js:98:17)
      at Request.<anonymous> (/home/sharelatex/sharelatex/web/node_modules/request/request.js:978:14)
      at Request.emit (events.js:117:20)
      at IncomingMessage.<anonymous> (/home/sharelatex/sharelatex/web/node_modules/request/request.js:929:12)
      at IncomingMessage.emit (events.js:117:20)
      at _stream_readable.js:944:16
      at process._tickDomainCallback (node.js:492:13)

I've tried all suggestions on https://pt.sharelatex.com/learn/Debugging_Compilation_timeout_errors, but no success.

The weird thing is: it compiles successfully on sharelatex.com! Any idea what might be happening?

how to invoke LaTeX with the -shell-escape flag

I'm trying to compile this code, but I get an error. ! Package minted Error: You must invoke LaTeX with the -shell-escape flag.

\documentclass{article}
\usepackage{minted}
\begin{document}
\begin{minted}{python}
print("hello world")
\end{minted}
\end{document}

How can I invoke shell-escape flag in CLSI

Synctex Doesn't Work

On my Ubuntu 14.04 LTS Virtual Machine, when I try to find a position in the PDF from source, or find a position in the source from the PDF, by double-clicking in the PDF or clicking the arrow keys, nothing happens. I assume this has to do with Synctex not being found by the CLSI application.

I installed Sharelatex and its dependencies (including Texlive) on Ubuntu 14.04 using the installation instructions for Ubuntu 13.03.

When I try this functionality, the following logs are produced (notice the error from CLSI):

[2014-08-12T22:44:26.444Z]  INFO: web-sharelatex/1147 on corriv: Checking if can access
    user: {
      "id": "53def8672a2968d8209f6742",
      "email": "[email protected]"
    }
    --
    project: {
      "id": "53e534d75cfc01960965b2ec",
      "name": "Test"
    }
[2014-08-12T22:44:26.444Z]  INFO: web-sharelatex/1147 on corriv: proxying to CLSI
    url: /project/53e534d75cfc01960965b2ec/sync/code?column=240&file=main.tex&line=232
[2014-08-12T22:44:26.459Z]  INFO: web-sharelatex/1147 on corriv: http request (response-time=21, req.remote-addr=127.0.0.1)
    GET /project/53e534d75cfc01960965b2ec/sync/code?column=240&file=main.tex&line=232 HTTP/1.1
    --
    req.referrer: http://<server>/project/53e534d75cfc01960965b2ec
    --
    req.user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.77.4 (KHTML, like Gecko) Version/7.0.5 Safari/537.77.4
[2014-08-12T22:44:34.309Z]  INFO: web-sharelatex/1147 on corriv: marking user as connected (project_id=53e534d75cfc01960965b2ec, client_id=48-jsQAIE77SYg0fXM0l)
2014-08-12T22:44:41.202Z]  INFO: web-sharelatex/1147 on corriv: mongo request (query_type=query, collection=sharelatex.projects, response-time=3)
    query: {
      "_id": "53e534d75cfc01960965b2ec"
    }
[2014-08-12T22:44:41.219Z] ERROR: clsi/1163 on corriv: server error
    Error: spawn ENOENT
        at errnoException (child_process.js:1001:11)
        at Process.ChildProcess._handle.onexit (child_process.js:792:34)
[2014-08-12T22:44:41.209Z]  INFO: web-sharelatex/1147 on corriv: mongo request (query_type=query, collection=sharelatex.users, response-time=2)
    query: {
      "_id": "53def8672a2968d8209f6742"
    }
[2014-08-12T22:44:41.210Z]  INFO: web-sharelatex/1147 on corriv: Checking if can access
    user: {
      "id": "53def8672a2968d8209f6742",
      "email": "[email protected]"
    }
    --
    project: {
      "id": "53e534d75cfc01960965b2ec",
      "name": "Test"
    }
[2014-08-12T22:44:41.227Z]  INFO: clsi/1163 on corriv: http request (response-time=13, req.remote-addr=127.0.0.1)
    GET /project/53e534d75cfc01960965b2ec/sync/code?column=256&file=main.tex&line=213 HTTP/1.1
[2014-08-12T22:44:41.210Z]  INFO: web-sharelatex/1147 on corriv: proxying to CLSI
    url: /project/53e534d75cfc01960965b2ec/sync/code?column=256&file=main.tex&line=213
[2014-08-12T22:44:41.227Z]  INFO: web-sharelatex/1147 on corriv: http request (response-time=29, req.remote-addr=127.0.0.1)
    GET /project/53e534d75cfc01960965b2ec/sync/code?column=256&file=main.tex&line=213 HTTP/1.1
    --
    req.referrer: http://<server>/project/53e534d75cfc01960965b2ec
    --
    req.user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.77.4 (KHTML, like Gecko) Version/7.0.5 Safari/537.77.4

Run clsi as revice fails with 'Headers already sent'

echo "node /clsi-sharelatex/app.js" | sudo -i -u apache

then when you request a compilation of latex you see in logs

http.js:724
    throw new Error('Can\'t set headers after they are sent.');
          ^
Error: Can't set headers after they are sent.
    at ServerResponse.OutgoingMessage.setHeader (http.js:724:11)
    at ServerResponse.header (/clsi-sharelatex/node_modules/express/lib/response.js:719:10)
    at ServerResponse.send (/clsi-sharelatex/node_modules/express/lib/response.js:164:12)
    at ServerResponse.json (/clsi-sharelatex/node_modules/express/lib/response.js:250:15)
    at ServerResponse.send (/clsi-sharelatex/node_modules/express/lib/response.js:152:21)
    at /clsi-sharelatex/app/js/CompileController.js:65:44
    at /clsi-sharelatex/app/js/CompileManager.js:177:24
    at /clsi-sharelatex/app/js/OutputCacheManager.js:127:15
    at /clsi-sharelatex/node_modules/async/lib/async.js:229:13
    at async.eachSeries (/clsi-sharelatex/node_modules/async/lib/async.js:127:20)
    at _asyncMap (/clsi-sharelatex/node_modules/async/lib/async.js:223:9)
    at Object.mapSeries (/clsi-sharelatex/node_modules/async/lib/async.js:213:23)
    at /clsi-sharelatex/app/js/OutputCacheManager.js:87:24
    at /clsi-sharelatex/node_modules/fs-extra/lib/mkdir.js:29:14
    at Object.oncomplete (fs.js:108:15)

Failing compiling. Wrong pointer type

Step 22/31 : RUN cd /var/www/sharelatex/clsi && grunt compile:bin; chown -R www-data:www-data /var/www/sharelatex
 ---> Running in a059da895ba6
Running "compile:bin" task
src/synctex/synctex_parser.c: In function 'synctex_updater_new_with_output_file':
src/synctex/synctex_parser.c:4174:52: warning: passing argument 4 of '_synctex_open' from incompatible pointer type [-Wincompatible-pointer-types]
  if (_synctex_open(output,build_directory,&synctex,&SYNCTEX_FILE,synctex_ADD_QUOTES,&io_mode)
                                                    ^
src/synctex/synctex_parser.c:2738:5: note: expected 'struct gzFile_s **' but argument is of type 'void **'
 int _synctex_open(const char * output, const char * build_directory, char ** synctex_name_ref, gzFile * file_ref, synctex_bool_t add_quotes, synctex_io_mode_t * io_mode_ref) {
     ^~~~~~~~~~~~~
src/synctex/synctex_parser.c:4175:52: warning: passing argument 4 of '_synctex_open' from incompatible pointer type [-Wincompatible-pointer-types]
   && _synctex_open(output,build_directory,&synctex,&SYNCTEX_FILE,synctex_DONT_ADD_QUOTES,&io_mode)) {
                                                    ^
src/synctex/synctex_parser.c:2738:5: note: expected 'struct gzFile_s **' but argument is of type 'void **'
 int _synctex_open(const char * output, const char * build_directory, char ** synctex_name_ref, gzFile * file_ref, synctex_bool_t add_quotes, synctex_io_mode_t * io_mode_ref) {
     ^~~~~~~~~~~~~

Compiling on Ubuntu 18.04.

Acceptance tests fail

The acceptance tests include the compilation of a set of example pdfs. The used texlive image is not public, it's name hints to texlive 2017.

Some of the examples require additional software, e.g. gnuplot, which in turn requires additional probably insecure white listing in the texmf config [1].

After hours of trying to get the examples to compile successfully, I tried the hosted version of sharelatex.
And it turns out that many of the examples do not succeed over there as well.

OSS: texlive 2017 + ghostscript, ci tests of clsi
Hosted: overleaf.com, manual submission via web interface, visual comparison

example	OSS	Hosted
`asymptote`	✖️	✔️
`biber_bibliography`	✔️	✔️
`epstopdf`	✔️	✔️
`feynmf`	✖️	✖️
`feynmp`	✖️	✔️
`fontawesome`	✔️	✔️
`fontawesome_xelatex`	✔️	✔️
`glossaries`	✖️	✖️
`gnuplot`	✅ [1]	✖️
`hebrew`	✖️	✔️
`knitr`	✖️	✔️
`knitr_utf8`	✖️	✔️
`latex_compiler`	✖️	✔️
`lualatex_compiler`	✔️	✔️
`makeindex`	✔️	✖️
`makeindex-custom-style`	✖️	✖️
`minted`	✅ [5]	✖️
`multibib_bibliography`	✔️	✔️
`nomenclature`	✖️	✖️
`references_in_include`	✔️	✔️
`simple_bibliography`	✔️	✔️
`subdirectories`	✔️	❔ [3]
`tikz_feynman`	✔️	✔️
`xelatex_compiler`	✔️	✔️

NOTES:

the visual comparison is probably not as accurate as the ci test.
replicating the manual tests via the web interface will take an hour+.

[1] das7pad/sharelatex-docker-images@295594c
[2] https://www.overleaf.com/read/jwytqrdxkvgt

clone the project to be able to make the following changes per example:
- adjust the main tex file
- adjust the compiler to the one listed in the options.json file of the example

[3] It is not possible to compile a directory via the web interface. The subdirectory example has it's main tex file in a subdirectory, while the main file tries to include from the root project directory. This is probably the reason for the failed compilation via the web interface. A dedicated project does not help either [4].
[4] https://www.overleaf.com/read/vjywyfpmqjzp
[5] das7pad/sharelatex-docker-images@a13e040...6b19ce5

Support custom timezones

When I use macros like \today, ShareLatex produces the document using server's system date.
Since the user can be in a different timezone, the document may be produced with an unexpected date.

To illustrate the problem, the current time where I live is May 17, 2015, 11:00pm (Brazil oficial time, GMT-3), but when I compile a document in ShareLatex, \today returns "May 18, 2015".

I think the API should have a timezone setting, so the datetime can be adjusted before compiling the document. The UI can also have an option to allow the user to select its timezone.

Support pandoc

I'm not sure I'm creating issue in a relevant repo, please move it if necessary.

I just came across the wonderful ShareLatex and I would like to use it with some of my projects. However, I do not use pure LaTeX, but instead have a .tex template and a few Pandoc Markdown files to build with pandoc.

An example of such a project is available in this repo. Although it's in Russian, you can see that PDF can be constructed with a single pandoc command.

Another example is this CV template (it's documented in English).

I find pandoc very useful since source code becomes much cleaner and easier to maintain.
It also provides ways to render output in many other formats (which could also be a nice ShareLatex feature on its own).

It would be very nice if ShareLatex could provide pandoc compiler!

I understand that my request may not be compatible with ShareLatex's idea and perhaps I just want a different service (SharePandoc?).

Document compilation problem

Similar symptoms have already been reported, but seemingly without a solution.

We have a Ubuntu 14.04 x86_64 desktop. Tex Live was installed manually today and latexmk 4.39 is available on the PATH. Sharelatex installation (release 0.1.1) went smoothly and the only issue we are experiencing is that documents do not get compiled.

Here are our clsi.log, web.log and filestore.log and here is our settings.coffee. The error stack in clsi.log looks like:

Error: URL returned non-success status code: 500 http://localhost:3009/project/5474b72ead9c82e062df051b/file/5474b72fad9c82e062df051e
    at Request.<anonymous> (/var/www/sharelatex/clsi/app/js/UrlFetcher.js:26:31)
    at Request.EventEmitter.emit (events.js:95:17)
    at Request.onResponse (/var/www/sharelatex/clsi/node_modules/request/index.js:815:10)
    at ClientRequest.g (events.js:180:16)
    at ClientRequest.EventEmitter.emit (events.js:95:17)
    at HTTPParser.parserOnIncomingClient [as onIncoming] (http.js:1688:21)
    at HTTPParser.parserOnHeadersComplete [as onHeadersComplete] (http.js:121:23)
    at Socket.socketOnData [as ondata] (http.js:1583:20)
    at TCP.onread (net.js:525:27)

The dirs seem to be OK:

$ ls -l /var/lib/sharelatex/data/
drwxr-xr-x 2 sharelatex sharelatex 4096 Nov 25 18:51 cache
drwxr-xr-x 3 sharelatex sharelatex 4096 Nov 25 18:51 compiles
-rw-r--r-- 1 sharelatex sharelatex 4096 Nov 25 19:06 db.sqlite
drwxr-xr-x 2 sharelatex sharelatex 4096 Nov 25 19:05 user_files

Thanks in advance!

How to bind port 3013 to external IP，such as 0.0.0.0 ?

Sorry, I'm not JavaScript coder and not familiar with the JS syntax. I just want to know when I type "grunt run", How can I make the server process linstening 3013 binding on external IP，such as 0.0.0.0.
I deploy the clsi-sharelatex on the ssh-interactive linux virtual-machine.
Thanks very much for your answer.

Provide sqlite as a database option

This would make it much easier to set up out of the box, since configuring a new MySQL database to use is a non-negligible task.

A valid Gruntfile could not be found.

I'm trying to install the CLSI on my personal pc running ubuntu and on a GCP VM but I always get this error: A valid Gruntfile could not be found.
sorry, this is the first time to use grunt !!
Thanks very much for your answer.

Steps to Reproduce

git clone https://github.com/overleaf/clsi.git
npm install
grunt install

Context

I'm trying to install the CLSI on my ubuntu desktop and GCP VM.

Technical Info

A valid Gruntfile could not be found. Please see the getting started guide for
more information on how to configure grunt: http://gruntjs.com/getting-started

Operating System and version: Ubuntu 18.04 on my PC / using google cloud VM

Multiple issues in dependancies install

Hi,

I am having difficulties in deploying this web-api on a VPS.

The OS is Linux Debian Jessie 8.6.

I first installed Node 0.10.13 and had some errors that seemed version related.
Then installed Node 10.10.0 and still had errors that seemed version related.
Then installed Node 6.14.4 and got rid of what seemed to be the version problems.

I had to install grunt-cli manually.

When using "grunt-run" i get error message "/root/LatexCompiler/clsi-sharelatex/node_modules/sequelize/lib/dialects/sqlite/connection-manager.js:31
throw new Error('Please install sqlite3 package manually');"

I tried installing sqlite3 manually with:
sudo apt-get install sqlite3
and still get the same error.

What do i need to do ?
Maybe the readme is missing some steps to get this API running ?
What versions of node/npm are you supporting ?

Thank you

overleaf / clsi Goto Github PK

clsi's Introduction

An open-source online real-time collaborative LaTeX editor.

Community Edition

Enterprise

Keeping up to date

Installation

Upgrading

Overleaf Docker Image

Contributing

Authors

License

clsi's People

Contributors

Stargazers

Watchers

Forkers

clsi's Issues

Steps to Reproduce

Expected Behaviour

Observed Behaviour

Context

Technical Info

Steps to Reproduce

Context

Technical Info

Recommend Projects

Recommend Topics

Recommend Org