thomasleister / prosody-filer Goto Github PK

Hi, thanks for your server set up. I noticed with the attachments that the storage location has an index.html file in the root directory (per user?), so actually once you get into the server you get into everything that was ever sent.
Are you doing this at all?
https://github.com/ThomasLeister/prosody-filer#automatic-purge

Maybe -max-depth 0 isn't what you wanted?

I set everything as explained.
I use "Alternative configuration for letting Nginx serve the uploaded files"

In "Systemd service file", if I set Group=nginx I get:
systemd[1]: Failed to start Prosody file upload server.
If I set Group=prosody-filer
Then it works fine...

Hi, thanks for this useful piece of Software. I have a small problem and I would like to ask for your review and help:

xmpp-filer_1              | 2020/02/29 08:20:29 Incoming request: PUT /_xmpp/upload/57bb1d75-2bd7-46b2-8360-55301d168628/RECORDING_20200227_193520573.m4a?v=d88af5b00418ba437dd063be44699bb24b04a9bd9770406e5a6fd7e13bf9f457
xmpp-filer_1              | MAC sent:  d88af5b00418ba437dd063be44699bb24b04a9bd9770406e5a6fd7e13bf9f457
xmpp-filer_1              | 2020/02/29 08:20:29 fileStorePath: 57bb1d75-2bd7-46b2-8360-55301d168628/RECORDING_20200227_193520573.m4a
xmpp-filer_1              | 2020/02/29 08:20:29 ContentLength: 36677
xmpp-filer_1              | 2020/02/29 08:20:29 Invalid MAC.

It runs on this dockerized setup together with Prosody 11.2 and the file sent was triggered by a resend Conversations client. It lives behind a Traefik reverse proxy.

It has worked great. And suddenly started having this problem. I upgraded to the latest master of the prosody-filer and tried again. Still having the same problem. Do you have any idea? Thank you!

(btw, I saw the other couple of issues around invalid MACs, but the problems there doesn't seem to apply to my setup)

It´s not possible to build this software by running

$ go build main.go

as there is no main.go file

Changing it to the more correct prosody-filer.go also doesn´t work

$ go build prosody-filer.go
prosody-filer.go:25:2: no required module provides package github.com/BurntSushi/toml: go.mod file not found in current directory or any parent directory; see 'go help modules'

This seems to be ( and I might be wrong, I don´t know much about go) because this software does not use a current golang way of packaging.

In addition the toml parser library this project uses had a big deprecation warning on the top of it´s readme.

If prosody-flyer is not in a usable state, maybe it makes sense to add a notice to the README.

I'm using the user provided Apache config and it complains about H2Direct on not being recognized, so I comment it out and Apache works, but when I try to upload something I get error 405 (method not allowed) on Gajim and Conversations.

https://golang.org/doc/go-get-install-deprecation

Instructions for compiling should be updated

I guess something went wrong here: https://github.com/ThomasLeister/prosody-filer/releases/tag/untagged-fb42cdf57e84b60fb7f8

What is the difference between
client_max_body_size 50m; in /etc/nginx/sites-available/uploads.myserver.tld
and
http_upload_external_file_size_limit = 50000000 -- 50 MB in /etc/prosody/prosody.cfg.lua

Should be change both when we want to change the file size limit?

Sadly, OMEMO doesn't hide the filename from the server, despite this meta datum can already contain sensitive information.

The URL contains a random value to prevent listing existing files via brute force, so it has high entropy.
Every request of the file via an XMPP client is done via the URL. So, instead of storing the file in the filesystem by its name, a hash of the URL can be used as name. This prevents the admin (or attackers getting read access to the filesystem) to get potentially sensitive meta data.
As encoding for the hash, I propose base32(hex) without padding to keep it short and safe even for systems with case-insensitive filesystems.

Instead of the full URL, only the path or even just the path info (see PATH_INFO at https://www.php.net/reserved.variables.server).

This could be even extended to use hkdf to derive 2 keys: one for obfuscating the filename and a second one to encrypt the content of the file - in case the uploader didn't use something like OMEMO.

It would be best if OMEMO would not only encrypt the file's content but also hides the file name and the content type. But as it's not the case, this can add an additional layer of protection. It, however, cannot protect against malicious admins as the admin could (a) read the process memory at the time of a file up-/download or (b) could just disable it or use a different software.

It seems that you currently do not sign the git tags or the published binary.
To enable (semi-)automatic updates of prosody-filer in production, it would be nice to have some way to automatically verify that the sources used to build the binary or the downloaded binary itself is indeed still coming from you ;)

I was trying to package 1.0.2 for Alpine. I got:

prosody-filer.go:25:2: no required module provides package github.com/BurntSushi/toml: go.mod file not found in current directory or any parent directory; see 'go help modules'

Which seems to be fixed in the latest develop head:

#27

Please cut a new release so that this is fixed.

Hi,

Just want to suggest to add in the howto a line explaining that there should be an uploads folder created in /home/prosody-filer and that it should belongs to the prosody-filer user
mkdir /home/prosody-filer/uploads
chown prosody-filer:prosody-filer /home/prosody-filer/uploads

In the readme file, it is written:
# Group=nginx # if the files should get served by nginx directly:

What would be the advantages over letting the default setting to Group=prosody-filer

SELinux prevents you from running a system service where the binary is in a user's home directory, or in your case, the root user's home directory.

To fix the problem, copy the binary to a proper directory such as /usr/local/bin and call it from there.

I'm sorry, perhaps this request is just not possible:
when an upload is delete with the purge script, the message that included the link to this file still exist. That means that if someone didn't receive the file before it was deleted (meaning it is in the client storage), this person sees a message with a link to the file (at least on gajim), but when he/she clicks on it, he/she gets a 404 error, as the file doesn't exist anymore.

Is that something that could be fixed easily?

By the way, thanks for the great work, prosody-filter works like a charm!

Hi,

in your readme file, you say to create a job cron.
I however have an issue with the purge command suggested.
I tried find /home/prosody-filer/upload -maxdepth 0 -type d just to see what I get. And I get /home/prosody-filer/upload
Shouldn't it be find /home/prosody-filer/upload/* -maxdepth 0 -type d ? Like this, I get results...

Hi,

all file uploads are repsonse with invalid mac:

Feb 01 02:09:24 prosody-filer[34172]: MAC sent:  596rogbetoh3hteb
Feb 01 02:09:24 prosody-filer[34172]: 2019/02/01 02:09:24 fileStorePath: upload/zko4b3j kbrv/1.png
Feb 01 02:09:24 prosody-filer[34172]: 2019/02/01 02:09:24 ContentLength: 26856
Feb 01 02:09:24 prosody-filer[34172]: 2019/02/01 02:09:24 Invalid MAC.

With the last release candidat are not this problem. Give it a reason for this problem?

The v2 protocol is described here: https://modules.prosody.im/mod_http_upload_external.html

To activate it in Prosody you need to put this into the config:
http_upload_external_protocol = "v2"

Hi,

why are files uploaded in a folder inside of /uploads ? WHy are they not directly uploaded in /uploads, without creating subfolders?

[Anon]:

damit es auch für Web-Clients funktioniert, würde ich auch OPTIONS-Requests beantworten und in der Antwort einen 'Allow'-Header sowie die hier aufgelisteten CORS-Header mitliefern (letztere auch für HEAD/GET/PUT): https://github.com/movim/movim/wiki/Configure-ejabberd
Bzw. einfach Custom-Headers User-konfigurierbar machen.

• Support CORS
• Support OPTIONS

The section on why to use this module is outdated.
Prosody now ships mod_http_file_share, that doesn't have the mentioned limitations of the old community module.

https://github.com/ThomasLeister/prosody-filer?tab=readme-ov-file#why-should-i-use-this-server

It would sound more professional to remove that section.

I keep my configs public and with gobs of services I've used, this is the first that seems to only allow hardcoded values in its file. Either allowing to be read from a file, an environment variable (could be used with systemd), or UNIX command would help clear this up.

There seems to be a bug in Prosody filer which leads to wrong MAC calculation and therefore blocks some uploads once in a while. Re-trying to upload the file solves the issue in most cases. Sometimes more that one retry is needed until successful upload.

I could not recreate the error case so far - it seems to be quite random to me.

Maybe this is related to any race conditions / parallel execution / comparison in the code.

Hi,

Prosody devs updated the mod_http_upload_external page.
This module should not be added to modules_enabledn but as a component:

Component "upload.example.org" "http_upload_external"
http_upload_external_base_url = "https://your.example.com/upload/service"
http_upload_external_secret = "your shared secret"

So you may want to update your readme file ;)

Yo, ThomasLeister,
Thanks for your work!
Works like a charm,
But i have some troubles
with HMAC verify on Prosody 0.12,
with all my clients (Conversation, Dino)
Filer returns error code with message from here:

So, i just rewrited a little to bypass this behavior
Commented the above, and rewrite this line

To this:

if a != nil || a == nil {

It works, but i'm a bit confused,
Because I want to do all the things right))

So, maybe @ThomasLeister could give me some advice?)

Hello, we are a team researching the dependency management mechanism of Golang. During our analysis, we came across your project and noticed that you have fixed a vulnerability (snyk references, CWE: CWE-200, fix commit id: 990373d). However, we observed that you have not tagged the fixing commit or its subsequent commits. As a result, users are unable to obtain the patch version through Go tool ‘go list’.

We kindly request your assistance in addressing this issue. Tagging the fixing commit or its subsequent commits will greatly benefit users who rely on your project and are seeking the patched version to address the vulnerability.

We greatly appreciate your attention to this matter and collaboration in resolving it. Thank you for your time and for your valuable contributions to our research.