Giter VIP home page Giter VIP logo

docker-piler's Introduction

WorkInProgress of the docker:

  • PUID and PGID (please test it ;-)
  • DONE, use the tar not the precompiled DEB, as the DEB is amd64 only.
  • 70% DONE, hope i can add amd64, arm, arm64, ppc64le, s390x, as Supported Architectures
  • 90% DONE, use ubuntu focal as docker base
  • docker ENV for password setting
  • generate secure random MYSQL passwords

IMPORTANT!

Note that piler stores all emails and attachments as separate files. You may tweak inode ratio, if necessary. IMPORTANT! Make sure you never lose/overwrite the key otherwise you won't access your archive ever again. So whenever you upgrade be sure to keep your existing key file. Also NEVER change the iv parameter in piler.conf after installation. The piler mysql database contains essential information, including metadata, permissions, tags, etc. If you lost the piler database, your archive would stop working! So you must take a good care of the piler database.

Usage

Here are some example snippets to help you get started creating a container.

docker

docker create \
  --name=piler \
  -e PUID=1000 `#optional` \
  -e PGID=1000 `#optional` \
  -e TZ=Europe/London `#optional` \
  -e PILER_HOST=archive.yourdomain.com \
  -p 443:443 `#optional` \
  -p 80:80 \
  -p 25:25 \
  -v </path/to/appdata/config>:/var/piler \
  --restart unless-stopped \
  woa7/piler:1.3.7

e.g.

PILER_VAR_DATA=/var/piler-data
docker create --name=piler -e PUID=1000 `#optional` -e PGID=1000 `#optional` -e PILER_HOST=archive.example.org -p 443:443 -p 25:25 -v ${PILER_VAR_DATA:-/default/path/to/appdata/config}:/var/piler --restart unless-stopped woa7/piler:1.3.7
docker start piler

or

  PILER_VAR_DATA=/var/piler-data
  docker run -d --name piler -p 25:25 -p 80:80 -p 443:443 -v ${PILER_VAR_DATA:-null}:/var/piler -e PILER_HOST=archive.example.org woa7/piler
  • Shell access whilst the container is running: docker exec -it piler /bin/bash

testing the beta based on Ubuntu 20.04 LTS Focal Fossa

PILER_VAR_DATA=/var/piler-data ; docker create --name=piler -e PUID=$(id -u) -e PGID=$(id -g) -e PILER_HOST=archive.example.org -p 25:25 -p 80:80 -p 443:443 -v ${PILER_VAR_DATA/config:-/dummy}:/config -v ${PILER_VAR_DATA/data:-/dummy}:/data woa7/piler:focal docker start piler ; docker logs -f piler

OR

PILER_VAR_DATA=/var/piler-data ; docker run -d --name piler -e PUID=$(id -u) -e PGID=$(id -g) -e PILER_HOST=archive.example.org -p 25:25 -p 80:80 -p 443:443 -v ${PILER_VAR_DATA/config:-/dummy}:/config -v ${PILER_VAR_DATA/data:-/dummy}:/data woa7/piler:focal

User / Group Identifiers

When using volumes (-v flags) permissions issues can arise between the host OS and the container, we avoid this issue by allowing you to specify the user PUID and group PGID.

Ensure any volume directories on the host are owned by the same user you specify and any permissions issues will vanish like magic.

In this instance PUID=1000 and PGID=1000, to find yours use id user as below:

  on the host OS:
  $ id username
    uid=1000(dockeruser) gid=1000(dockergroup) groups=1000(dockergroup)
default logins:
admin account:	admin@local
admin password: pilerrocks
auditor account:	auditor@local
auditor password: auditor

How to build

check out git clone [email protected]:woa7/docker-piler.git && cd docker-piler

Pick the latest deb package from Bitbucket download page (https://bitbucket.org/jsuto/piler/downloads/) and use it as the PACKAGE build argument, eg.

docker build --build-arg PACKAGE=piler_1.3.7-bionic-94c54a0_amd64.deb --build-arg PACKAGE_DOWNLOAD_SHA256=025bf31155d31c4764c037df29703f85e2e56d66455616a25411928380f49d7c -t woa7/piler .

How to run the image

Set the PILER_HOST env variable to match your hostname, eg.

docker run -d --name piler1 -p 25:25 -p 80:80 -e PILER_HOST=archive.example.org woa7/piler

Supported Architectures

Our image support only amd64 at the time, as architectures. as the sphinx is only amd64.

Documentation of piler it self:

http://www.mailpiler.org/wiki/current:index

piler features from: http://www.mailpiler.org/

Email archiving provides lots of benefits to your company. Piler is a feature rich open source email archiving solution, and a viable alternative to commercial email archiving products; check out the comparison with Mailarchiva.

Piler has a nice GUI written in PHP supporting several authentication methods (AD/LDAP, SSO, Google OAuth, 2 FA, IMAP, POP3). Be sure to try the online demo!

Piler supports

archiving and retention rules legal hold deduplication digital fingerprinting and verification full text search tagging emails view, export, restore emails bulk import/export messages audit logs Google Apps Office 365 and many more

The piler email archiver uses the following components:

mysql: piler stores crucial metadata of the messages sphinx: a search engine used by the gui to return the search results file system: this is where the encrypted and compressed messages, attachments are stored How do emails get to the archive? You configure your email server to pass a copy of emails to the piler daemon via smtp, since piler is an SMTP(-talking) daemon. Note that you don't need to create any system or virtual users or email addresses for the piler daemon to work, because it simply archives every email it receives.

When an email is received, then it's parsed, disassembled, compressed, encrypted, and finally stored in the file system: one file for every email and attachment. Also, the textual data is written to the sph_index table. The periodic indexer job reads the sph_index table, and updates the sphinx databases.

The GUI uses sphinx and mysql database to return the search results to the users.

Piler has a built-in access control to prevent a user to access other's messages. Auditors can see every archived email. Piler parses the header and extracts the From:, To: and Cc: addresses (in case of From: it only stores the first email address, since some spammers include tons of addresses in the From: field), and when a user searches for his emails then piler tries to match his email addresses against the email addresses in the messages. To sum it up, a regular user can see only the emails he sent or received.

This leads to a limitation: piler will hide an email from a user if he was (only) in the Bcc: field. This limitation has another side effect related to external mailing lists. You have to maintain which user belongs to which external mailing lists, otherwise users won't see these messages. Internal mailing lists are not a problem as long as piler can extract the membership information from openldap OR Active Directory.

Fortunately both Exchange and postfix (and probably some other MTAs, too) are able to put envelope recipients to the email, so the limitation mentioned above is solved.

docker-piler's People

Contributors

woa7 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

docker-piler's Issues

Typos in Dockerfile

In the Dockerfile line 25 is missing a backslash at the end:

When I run the original Dockerfile I get the following error:

Error response from daemon: Dockerfile parse error line 26: unknown instruction: PACKAGE="${PACKAGE:-PILER_1.3.6~BIONIC-78E5A44_AMD64.DEB}"

Line 25 is missing a backslash at the end when the backslash is added this error is resolved.

The backslashes on lines 57 and 61 should be at the end of the previous lines:

Error response from daemon: Dockerfile parse error line 57: unknown instruction: \

I still has issues with the curl commands executing:

/bin/sh: 1: Syntax error: Unterminated quoted string
The command '/bin/sh -c curl -fSLO "${SPHINX_BIN_TARGZ}" "${SPHINX_BIN_TARGZ_DOWNLOAD_URL}" && echo "$SPHINX_BIN_TARGZ_DOWNLOAD_SHA256 *$SPHINX_BIN_TARGZ" | sha256sum -c - || echo "sha256sum FAILD: ${SPHINX_BIN_TARGZ_DOWNLOAD_URL}" should $SPHINX_BIN_TARGZ_DOWNLOAD_SHA256 but is:" ; sha256sum $SPHINX_BIN_TARGZ' returned a non-zero code: 2

So I just manually entered the URLs as follows:

RUN curl -fSLO "https://download.mailpiler.com/generic-local/sphinx-3.1.1-bin.tar.gz"
RUN curl -fSLO "https://bitbucket.org/jsuto/piler/downloads/piler_1.3.6~bionic-78e5a44_amd64.deb"

Cloud not start with Volume Mapping

Hello,

I could not start the container with volume mapping. If I leave out "-v /var/piler-data...." it is working.

What am I doing wrong?

Thank you.

start.sh is missing an if statement to check to see if database already exists

If the container is ran with an restart flag (Ex. --restart unless-stopped) and the container is stopped and restarted (Ex. Host Reboot) then I get the following error:

Creating mysql database
ERROR 1007 (HY000) at line 1: Can't create database 'piler'; database exists
[cmd] /start.sh exited 1
[cont-finish.d] executing container finish scripts...
[cont-finish.d] done.
[s6-finish] waiting for services.
[s6-finish] sending all processes the TERM signal.
[s6-finish] sending all processes the KILL signal and exiting

Would you consider adding an if statement to the start.sh script to check if the database already exists to avoid this situation?

mariadb won't start with mapped volume /var/lib/mysql

To keep the database on updates and for backup reasons, i tried to start the piler with the following compose file:

version: '3.3'
services:
piler:
image: woa7/piler
ports:
- 80:80
- 25:25
user: 0:0
environment:
- TZ=Europe/Berlin
- PILER_HOST=mailarchiv.test.de
- PUID=1000
- PGID=1000
volumes:
- /usr/share/zoneinfo/Europe/Berlin:/etc/localtime
- /opt/piler/etc/piler:/etc/piler
- /opt/piler/var/piler:/var/piler
- /opt/piler/var/lib/mysql:/var/lib/mysql

Here the log:

...
User uid: 1000
User gid: 1000

[cont-init.d] 10-adduser: exited 0.
[cont-init.d] 99-custom-scripts: executing...
[custom-init] no custom files found exiting...
[cont-init.d] 99-custom-scripts: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.

  • Starting enhanced syslogd rsyslogd
    rsyslogd: imklog: cannot open kernel log (/proc/kmsg): Operation not permitted.
    rsyslogd: activation of module imklog failed [v8.32.0 try http://www.rsyslog.com/e/2145 ]
    ...done.
  • Starting MariaDB database server mysqld
    ...fail!
    [cmd] /bin/bash exited 1
    [cont-finish.d] executing container finish scripts...
    [cont-finish.d] done.
    [s6-finish] waiting for services.
    [s6-finish] sending all processes the TERM signal.
    [s6-finish] sending all processes the KILL signal and exiting.

Sphinx Reindex

I has the following issue:

Some Linux distributions (notably Debian and Ubuntu) have a daily cron job to reindex everything. Unfortunately this ruins the sphinx index files piler relies on. However the older emails are not lost you still have them, they are just disappeared from the sphinx index. To bring them back, perform the following steps.

Edit /etc/default/sphinxsearch, and set START=“no”.

I recommend you to use the piler shipped init.d/rc.searchd script to start searchd. You may call it from /etc/rc.local. (Note that it starts it as user piler, so make sure /var/piler/sphinx has proper ownership.)

Reindex old emails. After that older emails should appear after the next indexing is done.

cd /tmp

reindex -a

My question is since this is in a Docker container, do you happen to know where the '/etc/default/sphinxsearch' location is and what all needs to be modified to resolve this issue. I search from the sphinxsearch location and couldn't find it. I had to reindex to get old emails to show back up in search, but I have not made the proper modifications to keep this from happening again. Any suggestions? Running Ubuntu 18.04

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.