Giter VIP home page Giter VIP logo

mailad's Introduction

MailAD v1.1.4

Chat on Telegram Twitter Follow GitHub Issues GitHub Issues Closed GitHub repo size GitHub last commit GitHub commit rate Financial contributors

All Contributors

This page is also available in the following languages: [ EspaΓ±ol πŸ‡ͺπŸ‡Έ πŸ‡¨πŸ‡Ί] [ Deutsch πŸ‡©πŸ‡ͺ]

This is a handy tool to provision a mail server on linux linked to an Active Directory (AD from now on) server (Samba or Windows) with some constraints in mind, as this is a typical mail config to be used in Cuba as regulated by law and security enforcement requirements, but can be used on any domain. You can see a simple provision in this asciinema movie.

We have a docker development on going on another repository, take a peek and test it or contribute: MailAD-Docker

Rationale

This repository is intended to be cloned on your fresh OS install under /root (you can use a LXC instance, VM, etc) and setup on a main conf file as per the file comments, then run the steps on a makefile and follow the steps to configure your server.

After a few steps you will have a mail server up and running in about 15 minutes tops. (this time is based on a 2Mbps internet connection to a repository, if you have a local repository it will be less)

This tool is tested and supported on:

  • Ubuntu Bionic 18.04 LTS (legacy NOT recommended)
  • Ubuntu Focal 20.04 LTS (legacy NOT recommended)
  • Ubuntu Jammy 22.04 LTS (Recommended).
  • Debian Buster 10 (see note below please).
  • Debian Bullseye 11 (see note below please).

Note: If you are using a Debian Container on LXC (Proxmox for example) you need to tweak the dovecot install or it will not work, see this fix for more info

It's recommended that the instance of MailAD sits within your DMZ segment with a firewall between it and your users and a mail gateway like Proxmox Mail Gateway between it and the external network.

Features

This will provision a mail server for an enterprise serving corporate users. You can see the major features in the Features.md file, among others you will find:

  1. Low resource footprint.
  2. Advanced (and optional) mail filtering features that includes attachments, SPF, AntiVirus & Spam.
  3. Encrypted LDAP communication as an option.
  4. In place protection to major and known SSL & mail service attacks.
  5. Automatic alias using AD groups.
  6. Manual alias, manual ban, manual headers & body checks.
  7. On demand backup and restore of raw configurations.
  8. Really painless upgrades.
  9. Daily mail traffic summary to your inbox.
  10. Optional user privilege access via AD groups (local/national/international).
  11. Optional disclaimer/notice/warning on every outgoing mail.
  12. Optional aggressive SPAM fight measures.
  13. Weekly background check for new versions with a detailed email if you need to upgrade.
  14. Optional mailbox split by office/city/country

TODO

There is a TODO list, which serves as a kind of "roadmap" for new features, but as I (the only dev so far) have a life, a family and a daily job, you know...

All dev is made on weekend or late at night (seriously take a peek on the commit dates!) if you need a feature or fix ASAP, please take into account making a donation or found me and I will be happy to help you ASAP, my contact info is on the bottom of this page.

Constraints and requirements

Do you remember the comment at top of the page about "...with some constraints in mind..."? Yeah, here they are:

  1. Your user base and config came from AD as mentioned, we prefer Samba AD but it works on Windows too; see the AD requirements for this tool
  2. The username part of the email must not pass the 20 chars mark, so [email protected] will be cut to [email protected] this is not our rule, but a handycap of the LDAP directory as specified by Windows Schema.
  3. The mail storage will be a folder in /home/vmail, all mail will belong to a user named vmail with uid:5000 & gid:5000. Tip: that folder can be a NFS mount or any other type of network storage (configurable)
  4. You use a Windows PC to control and manage the domain (must be a domain member and have the RSAT installed and activated), we recommend a Windows 10 LTSC/Professional
  5. The communication with the server is done in this way: (See this question on the FAQ file to know more)
    • Port 25 (SMTP) is used to receive incoming traffic from the outside world or from a mail gateway.
    • Port 587 (SUBMISSION) is used to receive emails from the users to be delivered locally or relayed to other servers.
    • Port 465 (SMTPS) is used like port 587 but is only enabled as a legacy option, its use is discouraged in favor of port 587.
    • Port 993 (IMAPS) the preffered method to retrieve the email form the server.
    • Port 995 (POP3S) used like the 993, but discouraged in favor of IMAPS (unless you are in a very slow link)

How to install or try it?

We have a INSTALL.md file just for that, and also a FAQ file with common problems.

This is free software!

Have a comment, question, contributions or fix?

Use the Issues tab in the repository URL or drop me a message via Twitter or Telegram

Contributors ✨

Thanks goes to these wonderful people (emoji key):


danny920825

⚠️ πŸ€”

HugoFlorentino

πŸ€” πŸ’‘

Armando Felipe

πŸ€”

Koratsuki

πŸ€” πŸ’» 🌍

Gabriel A. LΓ³pez LΓ³pez

🌍

oneohthree

πŸ€”

Eddy Ernesto del Valle Pino

πŸ“–

dienteperro

πŸ“– πŸ’΅ πŸ€”

Joe1962

πŸ€” ⚠️

Sandy Napoles Umpierre

πŸ€” ⚠️

Please read the CONTRIBUTING.md file if you want to contribute to MailAD to know the details of how to do it. All kinds of contributions are welcomed, ideas, fixes, bugs, improvements and even a phone top-up to keep me online.

This project follows the all-contributors specification. Contributions of any kind welcome!

mailad's People

Contributors

allcontributors[bot] avatar dienteperro avatar edelvalle avatar glpzzz avatar oneohthree avatar stdevpavelmc avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

mailad's Issues

Need to separate the dovecot template tree based on versions

Describe the bug
We support Ubuntu Bionic & Focal, and also Debian buster; but Buster and Focal has dovecot v2.3 and Bionic has v2.2.

There are some config options (mainly on the 10-ssl.conf file) that changed and gives you some recurring warnings on Focal/Buster, some users has reported that

To Reproduce

  • Make a deploy on bionic: no warnings all is good
  • Make a deploy on focal/buster: you get a few warnings from dovecot about deprecated options

Expected behavior
No warnings

Possible solution
Create a different template base for dovecot, one for version 2.2 and other for version 2.3 and on provision time pich the correct one.

This will fix the warnings.

Reject messages without Subject (blank or tricky ones)

Message without Subject should be rejected
It does not matter if it is when sending or receiving the message, emails without subjects should not be allowed since they are mostly used by malicious applications to send SPAM. And it is good practice to remove them. The error message sent to the sender must be explicit when reporting what happened

[FEAT] Improve the Readme

Is your feature request related to a problem? Please describe.

Improve the readme with:

  • Badges
  • Better practives
  • Organization
  • Etc.

[BUG] ClamAV fails if no working DNS, related to #60

Describe the bug
ClamAV uses the DNS to get the update database fingerprint to detect updates, if the DNS is not functional or in a isolated environment (internal domain) you will never get the updates working

Possible solution
In the conf-check stage test for the dns TXT record of current.cvd.clamav.net and if not fails warning the user, line we mention in the #60

[BUG] problem with the backups and custom restore

Describe the bug
Some times the provision/upgrade/force-upgrade fails and the reference for the backup good backup is changed; then on the next provision/upgrade/force-upgrade command you save an give for good a partial/not-working backup

Expected behavior
Update the backup link ONLY if the action that triggers the backup end with no trouble.

Need a way to create custom access rules for some users

Is your feature request related to a problem? Please describe.
The access control for users (international/national/local) is good, but some times you need custom rules, for example an user with just national access but one or more specific addresses outside .cu

Or the case of a local access account that need to exchange mails with just ONE email in .cu (outside your domain)

Describe the solution you'd like
A way to declare restrictions inside the AD Admin tool

[FEAT] Add DNS reverse black list support

Is your feature request related to a problem? Please describe.
DNSRBL support is a must in a modern email server.

Describe the solution you'd like
Enable this feature like an option, and when enabled a simple file to configure the DNSRBL sources and it's scores. Adding by default only a few well known servers, with low false positives count.

[BUG] SpamAssassin dies if not update possible, then amavisd and the mail server collapse

Describe the bug
If your setup does not have a working DNS (like a local domain with no upstream DNS) the spamassassin service will die and amavisd-new will die also, breaking the mail flow.

Possible solution

  • Check in conf-check the test stage if we have selected SPAMD the DNS can resolve the TXT register for spamassassin 2.4.3.updates.spamassassin.org it's a CNAME of "3.3.3...." and it return the TXT fingerprint of the database
  • If not working notice the user with an error about it, to fix the error or disable the SPAMD in the config

[FEAT] Switch from dig to host for DNS testing

Is your feature request related to a problem? Please describe.
As pointed out by @HugoFlorentino by using host instead of bind we are saving the install of a few packages and in some cases host is already installed.

~# dpkg -S $(which host)
bind9-host: /usr/bin/host
~# dpkg -S $(which dig)
dnsutils: /usr/bin/dig

By this we save time and space...

SpamAssasin and ClamAV support

Is your feature request related to a problem? Please describe.
Some setups can not afford a mail gateway, in this case (or just because you want) we need the option to enable Spam & AV filtering

Describe the solution you'd like
A config option to enable or disable [default] the Spam & AV filtering

AV filtering will use ClamAV & FreshClam, the updates from FreshClam must be a two choice option:

  • From an official repository [default]
  • From an lternate mirror (alternate mirrors are needed in Cuba as per the US Embargo laws)

SpamAssassin rules must be eligible being [off by default]

In any case we need to use an upstream HTTP proxy if needed (server, port, user password)

Any alternatives or comments over this new feature?

Make the everyone alias accesible from outside the domain but optional

Is your feature request related to a problem? Please describe.
I hit an user that needs the everyone alias to be accessible from outside, it's a requirement of the enterprise

So the most common sense is to make this option optional, being the default not accessible from the outside

AV filtering failing on focal

Describe the bug
AV filtering is failing on Ubuntu Focal (unconfirmed on buster & bionic) due to permissions problems

Expected behavior
Not fail

Solution
Make clamav member of amavis group

adduser clamav amavis

[FEAT] Create a FAQ

MailAD is getting impulse, and there are some key points about the config that are giving troubles to the users

A FAQ to compile them will be nice (both, english and spanish)

Reporte de los Correos del dia

Seria bueno contar con un report, que se lo mande a una direccion o a una lista de informatico donde este un reporte con algunos datos como lo da zimbra:

Grand Totals

messages

3336 received
15865 delivered
0 forwarded
388 deferred (6839 deferrals)
16 bounced
3330 rejected (17%)
0 reject warnings
0 held
0 discarded (0%)

264409k bytes received
1378m bytes delivered
804 senders
238 sending hosts/domains
1008 recipients
86 recipient hosts/domains

Per-Hour Traffic Summary

time received delivered deferred bounced rejected

0000-0100 76 496 266 0 61
0100-0200 232 1071 259 0 87
0200-0300 51 238 285 0 175
0300-0400 29 121 266 0 112
0400-0500 209 927 270 0 136
0500-0600 53 214 268 0 138
0600-0700 108 440 292 0 168
0700-0800 247 1100 309 1 137
0800-0900 108 586 268 1 126
0900-1000 266 1249 303 0 193
1000-1100 114 518 297 0 166
1100-1200 125 945 330 3 140
1200-1300 106 507 292 1 150
1300-1400 278 1240 315 0 156
1400-1500 150 676 305 0 177
1500-1600 255 1143 294 2 168
1600-1700 198 844 304 3 151
1700-1800 105 804 289 2 160
1800-1900 101 427 288 1 161
1900-2000 83 354 296 0 111
2000-2100 70 308 307 0 110
2100-2200 247 1106 315 0 152
2200-2300 104 461 284 2 138
2300-2400 21 90 137 0 57

Host/Domain Summary: Message Delivery (top 50)

sent cnt bytes defers avg dly max dly host/domain

Host/Domain Summary: Messages Received (top 50)

msg cnt bytes host/domain

top 50 Senders by message count

top 50 Recipients by message count

top 50 Senders by message size

top 50 Recipients by message size

Dovecot auth daemon is mapping local users with AD users on username

On the mailad server if you have a local linux user that has a name equal to a AD username (sAMAccountName LDAP attribute) then dovecot will try to deliver & use the maildir of the local user instead the AD/LDAP/Virtual one.

Proposed fixs:

  • Quick & dirty: avoid creation of a local user with the same name that one in the AD
  • Long term: configure dovecot to not use local database for autentication

Spanish translation

This will be an easy one.

  • Create a AD_Requirements_ES.md file as a Spanish translation of the AD_Requirements.md file.
  • Create a README_ES.md file as a Spanish translation of the README.md file.
  • Update the links in the README_ES.md file pointing to the AD_Requirements.md to the file AD_Requirements_ES.md
  • Add a note on top of the README.md file (in spanish) about the existence of the Spanish version
  • Copy the file mailad.conf to mailad_ES.conf and translate mailad.conf to English

That's for the first part.

[FEAT] Simplify the AD field handling

Actually, to enable a user with his email we are setting 4 fields if the AD:

  • Mail (Mail)
  • Main Storage (Office)
  • User Quota (Telephone number)
  • Maildir (Webpage)

The initial reason behind this was the integration with Postfix and Courier/Dovecot, but as I have gained more insight of the integration and softwares, and also by users request to simplify that, and a possible standard ldap fields usage.

I will review that to try to simplify it.

  • Use the UserPrincipalName as the mail atrribute, this approach has pros and cons...
  • Use the sAMAccountName as maildir element: the challenge is to trick postfix and dovecot to add the final "/" to sign of a maildir.
  • Remove the setting for the mail storage completely from the AD. Just hard coded in the settings while provisioning. (this may be the easiest to implement)
  • Try to use the ldap quota attribute for the quota.
  • Try to use a default hard coded quota (provision stage) and just override it if the specific user has a particular quota set

Cheers.

[BUG] Clamav-freshclam malfunction not getting the update

Some bugs was found in the latest commits about clamav & freshclam

  • DatabaseMirror parameter can be expressed multiple times, if https you can omit the prefix, but no for http
  • We must increase the timeouts as in slow networks this can be a problem and the update never finish

The header_checks & body_checks are not preserved when upgrading

Describe the bug
When you do an upgrade the header_checks & body_checks files are not preserved, then you lose all data on those files.

Not quite, the files are present on he backup file, but the users does not know that.

Expected behavior
Both files are preserved in the event of a upgrade (the same way you do with the virtual_aliases file)

Notifications go to a group instead to the mail admin

Is your feature request related to a problem? Please describe.
In an enterprise where there are more than one sysadmin the notifications must go to all of them instead to only the configured one.

Aka: redirecting abuse@, posmaster@ and the notification for mail summary via a group alias

Describe the solution you'd like
Be able to tell mailad that I want notifications via a group alias address instead of the mail admin

[BUG] DNSBL it not working properly...

Describe the bug
DNSBL is not working properly, in fact it's not working at all!

My mistake, in the dev env I manually enabled the transports in the master.cf file to enable postscreen but not included the stept to enable/disable them on production.

This cause that the DNSBL (that depends on postscreen) will never happens as postscreen is not enabled. Huge mistake from me, I apologize for the inconvenience.

This is a top priority bug.

Add travis testing

Use travis to validate the test and install part, that will require a few task:

  • Create scripts to setup the hostname info as per the config file
  • Emulation of a real AD service to interact to (docker?)
    • Setup local env for travist to fake that AD service as mailad.cu
  • Others not discovered yet (please Add/suggest as needed)

Issues with PING Option to AD

Issues with PING Option to AD. On my LAN, the segmentation is classified by Security levels. And based on criteria, I have the AD in a VLAN alone with no one else. And the mail server in another VLAN. In the firewall there is a rule that allows mail to reach the AD through LDAP port 389, but the ICMP protocol is not allowed. Therefore, the AD check using PING does not work in cases like these. Not to mention that the ICMP in the servers is totally discouraged

Daily mail's traffic sumary in your inbox

Is your feature request related to a problem? Please describe.
Almost all mail server solutions has a way to report mail usage statistics, and MailAD lacks of that feature

Describe the solution you'd like
A solution like the use of pflogsumm would be nice...

Use LE certificates on provision

Is your feature request related to a problem? Please describe.
Additional steps after install to use a LE certificates is a pain...

Describe the solution you'd like
Use LE certificates from start, let's say we create a LE folder inside mailad folder (exclude it from git!) and place there the fullchain & privkey certs to be used during provision.

Then when you get to the SSL step, use the LE certs (a link to there must be created and certs folder & links must be secured with correct perms) to make it easy for the user.

Once renew of the certs just overwrite the certs in the LE files and restart the server.

Any other approach for this?

A way to distinc from users with full access and other with only country reach (*.cu)

Is your feature request related to a problem? Please describe.
Some enterprises state as a policy that some group of users must have local or national access only

Describe the solution you'd like

A way to declare a group in the AD and auto-magicaly all members of that group has only local access (ie: only inside the local domain)

The same but for a country wide restriction (*.cu) policy

We can call the groups the name we want [via config on deploy] or use some default group names... what do you think it's must be the default?

Integracion a mailpiler

Poder ponerle una direccion de que le mande una copia de todos los correos o a un buzon local o al buzon del mailpiler de los correos tanto entrada como saliente

[BUG] ClamAV activation script hits a chicken vs. egg dilema

Describe the bug
A script is set hourly to check for clamav-daemon is running fine at provision stage, it depend on clamav-freshclam getting the update and notify clamav-clamd via a socket that the update is done and must reload... but the socket is not there because clamav-daemon ran and died because no updates exist... a chicken vs. egg dilema

Possible solution

  • Restart or reload the clamav-daemon daemon before the hourly script test for it.
  • Proposed schema is to restart, wait and finally test for it.

Support Ubuntu LTS and Actual Debian stable (Buster)

Is your feature request related to a problem? Please describe.
Actually MailAD is based on a past LTS version of Ubuntu, support of the actual LTS (Focal Fosa 20.04) is imperative, also support for the Debian stable branch: Buster 10.x

Describe the solution you'd like
It work's on the 3 flavours, Ubuntu 18.08 & 20.04 (Focus on this) and also on Debian Buster

Translations

Is your feature request related to a problem? Please describe.
The main target audience of this software are in Cuba, so why not a Spanish translation?

In a first approach just the documentation translation...

Describe the solution you'd like
Built in i18n support to get messages in your own language, being English the default and Spanish the most wanted

Describe alternatives you've considered
Gettext i18n integration, like this sample: https://github.com/meonkeys/bash-i18n-example/

But open to any other alternative that allows full internationalization is welcomed.

[BUG] Fail when provisioning a setup with no spam/av

Describe the bug
After the recent corrections, the services function on some scripts (restart services) is given for granted that all services are in place, when no av or spamd is configured it fails

So we must prepare that function for that: avoid handling the non functional services

Suggest and propose erasure of maildir folders left behind

Problem description

When you run the server for a while (more than a year) some users may exit the organization, so you remove them from the AD as requested by the human resources dept. Or maybe you disable them, and at some point you remove them.

But in the guts of the server the maildir folder remains!

And for an experience in a client this wasted space may get to dangerous levels, [~20GB]

So how to remove them?

Possible solution*

Create a monthly script that checks stalled folders on the mail storage against real users on the AD

This can report:

  • Unsed mail folders and his total size
  • Mail folders aging almost a year (> 9 monts) with a warning about removal after a year
  • Notice of maildirs removal after 1 year and a month and the gained space.

The period of removal alert (>9 months) and the removal (>1 year) may be configurable, by default the Cuba's standard is removal after one year

EVERYONE alias is externally accesible when activated

Describe the bug
If you activate the EVERYONE alias, this is accessible from outside the domain

To Reproduce
Steps to reproduce the behavior:

  1. Setup a env with the EVERYONE account set
  2. Provision the server
  3. Send an email to the EVERYONE declared address from within the domain [it works]
  4. Now send an email to the EVERYONE declared address from OTHER domain [it works]

This can be used for spamming, phishing, etc.

Expected behavior
The EVERYONE alias when active must be not accessible from outside the domain

Possible solution
http://www.postfix.org/RESTRICTION_CLASS_README.html se the section named Protecting internal email distribution lists

Posterior Actions
Add a specific test in the test suite for this case

[BUG] ClamAV update mirror must be configurable and exclude de protocol in the url

Describe the bug
In some cases you will have restrictive parent proxy that forbids a word or any IP based address, or just the mirror we use dies, you may need a fast way to change the non-default mirror from a few working options...

Also the mirror url must not have the protocol part, aka updates.clamsv.com (ok) vs http://updates.clamsv.com (worng)

Possible solution

  • Modify the conf file to include the alternate mirrors
  • Change the default one from the IP address to the name based address one

[FEAT] Stats or feedback

Is your feature request related to a problem? Please describe.
There is no way of track the usage of MailAD, some stats will be OK.

Also that could help in getting users informed about latest changes and bug fixes

Describe the solution you'd like
In the config there will be an option (checked by default) about sending an email to the creator as ping-back

That email will have:

  • Date
  • Mailserver name
  • Mail Admin email
  • MailAD "version" installed

Optionally that email may be sent monthly and you will ALWAYS receive a copy of it.

This feature is a opt-out one, enabled by default, the user must disable it.

A pool held on the telegram channel result in no problem from he users if that is made a optional feature.

Captura de pantalla de 2020-09-22 21-44-53

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.