Giter VIP home page Giter VIP logo

dmda's Introduction

DMDA

DMDA (Database Mail Delivery Agent) is a Java application used to store email in a database. It is designed to sit behind a Mail Transfer Agent such as postfix and will store emails in the configured databases based on the recipient's email address's domain.

This application was developed as an alternative for Apache James. James is a large POP3/IMAP server that does a lot of things including storing email in the database however it is very heavy weight.

The main goals of this tool were:

  • Reliability: The server uses HikariCP to manage the database connection pool and will reconnect should the connection drop
  • Data integrity: The tool will either reject an email or store it. There will be no dropped emails.
  • Light weight: The tool is small, fast and easy to configure.

Use

See the bin folder for creating a linux daemon for DMDA or see the releases page for an RPM distribution. Instructions on daemon installation and running can be found inside the bin folders README.

For local running run mvn install first and have a config.xml (see the config.xml.sample for information) in the folder you run the following command:

java -jar ./target/dmda-x.x-SNAPSHOT-jar-with-dependencies.jar

Configuration

All application configuration is defined in the config.xml file, with the exception of anti-virus connection information which can also be configured via the following environment variables.

Name Description
DMDA_AV_MODE The anti-virus mode. One of clamd or none.
DMDA_AV_SERVER The anti-virus server hostname.
DMDA_AV_PORT The anti-virus server port.
DMDA_AV_TIMEOUT_MS The anti-virus scan timeout in milliseconds.

If anti-virus configuration exists in both environment variables and the XML config file, the configuration options set in the environment variables will take precedence.

See the sample config file for all other options.

Health Checks

DMDA provides the following basic application health checks, which can be queried over HTTP. These are disabled by default, but when enabled the following endpoints become available:

Endpoint Description Returns
/smtp-status Checks that the SMTP server is running. An HTTP 200 OK if the SMTP server is running, otherwise a 503 Service Unavailable.
/db-status?security-token=foobar Checks that a valid connection can be made to each database in the database_list. Note that a security token needs to be provided to access this endpoint, and it must match the one defined in the config. A list of databases (identified by DB name and JDBC connection URL) and a boolean value indicating if the application can establish a valid connection to the database.
/av-status Checks that a valid connection can be made to the virus scanner. An HTTP 200 OK if a connection to the AV service can be established, or a 503 Service Unavailable otherwise.

Note that these endpoints should only be exposed locally to monitoring tools such as Monitis, Zabbix, Nagios etc. or used within Docker containers as HEALTHCHECK directives. They are not designed to be accessible from the internet.

Requirements

  • Java : >=1.8

dmda's People

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dmda's Issues

Test ClamAV is up at start

Currently DMDA doesn't test if ClamAV is accessible until an email arrives.
Ideally when DMDA starts it would test ClamAV is up if configured

Specify -Xmx flag

Currently max heap defaults to whatever the environment default is, which appears to be an undefinable thing as per the spec (because the claim of 1/4 system RAM or 1GB whichever is smaller is clearly untrue).

Need to do some testing to see what an appropriate value is, but gut feeling from the amount of throughput and complexity is pretty low. I'd say safe in the region of 128m to 256m?

Rotate logs using date instead of filesize

It would be nicer if logs were rotated based on date. From my understanding of Log4j, this should be a simple case of constructing a DailyRollingFileAppender instead of a basic FileAppender.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

This repository currently has no open or pending branches.

Detected dependencies

None detected

Use Maven properly

The maven pom.xml should be set up so that the project can be built using maven, rather than using it just as a way to get some jar files.

Strip attachements from emails and store them in a serperate table

DMDA should strip attachments from the email. These attachments will then be stored in the database using a new optional query for attachments. This query should have the following bind parameters:

  • The binary blob
  • The attachment name
  • The email reference
  • The mime type

Logging timestamp doesn't use a 24 hour time format

Currently 3PM appears as 03:00, probably because the wrong format mask is being used. Also, I would naively expect Log4j to handle timestamping of log rows without customisation, is there a reason we're specifying our own mask?

ClamAV timeout too short

Currently DMDA doesn't specify a timeout when calling the ClamAV library, meaning it had a default timeout of 500ms when waiting for an OK/Virus response.

In the case of large files that are streamed to ClamAV it can take multiple seconds to fully scan before a response is sent leading to errors:

java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.net.SocketInputStream.read(SocketInputStream.java:127)
at fi.solita.clamav.ClamAVClient.readAll(ClamAVClient.java:127)
at fi.solita.clamav.ClamAVClient.scan(ClamAVClient.java:95)
at fi.solita.clamav.ClamAVClient.scan(ClamAVClient.java:106)
at uk.co.fivium.dmda.AntiVirus.ClamAVScanner.scan(ClamAVScanner.java:18)
at uk.co.fivium.dmda.AntiVirus.ClamAVScanner.checkContent(ClamAVScanner.java:29)
at uk.co.fivium.dmda.EmailMessages.EmailMessage.addData(EmailMessage.java:76)
at uk.co.fivium.dmda.EmailMessages.DatabaseMessageHandler.data(DatabaseMessageHandler.java:74)
at org.subethamail.smtp.command.DataCommand.execute(DataCommand.java:64)
at org.subethamail.smtp.server.RequireTLSCommandWrapper.execute(RequireTLSCommandWrapper.java:30)
at org.subethamail.smtp.server.CommandHandler.handleCommand(CommandHandler.java:99)
at org.subethamail.smtp.server.Session.runCommandLoop(Session.java:244)
at org.subethamail.smtp.server.Session.run(Session.java:145)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Domain matching should be case insensitive

Inbound emails currently match based on domain in a case sensitive manner when determining database to insert into, e.g. Ecase.co.uk would not correctly match ecase.co.uk

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.