moggers87 / salmon Goto Github PK

A mail server doesn't need Jinja2.

There's a bunch of other crap that we shouldn't be doing (spam filtering is better done else where).

If someone really wants this stuff in salmon, I'd more than happy to merge a PR for a "salmon-extras" package.

It may be useful to make the FSM somewhat pluggable, e.g. state being stored on the RCPT TO address rather than the MAIL FROM address.

Upstream looks dead, probably hasn't got Py3k support.

I'm trying to set up a very simple mail server: listen to emails for a particular domain, determine whom it should be sent to based on the address and send the email using ISP mail server as a relay. A very simple mailing list server.

This was very easy with Lamson. With Salmon, there was a bug(?) in SMTPReceiver that only one recipient was handled. With LMTPReceiver (which is now the default), this is fixed but now there is suddenly the nightmare of setting up a mail server gateway in front of Salmon. The main reason to use Lamson/Salmon - at least for me - was to avoid setting up and maintaining these huge massive complex mail servers and just have a very simple process running and doing the simple thing. Or perhaps I'm missing something and it should be as easy with Salmon too somehow? I'm a bit confused on how Salmon is supposed to be used.. The only instructions I found were: "Salmon is best deployed behind another mailserver such as Postfix or Sendmail - much in the same way as you host a WSGI application behind Apache or Nginx."

For these reasons, I would really appreciate some help on how to set up Salmon so that it works as a mail server. If that really requires some mail server gateway in front, then it'd be great to have some instructions or at least pointers on how to achieve this in a very lightweight way. I don't need mailboxes, I don't need mail sending server (I use my ISP mail server as a relay). If you have any ideas how to proceed, I'd be really happy to hear. If I get it working, I could perhaps write some instructions in case that's useful for other users. Anyway, thanks for this great package and your efforts!

Is there some forum where it'd be possible to discuss and ask for help from others? Mailing list perhaps or IrC channel? If not, would you be interested in simply enabling gitter chat for this repo? I've found that very convenient. Or is it ok to ask for help via these github issues?

It would be really nice to have LMTP support

This line:

https://github.com/moggers87/salmon/blob/master/salmon/server.py#L128

shouldn't it reraise the exception?

Otherwise the email is never delivered and no indication of this fact is given to the sender.

Python 2.7 will be end of life'd sometime in 2020, so that will leave us free to move away from smtpd:

http://aiosmtpd.readthedocs.io/

aiosmtpd promises a nicer API and a few ESMTP extras. It also comes with a LMTP class, which means we'll have feature parity between SMTPServer and LMTPServer.

QueueReceiver is quite bad at handling a large influx of messages in a reasonable amount of time.

Implementation-wise QueueReceiver is the only receiver class that doesn't use threading. Though it's quite a simple class, I think it's also what makes it slow: we're waiting for each message to be processed before fetching another.

Investigate how threading could improve performance and how badly it would complicate code.

I didn't test these, will probably reveal other bugs

MailBase.keys should return the normalised header names. The raw header names will still be available from MailBase.mime_part.keys.

When we refactored how MailBase parsed data from the initial email object, we missed the keys method - previously there was no way for the header names not be normalised already.

Something that was brought up in #50 - sometimes developers will have other services running on their local machines that will conflict with the port we use in Salmon's testsuite.

Rather than keep on changing this port number every time we come across someone who happens to be using that port for something or other, we should either:

• Allow the user to change the port number used via an environment variable
• Or automagically choose any unused port at runtime

The first one is easiest to implement, but the second option sounds more fun.

_parse_charset_header isn't very well commented nor very readable (though this has been partially fixed in a branch that is yet to be merged)

If we could use decode_header from Python's email package, that would make our code a lot readable.

Also, check performance of the two implementations.

E.g. run and log

Librelist is actually a currently running service. It's technically not running salmon.

PyDNS has had no new releases for over 2 years. We should investigate more active alternatives/forks.

Py3DNS has made a release in the last year, probably a good starting point.

We already implement most methods required for a dict-like interface, but we're missing items.

We also return duplicate header names via keys, but currently have no way to get at all values. It might be a good idea to add get_all to MailBase as a friendly shortcut (like the Python email API has)

It's been out of support for over two years, time to move on.

I created a simple Salmon project. How can I run the unit tests? Running python setup.py test requires directories run/undeliverable and logs to be present and it writes something there. I would have thought that these were for actual production use, not for unit tests. Why do the unit tests mutate the state of these directories/files?

It should.

Also make sure that the sendmail command can accept multiple recipients - I'm not 100% that's supported but it's implied by code and comments in commands.py

from pep8 maximum line length section:

it is okay to increase the nominal line length from 80 to 100 characters (effectively increasing the maximum length to 99 characters)

since 01 Aug 2013

I was trying to learn to use queue but I got weird errors, for instance:

$ salmon queue --count undeliverable
Traceback (most recent call last):
  File "/.../bin/salmon", line 9, in <module>
    load_entry_point('salmon-mail==2.99.99', 'console_scripts', 'salmon')()
  File "/.../lib/python2.7/site-packages/salmon/commands.py", line 75, in main
    cmd(**vars(args))
TypeError: command() got an unexpected keyword argument 'queue'

Probably what I'm doing, isn't correct, but I suppose that particular error suggests a bug.

We really need to get the ball rolling on #33 so let's set Sphinx up - once it's done, PRs for docs can start.

Should also help with #20

This was missed in #40

So after some thinking and playing around with the current Salmon in master, I've decided to see if I can get the same result (being able to access a plain Python email message without any double processing) without too much trouble.

Currently looking at attaching a reference to the corresponding Mime part on MailBase and making header/body decoding lazy

Currently using Python's SMTPd as the basis for our server implementatoins. This has no connection limiting and will just spawn n threads for n connections.

A naïve connection limit could be implemented as a mixin that could be used with both Python's SMTPd and LMTPd.

I probably won't get around to doing this, so pull requests are most welcome ☺

Not sure if this is a proper place to ask for questions/help (see #45).

I'm now wondering how I can just ignore emails which are sent to some other host than what I've specified.

My undeliverable queue fills up with some spam that my server receives. That spam is designated to some weird email addresses with weird hosts. I'd like just ignore the mail if the "To" host isn't one of my accepted host names (I have a few of them). Is there a place to list these valid hostnames or are they automatically parsed from the used handlers? And how then ignore mail that doesn't match these host names?

Anyway, thanks a ton for working on this Lamson fork, this is great.

Connecting via telnet, everything looks fine:

220 localhost.localdomain Salmon Mail router SMTPD, version 2.99.99

But when using something like Python's smtplib results in an error because Salmon is actually returning this:

\xff\xfe\x00\x002\x00\x00\x002\x00\x00\x000\x00\x00\x00 \x00\x00\x00l\x00\x00\x00o\x00\x00\x00c\x00\x00\x00a\x00\x00\x00l\x00\x00\x00h\x00\x00\x00o\x00\x00\x00s\x00\x00\x00t\x00\x00\x00.\x00\x00\x00l\x00\x00\x00o\x00\x00\x00c\x00\x00\x00a\x00\x00\x00l\x00\x00\x00d\x00\x00\x00o\x00\x00\x00m\x00\x00\x00a\x00\x00\x00i\x00\x00\x00n\x00\x00\x00 \x00\x00\x00S\x00\x00\x00a\x00\x00\x00l\x00\x00\x00m\x00\x00\x00o\x00\x00\x00n\x00\x00\x00 \x00\x00\x00M\x00\x00\x00a\x00\x00\x00i\x00\x00\x00l\x00\x00\x00 \x00\x00\x00r\x00\x00\x00o\x00\x00\x00u\x00\x00\x00t\x00\x00\x00e\x00\x00\x00r\x00\x00\x00 \x00\x00\x00S\x00\x00\x00M\x00\x00\x00T\x00\x00\x00P\x00\x00\x00D\x00\x00\x00,\x00\x00\x00 \x00\x00\x00v\x00\x00\x00e\x00\x00\x00r\x00\x00\x00s\x00\x00\x00i\x00\x00\x00o\x00\x00\x00n\x00\x00\x00 \x00\x00\x002\x00\x00\x00.\x00\x00\x009\x00\x00\x009\x00\x00\x00.\x00\x00\x009\x00\x00\x009\x00\x00\x00

This is utf-32 encoded text, not ASCII as most SMTP (and LMTP) clients will be expecting.

This is because we're monkey patching smtpd and lmtpd's __version__ with our own text, but because I've blindly added from __future__ import unicode_literals to every module in Salmon __version__ is now being patched as unicode rather than bytes. Something further down the chain is then encoding this as UTF-32 for some reason (rather than ASCII under Python 2, or UTF-8 under Python 3).

I'm in the process of converting the old Lamson docs (from the old gpl3 version that salmon was forked from ) into rst in the form of a sphinx project so it can be deployed as a rtd project or through github sites.

I still need to do some tweaking and cleanup to match changes in salmon since the fork but before I put too much more work into tweaking stuff, I wanted to see if there's an interest in such a PR

It seems that if the list of recipients contains several email addresses that Salmon needs to handle, it only handles one and delivers the mail there. At least I've been trying to send one email to several recipients and only one of them receives it each time.

I might be doing something totally wrong here, but at least to my understanding this handler should just change the subject and send the email to me:

import logging
from salmon.routing import route, stateless
from config.settings import relay

@route("(address)@(host)", address=".+", host=".+")
@stateless
def START(message, address=None, host=None):

    logging.debug("Current subject: {0}".format(message['Subject']))
    message['Subject'] = 'New subject'
    logging.debug("New subject: {0}".format(message['Subject']))

    relay.deliver(message, To=['[email protected]'])

However, logs/salmon.log shows that the subject doesn't change:

2016-05-23 12:33:50,559 - root - DEBUG - Current subject: original subject
2016-05-23 12:33:50,559 - root - DEBUG - New subject: original subject

And indeed, the message I receive has the original subject. Otherwise forwarding to my email address works.

How can I change/modify the subject field?

This should do the trick: https://pypi.python.org/pypi/ordereddict

Some comments don't reflect what the code actually does. Some of this was inherited, some of it was our own doing (e.g. salmon will now happily send non-ascii/non-utf8 emails)

The README needs going over too.

Unless someone has a Python 2.4 install somewhere, I think I'm going to drop 2.4 support and upgrade the code base to use 2.6 features

python-chardet claims to know this encoding, so either there's a bug in our code or chardet is a liar

If you try python setup.py install you'll get one of the errors in the title depending on what version of Python you are running. This isn't actually a Salmon issue, but appears to be python-daemon:

Python 2:

  File "/home/moggers/workspace/salmon/env/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 251, in finalize_options
    'dist_version': self.distribution.get_version(),     
  File "/tmp/easy_install-sAzJaZ/python-daemon-2.1.2/version.py", line 656, in get_version                        
  File "/tmp/easy_install-sAzJaZ/python-daemon-2.1.2/version.py", line 651, in get_version_info                   
  File "/tmp/easy_install-sAzJaZ/python-daemon-2.1.2/version.py", line 552, in get_changelog_path                 
  File "/home/moggers/workspace/salmon/env/lib64/python2.7/posixpath.py", line 122, in dirname                    
    i = p.rfind('/') + 1                                 
AttributeError: 'NoneType' object has no attribute 'rfind'  

Python 3:

  File "/home/moggers/workspace/salmon/env3/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 251, in finalize_options
    'dist_version': self.distribution.get_version(),    
  File "/tmp/easy_install-23sae6da/python-daemon-2.1.2/version.py", line 656, in get_version                     
  File "/tmp/easy_install-23sae6da/python-daemon-2.1.2/version.py", line 651, in get_version_info                
  File "/tmp/easy_install-23sae6da/python-daemon-2.1.2/version.py", line 552, in get_changelog_path              
  File "/home/moggers/workspace/salmon/env3/lib64/python3.6/posixpath.py", line 154, in dirname                  
    p = os.fspath(p)        
TypeError: expected str, bytes or os.PathLike object, not NoneType 

This problem needs reporting upstream, but I'm not sure there's actually a bug tracker.

Implement some docs.

Depends on #20

Currently, we expect to find the relative import conf when running an app, and conf.settings is hardcoded

We should do configuration a bit more like Django - the config module can be anywhere and app developers import from something like salmon.config.settings

Fix all nosetests make sure we still have 100% code coverage (or get as close as we can to it)

Provide generic CBSs (and mixins to complement decorators?)

Take Django's approach to CBVs (thread safe once you've called as_view, standardised methods to override, e.g. have a method that decides how to respond to SMTP clients, a method to convert or validate the message)

New example projects are needed.

Content filters to remove or reject mail (e.g. html messages to a plaintext mailing list) are more useful than the current spam module (which has never been tested and is going to be removed).

Some concrete examples are needed.

Django and Celery have similar options - it may be preferable to some (i.e. me) than just relying on log files.

Sorry but you can't just fork a project at some random point and declare it licensed contrary to what I want. I own copyright and that means if I want my BSD license then that's what you use.

Basically you ripped out my license modifications which then means you took away the rights I have to give you rights.

I'd appreciate it if you'd put the license back the way it is supposed to be. It's that way for a reason.

Python 3 support would be a great selling point

Our current Mail* classes are fine for the most part, but sometimes (e.g. Inboxen) we want our emails with warts and all. Or maybe we want to do something different with them?

A simple decorator that accepts a class would do just fine.

I'd really like PGP emails to be able to pass through Salmon transparently enough for the signature to remain intact.

It's completely opaque to revision control and is just a pre-packaged version of the directory structure of salmon/data

We should generate the zip automatically somehow or even remove it completely if it's not needed. We should look at what Django does when creating a new project.

Thanks for this great project! I noticed there hasn't been a release in a while, last one is version 2, from 20 Jun 2014. Almost three years old.. Would you like to make a new release? Or is the state after every commit equally stable and ready for production? If you'll make releases, may I suggest using semantic versioning so it's easy to notice, for instance, if the API changes in possibly backwards incompatible ways.

Is there any documentation on how to get this server up and running, I went to the lamson server site, but is down, and there doesnt seem to be a backup of the documentation on the wiki of the github project either.