macOS Server v5.7 no longer supports APSP Mail etc. Certificates about dovecot-xaps-daemon HOT 77 CLOSED

Ran into this problem as well and used Vagrant to quickly pull up a virtual High Sierra macOS VM. Here the steps, in case it's helpful for someone:

1. Install Vagrant
2. In your project folder add a Vagrantfile with this content:

Vagrant.configure("2") do |config|
  config.vm.box = "monsenso/macos-10.13"
end

3. Start VM

$ vagrant up

4. Then disable USB ports, start the machine and install macOS Server from the app store. After exporting the files from the keychain copy them to the local machine:

$ vagrant plugin install vagrant-scp
$ vagrant scp default:/Users/vagrant/Documents/PushEmail.p12 .

5. Stop or destroy the virtual machine when done

$ vagrant halt

from dovecot-xaps-daemon.

@freswa Thanks again.

That’s too bad. APNS is therefore dead along with Server.app for open source mail servers.

It looks like the only reasonable approach left is to file a report at https://bugreport.apple.com/ and request that Server.app version 5.7 and beyond provide support to generate bundled Mail/Calander/Contacts APNS certificates that can be exported and used to configure open source servers.

from dovecot-xaps-daemon.

@dreness Your link points to a text, which is related to the APNS protocol, not the certificate portal. I think we can easily support the HTTP/2-base APNs since it also allows authentication via certificates. So why would the support of the certificate creation and renewal API be affected here?

Apparently I had assumed that cert-based auth was going away along with the binary push protocol, which looks like an error on my part. I don't see any deprecation notices when going through the motions of requesting a new APNS cert via the developer portal, and the docs do say that cert auth is supported for the new push service, so perhaps you are not yet doomed :)

from dovecot-xaps-daemon.

@essandess

1. no, FQDN isn’t relevant to APNS.
2. no, LDAP config isn’t relevant to APNS.
3. see point 1 :)

APNS certs are used only in the context of client CERT authentication to authenticate your server when it connects to the push servers. If your server presents a APNS cert trusted by the APNS servers (based on who signed it), that’s all that is needed to interact with APNS.

from dovecot-xaps-daemon.

Hi all,

Just wanted to say thanks for all the good tips & tricks here.

I also wanted to report that in my case (OSX 10.13.3, server.app 5.4), exporting the com.apple.servermgrd.apns.mail key was not enough : as the name implies, it contained only a key, and therefore the openssl tool wasn't able to extract any certificate from it (it returned a zero size file).

Using intuition and trial and error I was able to find the right certificate by opening the corresponding identity entry and then finding the linked "preferred" certificate :

Then I exported the certificate in .pem format and used it to perform the openssl s_client test which ran successfully.

from dovecot-xaps-daemon.

I've hacked together a solution for getting certificates that works on Catalina with the help of files from macOS Server 5.10. (Caveat: I haven't actually tested my certificates yet, but I'll update here if they don't work for some reason. The mail cert works.) It's also worked, with adjustments, on other combinations of OS/Server versions; open an issue in my repo if you need some help.

It's here. A lot of the request uses standard CSRs, so I implemented it using openssl commands. The remaining part is an unusual signature, and I found that the code for it still exists in Server 5.10 binaries and wrote a wrapper to access it. My solution doesn't use the keychain (just files in the working directory), so it also eases the exporting and converting steps.

Hoping Apple continues signing certs and ideally makes it easier!

cc apple/ccs-calendarserver#508 lionheart/openradar-mirror#19283 matthewpowell/pushnotify#1

from dovecot-xaps-daemon.

It's not possible to issue certificates for the bundles

• com.apple.mail
• com.apple.calendar
• com.apple.contact

via https://identity.apple.com/pushcert/.
See also this comment from an Apple employee: apple/ccs-calendarserver#508 (comment)

FastMail and Yahoo have a license for Apple Mail Push which is stated by the FastMail devs here: https://github.com/cyrusimap/cyrus-imapd/tree/master/docsrc/imap/reference/admin/eventsource.rst#apple-push-service

from dovecot-xaps-daemon.

@freswa Thank you.

Just to be clear, I read @dreness’s answer as:

1. It is possible to use https://identity.apple.com/pushcert to generate APNS certs for dovecot-based IMAP Mail servers.

“Apple’s developer portal can vend APNS certs, and those should work with this server and third party clients that support APNS”

2. It is not possible to generate bundled APNS for Contacts and Calendar servers.

“Apple’s calendar and contacts apps only support APNS when used with push certs issued to a bundle ID that can only be used by Apple (and only by Server.app)”

3. Write https://bugreport.apple.com/ to ask for a policy change to address number 2.

Is this correct?

If number 1 is correct, it would be helpful to test the possibility of using https://identity.apple.com/pushcert/ to generate a working Mail push certificate for IMAP.

from dovecot-xaps-daemon.

Apple’s developer portal can vend APNS certs

It is in general possible to generate APNS certs via the pushcert portal. But the portal is actually made for MDM certificates. The request for such an MDM certificate has to be signed by the MDM software vendor though.

and those should work with this server and third party clients that support APNS

In general MDM and normal apps use the same APNS. So you can issue APN with any certificate.
BUT the push notifications you send to apples server are limited to the app with the bundle id of the certificate. Which means that spotify developers are not able to send push notifications to any other app but their own.

Apple’s calendar and contacts apps only support APNS when used with push certs issued to a bundle ID that can only be used by Apple (and only by Server.app)

This is actually not a policy but a technical requirement. The certificates determines the app to deliver the PN to. There is no mention of a bundle id in the payload. So Apple would have to create a whole new APNS which won't happen.

What could happen is that Apple opens up the mechnism they are issuing the certificates for server.app with as that wouldn't require any technically changes in their APNS.

from dovecot-xaps-daemon.

Just for the record, I spent 2 hours trying to make it work with https://identity.apple.com/pushcert and could not manage to get a APNS certificate for com.apple.mail.
I ended up using an old version of macos server on an outdated OS. Let's hope they maintain this as long a possible, and provide a way to sign certificates for 3rd-party mail servers.

from dovecot-xaps-daemon.

That's exactly what I'm doing too: I have an old macos server version installed in a parallel desktop VM I use once a year to regenerate push mail certificates...

from dovecot-xaps-daemon.

Thank you all for the updated information. Would you please comment on the specific DNS, and other configuration steps you used to get this working?

It appears that a medium–longish term solution will be to keep around a bootable High Sierra system with Server.app 5.7 running, then use it to refresh carts once a year.

I haven’t dug very deep down into the way that Server.app generates APNS certificates, or the APNS certs themselves, and would appreciate knowing these configuration details in preparation to migrating to upgraded server hardware:

• Must the FQDN on old Server.app v. 5.7 (the one used to generate APNS certs) be the exact same FQDN of the new server?
• Must Open Directory on the old Server.app v. 5.7 be configured as a Directory Master for working APNS certs? Or is LDAP configuration irrelevant for this process?
• How is APNS configured to know the FQDN of the server?

I’m guessing that the FQDN must be the same from the old server and the new server, which sets up a bit of a PITA for DNS name collision on the LAN while migrating and using this workaround.

All advice would be appreciated.

from dovecot-xaps-daemon.

I'm building mail server on a macOS box with a postfix+dovecot+rspamd+solr stack. From the parts I have so far, I see that this will be a much better mail server than the old one Apple configured in Server.app.

I hadn’t considered running Server.app v.5.7 in a VM to get the APNS certs.

Anyone have thoughts or experience on the choice of one of the open source VM options to run macOS 10.13? VMware? Is this easy? Or is it easier to keep an old box around?

from dovecot-xaps-daemon.

I’ve set up a VirtualBox VM for this purpose. It’s a little tricky at first, because you need to convert the macOS install image for VBox, but after that it works flawlessly and easy. There are a couple of guides on the web covering the installation process.

from dovecot-xaps-daemon.

@pagaille and All,

Thanks for the report. If you could make a gist of your steps, that would help immensely.

I'm done building my mail server and now on the final stretch migrating to apple/ccs-calendarserver. I've found that apple/ccs-calendarserver no longer builds out-of-the-box because of new openssl and dependency versions. Nevertheless, it's pretty straightforward to get a build going that uses MacPorts library dependencies, which is in any way more secure than building with app-specific libraries within an app that is updated less frequently. My WIP notes and working fork/branch:

• https://gist.github.com/essandess/4c9b5d906896ff4aef6170fc18f2c293
• https://github.com/essandess/ccs-calendarserver/tree/essandess_specific

I'm hoping that APNS will also work with the new CalendarServer too, though I haven't figured out how to configure this yet.

I'm at the stage of wrapping CalendarServer within a secured reverse proxy, the way it comes on Server.app v5.7, though without Server.app's web services.

from dovecot-xaps-daemon.

@dreness Thank you for your comments.

I've made a couple of related PRs over at apple/ccs-calendarserver that address the issues I observe trying to build its latest:

• apple/ccs-calendarserver#536
• apple/ccs-twistedextensions#20

Also, if you would be able to provide guidance on the issue of Python 2.7 EOL-ing on 1 January 2020, that would be appreciated:

• apple/ccs-calendarserver#537

I imagine (hope) that there's a frenzy of activity behind the scenes to migrate any macOS dependencies on Python2 to Python3, and also hope that the lessons learned from that exercise can be applied to bring apple/ccs-calendarserver up-to-date with a clear path for dependency security patches.

Going through this process, I've appreciated anew the solid code base that CalendarServer is built with, and hope that it can be maintained because as I see it, CalendarServer is really the best open source option for serving calDAV and cardDAV.

from dovecot-xaps-daemon.

In case any of you have your own apple/ccs-calendarserver instances running and have Server.app APNS certs, it should be very easy to test APNS for both apns:com.apple.calendar and apns:com.apple.contact. Here's the relevant code blocks and an example .plist configuration.

It appears that this should be straightforward for all Mail/Calendar/Contact services:

https://github.com/apple/ccs-calendarserver/blob/fbd0e11675cc0f64a425581b5c8398cc1e09cb6a/twistedcaldav/stdconfig.py#L861-L893

https://github.com/apple/ccs-calendarserver/blob/a5b19e00a25f16ff2d56f899228bc6432b8b8828/twistedcaldav/test/test_config.py#L88-L118

from dovecot-xaps-daemon.

Thanks @essandess

I don't really see the added value of a gist here ? But I'm not really used to it.

For anybody interested : here are the binaries for CentOS 7.6 : https://nextcloud.gaillet.be/index.php/s/8zXcSsgrTWCjkAY

By the way I use Nethserver to replace OSX Server. Close to perfect, great distro actively developed, great community, good support, subscription support available.

from dovecot-xaps-daemon.

All, I have a mail-server MacPorts submission that’s basically a combination of the now-defunct macOS Server.app Mail service with a lot of newer capabilities: solr fts, Rspamd milter, DKIM, improved spam training workflow, and more. Any comments or corrections would be appreciated:
macports/macports-ports#4914

I haven’t gotten around to APNS yet, but plan to do so after writing a ccs-calendarserver port.

from dovecot-xaps-daemon.

Anyone recently managed to create/renew the APSP mail certificates (using an old version of Server)? I'm trying to renew my certificated and all I get is an "unexpected error". Very strange.

from dovecot-xaps-daemon.

ATM I get the following raw response:

<key>Response</key>
    <dict>
        <key>Status</key>
        <dict>
            <key>ErrorDescription</key>
            <string>Something went wrong. Please retry after sometime. :: </string>
            <key>ErrorMessage</key>
            <string>Internal Server Error :: </string>
            <key>ErrorCode</key>
            <integer>-80001</integer>
        </dict>
    </dict>

@ahvlima Maybe try again tomorrow?

from dovecot-xaps-daemon.

Thanks @freswa for trying it out.

That's what I'm getting (-80001). First 2 attempts I got the description. On all later attempts got only the error number.

Trying again tomorrow was my first idea. I just hope it's still possible to renew. I have a Sierra VMWare instance that I've been using for 3 years (renewed 2 times already). Will try tomorrow :) Just hope to get it renewed before it expires (in 2 days).

from dovecot-xaps-daemon.

@ahvlima API works again.

from dovecot-xaps-daemon.

from dovecot-xaps-daemon.

Working here too - phew 👍

from dovecot-xaps-daemon.

Back to my deprecation analysis, I'm considering moving to a High Sierra VM, if Apple also decide to phase out by OS version. And I don't think you can only renew once a year, so I will give it a try in a few months. I do not see light at the end of the tunnel, unless Apple decides to change the rules of the game.

from dovecot-xaps-daemon.

I have a working APNS plugin with the latest version of dovecot core 2.3.7.1 and have submitted a PR to add this capability to MacPorts dovecot. See macports/macports-ports#5482 and specifically the patch file.

I’ve also created a fork apns of the dovecot core repo, https://github.com/essandess/core/tree/apns, which will make it easier to create patch files from the upstream repo. This also has a patch file https://github.com/essandess/core/blob/apns/dovecot-core-pushnotify.patch.

I’ve confirmed that APNS works on a macOS 10.14 server and iOS 13 Mail clients.

from dovecot-xaps-daemon.

@ahvlima

We must take good care of these VMs.

Would you please post the steps and tack you’re using to create a working VM?

from dovecot-xaps-daemon.

I have a working APNS service on a mail server and would like to configure a working ccs-calendarserver for APNS.

I have all the valid APNS certificates and keys—both in my System Keychain and as pkcs12 and PEM files—but haven’t been able to figure out the right calendarserver configuration to get APNS up and running on ccs-calendarserver. I also have the full TLS chain of trust from https://www.apple.com/certificateauthority/ and have verified that all certs are valid.

It’s not clear what the APNS settings should point to: PEM files? System Keychain items?

For example, when I use PEM files in the configuration as shown immediately below, I get this error when I launch calendar server:

Cannot retrieve CalDAV APNS passphrase from keychain

calendarserver.plist (just showing CalDAV section):

            <key>APNS</key>
            <dict>
                <key>Enabled</key>
                <true/>
                <key>CalDAV</key>                                                                       
                <dict>                                                                                  
                    <key>CertificatePath</key>                                                          
                    <string>/var/calendarserver/Library/CalendarServer/etc/apns/com.apple.calendar.cert.pem </string>             
                    <key>PrivateKeyPath</key>                                                           
                    <string>/var/calendarserver/Library/CalendarServer/etc/apns/com.apple.calendar.key.pem</string>                            
                    <key>AuthorityChainPath</key>                                                       
                    <string>/var/calendarserver/Library/CalendarServer/etc/apns/com.apple.calendar.chain.pem</string>                                                                                                       
                </dict>                                                                                 

What should the settings for CertificatePath, PrivateKeyPath, and AuthorityChainPath point to? Also, I see the settings Passphrase, KeychainIdentity, and Topic in https://github.com/apple/ccs-calendarserver/blob/master/conf/caldavd-stdconfig.plist . Should these be set as well? To what settings?

I should be very having working APNS on calendarserver, but don’t see any documentation for the configuration and am unsure what the correct magic incantation is.

When I get this up and running, I’ll amend the MacPorts PR for ccs-calendarserver to include APNS.

Relevant PRs:

• macports/macports-ports#4978
• macports/macports-ports#5482

from dovecot-xaps-daemon.

Thank you everyone for this thread... I've had Dovecot with APNS for years until Apple removed the feature. Does anyone have a working manual how to reissue the certificate?

@m3nu , I tried to follow your way but ran into few things - the VM Vagrant is importing is a clean installation of High Sierra. Whenever I go to App Store and try to install MacOS Server I am getting the message that I will need to upgrade MacOS to new versions to use Server... AM I doing something wrong?

from dovecot-xaps-daemon.

The easiest is to just copy /Applications/Server.app from an existing backup into a new High Sierra VM.

If you don't have a backup, try these steps: https://support.apple.com/en-us/HT201377

from dovecot-xaps-daemon.

You can also find old versions to most Apple software in your developer account. I didn't check if Apple Server is there, but it should.

from dovecot-xaps-daemon.

Thanks a lot everyone! I managed to make it working properly.

For those who are interested, I've made a full walkthrough how to set up certificate/key, service and plugin here: https://tinyurl.com/vupqa4o. Credits for this walkthrough go to @m3nu , @essandess, @pagaille and of course @st3fan - without your helpful comments and commits it wouldn't be that easy.

from dovecot-xaps-daemon.

from dovecot-xaps-daemon.

@pagaille apologies - for some strange reason the previous link worked in one browser and didn't work in another.

It's quite long, not really format for the issue comment :) If maintainer would be okay I can format it and put on GitHub.

from dovecot-xaps-daemon.

Also, APNS also works in MacPorts Calendar and Contacts Server. See macports/macports-ports#4978.

from dovecot-xaps-daemon.

@essandess is see you have made it working for the latest dovecot version, am i correct? on the macport page: macports/macports-ports#5482

if so, can i use that patched dovecot version on my linux (debian) install? because what i know is that macports is for software on the mac.

from dovecot-xaps-daemon.

Yes, APNS works for the latest dovecot. See the apns branch of my fork: https://github.com/essandess/core/tree/apns. This has both a patch file that works, and the the branch itself.

I’ll go rebase it to the latest, but the this patch file works: https://github.com/essandess/core/blob/apns/dovecot-core-pushnotify.patch

You’ll also need the pushnotify.pl perl script available from MacPorts here: https://github.com/macports/macports-ports/blob/master/mail/dovecot2/files/pushnotify.pl. And note for the future that this port will be renamed to dovecot soon.

from dovecot-xaps-daemon.

@essandess
what do you think about the guide from @lsergei posted a few post earlier
https://tinyurl.com/vupqa4o

as i understand, there there is not used a patched dovecot, but just added the plugin which has to be compiled, and added the daemon.

whats the difference comparing to your patched dovecot version which also needs a perl script

from dovecot-xaps-daemon.

as i understand, there there is not used a patched dovecot, but just added the plugin which has to be compiled, and added the daemon.

whats the difference comparing to your patched dovecot version which also needs a perl script

Those instructions explicitly use the dovecot-xaps-plugin provided by the author of this very repo, and the basis of the patches in my repo. I had to make several modifications to make them work with the latest dovecot version and with a cmake build.

I haven’t tracked the original code enough to know if it still works with dovecot, but afaik dovecot must be modified to be aware of XAPPLEPUSHSERVICE and some other stuff.

from dovecot-xaps-daemon.

FWIW, here’s the history of this APNS code base:

All the credit for this goes to Matthew Powell on GitHub. He wrote the Dovecot patch and notification daemon to get push notifications working. I just packaged everything into nice FreeBSD ports.

Much of this knowledge was discovered a few years ago by Stefan Arentz in his dovecot-xaps-plugin project.

My repo is a modification of Matthew Powell’s Dovecot patch and notification daemon.

from dovecot-xaps-daemon.

as i understand, there there is not used a patched dovecot, but just added the plugin which has to be compiled, and added the daemon.
whats the difference comparing to your patched dovecot version which also needs a perl script

Those instructions explicitly use the dovecot-xaps-plugin provided by the author of this very repo, and the basis of the patches in my repo. I had to make several modifications to make them work with the latest dovecot version and with a cmake build.

I haven’t tracked the original code enough to know if it still works with dovecot, but afaik dovecot must be modified to be aware of XAPPLEPUSHSERVICE and some other stuff.

Allright so if I understand it correct
Dovecot has to be modified
So it is not just a matter of adding the plugin on a vanilla dovecot install

That is a bit confusing when I read his how to page

Yes with every new dovecot version the patch needs te be updated as well right?

That’s the downside, because it would be easier when you could just install the repo version with a plug-in, so when you update the server, dovecot gets updated too. Now dovecot updates are to be done manually

from dovecot-xaps-daemon.

I think there is a misunderstanding here. You absolutely can use this plugin with an unpatched vanilla Dovecot. You just have to compile the plugin and the corresponding daemon. There are no patches or perl scripts required for Dovecot whatsoever. I even installed it from binary packages. I only had to add the xaps.conf config file from the repo.

from dovecot-xaps-daemon.

I think there is a misunderstanding here. You absolutely can use this plugin with an unpatched vanilla Dovecot. You just have to compile the plugin and the corresponding daemon. There are no patches or perl scripts required for Dovecot whatsoever. I even installed it from binary packages. I only had to add the xaps.conf config file from the repo.

@A1bi
Oh
Which binaries did you use?
Because the ones here are source files that need to be compiled

And on what dovecot version do you use the aps? And on what distro?

from dovecot-xaps-daemon.

I meant that I used binary packages for Dovecot itself. The plugin I compiled from source, which is pretty straightforward and the actual compilation only takes a few seconds. The instructions in the Readme file are all you need. The only minor downside is that you need to recompile the plugin with every major Dovecot update because of changes in the ABI. But again, this is a five minute job.

I'm on the most recent version of Dovecot (2.3.9.2). I'm using FreeBSD instead of Linux, but that shouldn't make a difference here.

from dovecot-xaps-daemon.

Cool thnx

from dovecot-xaps-daemon.

@Cullum @matthewpowell @st3fan @freswa Would any of you please provide some background or opinions on the distinction between the two different dovecot APNS approaches in dovecot-xaps-plugin and pushnotify?

from dovecot-xaps-daemon.

Seems to work fine with the regular dovecot and just the plugin and daemon

Just tested it on my Debian server

from dovecot-xaps-daemon.

when in all the processes is the /var/lib/xapsd/xapsd.json file filled?
upon starting for the first time it is empty

when does a client get added? upon service restart? upon dovecot login? or upon setting the mail client to push?

when i empty the file, the clients do not get re-entered right away, so then notifications do not appear

from dovecot-xaps-daemon.

iOS issues a XAPPLEPUSHSERVICE command via IMAP which contains all the needed tokens. The dovecot plugin passes them to the daemon which stores them in the xapsd.json file. You shouldn't do anything with the xpasd.json file. It's all handled by the daemon.

from dovecot-xaps-daemon.

@essandess I started with Apple's own Dovecot patches and built the APNS daemon from there – this seems to be a cleaner reimplementation using plugins.

I also added a script to notify on all changes to the INBOX so that iOS devices get message read/flagged notifications in addition to new mail. It's messy (I described it in the README as an "evil hack") but it works well enough.

As for why the two projects exist... this project didn't exist when I was first figuring this stuff out back in 2012. I posted it to GitHub much later when a couple people were asking me about it and this seemed to be the easiest way to share my work.

from dovecot-xaps-daemon.

@essandess Can you explain how to generate apns:com.apple.mail APNS certificates with your solution? Or are you still using old pre-5.7 MacOS server for this purpose?

from dovecot-xaps-daemon.

@matthewpowell Thank you very much for that insight. It’s great to have two working solutions. I plain to maintain the update of @matthewpowell’s modified dovecot approach as a MacPorts variant, but would also entertain calls to use @st3fan’s approach.

from dovecot-xaps-daemon.

@essandess Can you explain how to generate apns:com.apple.mail APNS certificates with your solution? Or are you still using old pre-5.7 MacOS server for this purpose?

It’s necessary to use macOS Server.app version 5.6 or below:

• Download a High Sierra image, use it to create a VM on the platform of your choice (I use VMWare Fusion, and there are other options, some free)
• Install macOS Server.app 5.6, either from the App Store or simply copied from an old backup
• Renew APNS certificates
• Export the APNS certs from your Keychain as p12 files and use OpenSSL to convert to pem format for use within dovecot APNS plugins, or Apple Calendar and Contacts Server.
• On MacPorts, see port notes dovecot2 +apns and port notes calendar-contacts-server +apns .

from dovecot-xaps-daemon.

I tried activating APNS today using the existing apns variant of MacPorts dovecot2. Turning the daemon on with launchctl reliably (that is: tested twice with hard reboot in between, that was enough on a production system) hosed my macOS Mojave + Server + MacPorts dovecot2/postfix/etc. A safe reboot and removing APNS was required to get the system stable again. It is not clear at all if the apns daemon is the actual culprit or that it triggers a bug in another part on my specific system, or the it is a conflict between different elements of my setup, but it did trigger it reliably. I am thus interested in the 'modified dovecot' approach which may not trigger this problem.

from dovecot-xaps-daemon.

there must be some script/code that does the certificate renewal and i'm wondering if that is part of the dovecot software or keychain or server.app, would be nice to kinda grab that functionality so this process could run without macos

from dovecot-xaps-daemon.

there must be some script/code that does the certificate renewal and i'm wondering if that is part of the dovecot software or keychain or server.app, would be nice to kinda grab that functionality so this process could run without macos

That's the very point of this ticket, IMO that's what we should focus on here, rather than discussing alternatives to XAPS-daemon which are unfortunately out of scope...

To date the situation is still the same, issuing a valid APNS certificate for apns:com.apple.mail (only option working with iOS mail client) is still only possible via old pre-5.7 MacOS Server.

from dovecot-xaps-daemon.

I have been using this variant reliably for a while, and before it was merged into MacPorts.

To the best of my knowledge, there’s nothing in the modified dovecot that would cause the issues you describe, and I have never experienced them.

HOWEVER, I received another report of apns squirliness from a MacPorts user. I’ve asked them to compare to your description.

Please provide basic setup notes: macOS and dovecot versions, etc. I’m on macOS 10.15.3 and dovecot 2.3.9.2.

I do not wish to hijack this thread with issues from the parallel approach, but it is possibly relevant in providing impetus to do the work to adopt dovecot-xaps-daemon. And if possible it’s good to keep two working approaches around.

I tried activating APNS today using the existing apns variant of MacPorts dovecot2. Turning the daemon on with launchctl reliably (that is: tested twice with hard reboot in between, that was enough on a production system) hosed my macOS Mojave + Server + MacPorts dovecot2/postfix/etc. A safe reboot and removing APNS was required to get the system stable again. It is not clear at all if the apns daemon is the actual culprit or that it triggers a bug in another part on my specific system, or the it is a conflict between different elements of my setup, but it did trigger it reliably. I am thus interested in the 'modified dovecot' approach which may not trigger this problem.

from dovecot-xaps-daemon.

discussing alternatives to XAPS-daemon which are unfortunately out of scope

Yes, please post the MacPorts issues to https://trac.macports.org/

from dovecot-xaps-daemon.

One year has almost passed since we started discussing this issue. Apple kindly sent me a mail reminding to renew the APNS certificates. Here is the procedure for the record:

• Use the server manager, go to the Mail category and click "Edit Notifications" then "Renew"

• Use the keychain to export both the apns.mail key (p12 extension) and the associated certificate (cer extension).

• Issue the following commands to convert the exported certificate and the key to pem format (NB @st3fan you might want to edit your readme according this)

openssl x509 -inform der -in certificate.cer -out certificate.pem
openssl pkcs12 -in com.apple.servermgrd.apns.mail.p12 -nocerts -nodes -out key.pem

• You might want to test the connection using that command :

openssl s_client -connect gateway.push.apple.com:2195 -cert certificate.pem -key key.pem

• Finally copy those two files to your mail server and restart the xapsd service.

from dovecot-xaps-daemon.

lets hope it stay's this way at apple's part so we can keep enjoying push

from dovecot-xaps-daemon.

It's July 5 and it's still signing! So far, so good. Just renewed and this time I had no problems.

from dovecot-xaps-daemon.

I've hacked together a solution for getting certificates that works on Catalina with the help of files from macOS Server 5.10. (Caveat: I haven't actually tested my certificates yet, but I'll update here if they don't work for some reason.) It's also worked, with adjustments, on other combinations of OS/Server versions; open an issue in my repo if you need some help.

It's here. A lot of the request uses standard CSRs, so I implemented it using openssl commands. The remaining part is an unusual signature, and I found that the code for it still exists in Server 5.10 binaries and wrote a wrapper to access it. My solution doesn't use the keychain (just files in the working directory), so it also eases the exporting and converting steps.

Hoping Apple continues signing certs and ideally makes it easier!

cc apple/ccs-calendarserver#508 lionheart/openradar-mirror#19283 matthewpowell/pushnotify#1

That sounds really nice, i'm going to give it a try soon because my cert will expire in september

how did you find out about this? can you tell a bit about what you did to find out? i'm curious.

tcpdumping the old method ? tell us :)

as far as i can see you need the serverOS only for the certs. i'm i right?

from dovecot-xaps-daemon.

Hoping Apple continues signing certs and ideally makes it easier!

The 'legacy' APNS certificate portal should keep working until November 2020, according to https://developer.apple.com/news/?id=11042019a, which states:

If you send push notifications with the legacy binary protocol, we recommend updating to the HTTP/2-based APNs provider API as soon as possible. You’ll be able to take advantage of great modern features, such as authentication with a JSON Web Token, improved error messaging, and per-notification feedback.

The Apple Push Notification service (APNs) will no longer support the legacy binary protocol as of November 2020.

I would guess that after November 20, it will no longer be possible to acquire APNS certs that can push Mail.app, regardless of the macOS Server version. Sorry for the bummer news, but I'd rather be a bummer than see people get blindsided...

from dovecot-xaps-daemon.

@dreness Admittedly I am new to this stuff, but I don't see the connection. Why would deprecation of the binary protocol mean deprecation of that portal? Also, a similar project https://github.com/matthewpowell/pushnotify has a commit that says it updated to HTTP/2, so apparently they are able to push Mail.app without the legacy protocol. In any case, thanks for the warning that the binary protocol will be shut down. Am I missing something about the certificate portal aspect though? Thanks.

from dovecot-xaps-daemon.

@dreness Your link points to a text, which is related to the APNS protocol, not the certificate portal. I think we can easily support the HTTP/2-base APNs since it also allows authentication via certificates. So why would the support of the certificate creation and renewal API be affected here?

from dovecot-xaps-daemon.

I've hacked together a solution for getting certificates that works on Catalina with the help of files from macOS Server 5.10. (Caveat: I haven't actually tested my certificates yet, but I'll update here if they don't work for some reason.) It's also worked, with adjustments, on other combinations of OS/Server versions; open an issue in my repo if you need some help.

It's here. A lot of the request uses standard CSRs, so I implemented it using openssl commands. The remaining part is an unusual signature, and I found that the code for it still exists in Server 5.10 binaries and wrote a wrapper to access it. My solution doesn't use the keychain (just files in the working directory), so it also eases the exporting and converting steps.

Hoping Apple continues signing certs and ideally makes it easier!

cc apple/ccs-calendarserver#508 lionheart/openradar-mirror#19283 matthewpowell/pushnotify#1

could we also run this in like a linux host instead of a macOS host?

from dovecot-xaps-daemon.

@Zjemm Until it's full reversed, it depends on a Mach-O binary which you may try to run with Darling

from dovecot-xaps-daemon.

@Zjemm Linux support is coming and basically done, it just needs more polishing. You can try the linux branch. You need Docker, and you need to take servermgr_certs from macOS Server.app 5.10, and put it in the directory with hashy.c. Currently you need to separately compile https://github.com/shinh/maloader in the path ~/tmp/maloader, and change the absolute path in hashy.c. (Lots of warnings about "undefined symbol" are normal and harmless.)

from dovecot-xaps-daemon.

https://github.com/shinh/maloader

@scintill cool, let me know if i can test things for you

from dovecot-xaps-daemon.

@Zjemm and others: The master branch now supports Linux (and only Linux¹), still requiring the binary from Server.app and now requiring Docker. It also supports taking the binary from any version of Server.app now. Please let me know your test results. If you don't need a certificate, just cd test; ./run will do a local-only test to see that it can reproduce an expected output.

[¹]: I think Linux support will hit a greater percentage of people, and I don't want the overhead of supporting two platforms. When it's fully reversed, there will be much less code, so macOS support could be added.

from dovecot-xaps-daemon.

first of all, great work

unfortunately i'm not a docker fan myself (i'll stick with lxc/lxd containers :) ) so i can't run any test with this version.
wouldn't it not be much cleaner to put it in a chroot ? then there is less dependancies

from dovecot-xaps-daemon.

from dovecot-xaps-daemon.

My code is now fully open-source! It runs on Linux with minimal dependencies (no docker).

If someone wants to make a nicer implementation, it looks like there is some overlap with how MDM vendor certs work, so you could start from an existing codebase that does that, and randomly select one of the certs from vendorcerts/ (I don't know why, but the original code chooses randomly.) You can compare your output against test/expected/request-body.

from dovecot-xaps-daemon.

nice, and much cleaner like this.

first test looks good.

i'll try to renew my real certs soon to see if the certs also work correctly

from dovecot-xaps-daemon.

from dovecot-xaps-daemon.

This is getting off topic...

from dovecot-xaps-daemon.