Upon certificate issues, STARTTLS is ignored and the password sent in plaintext about imapsync HOT 12 CLOSED

Hello Dennis,

STARTTLS is supported.
Your paranoid imap server does not support plaintext authentication even with encryption.
The output should print what authentication it allows.

What are your command line options?

--authmd51 : Use MD5 authentification for host1.
--authmd52 : Use MD5 authentification for host2.
--authmech1 : Auth mechanism to use with host1:
PLAIN, LOGIN, CRAM-MD5 etc. Use UPPERCASE.
--authmech2 : Auth mechanism to use with host2. See --authmech1
--ssl1 : Use an SSL connection on host1.
--ssl2 : Use an SSL connection on host2.
--tls1 : Use an TLS connection on host1.
--tls2 : Use an TLS connection on host2.

On 19/01/2014 20:29, Dennis Schridde wrote:

Currently there seems to be no STARTTLS support in imapsync, is that correct? |--tls1/2| on my system still results in

|4 NO [PRIVACYREQUIRED] Plaintext authentication disallowed on non-secure (SSL/TLS) connections.
|

—
Reply to this email directly or view it on GitHub #15.

Au revoir, 09 51 84 42 42
Gilles Lamiral. France, Baulon (35580) 06 20 79 76 06

from imapsync.

Hello Gilles!

My server surely allows PLAIN and LOGIN over an encrypted connection. It authenticates against PAM, so there is no other way.

The commandline options were:

imapsync --dry --tls1 --host1 ... --user1 ... --password1 ... [host2=host1] --folder ... --delete2duplicates ----useheader Message-ID --usecache

from imapsync.

Hi Dennis,

Ok but the error message

"NO [PRIVACYREQUIRED] Plaintext authentication disallowed on non-secure (SSL/TLS) connections."

is given by the imap server, not by imapsync.

Try also

imapsync ... --ssl1

Then debug what's wrong with your server, maybe it wants certificates or something like that?

My server surely allows PLAIN and LOGIN over an encrypted connection. It authenticates against PAM, so there is no other way.

Au revoir, 09 51 84 42 42
Gilles Lamiral. France, Baulon (35580) 06 20 79 76 06

from imapsync.

The problem seems to be that imapsync cannot verify the server certificate (own CA). After STARTTLS fails locally, it tries to send CAPABILITY, which likely fails because the server expects the client to finish the STARTTLS sequence instead. Afterwards imapsync just reconnects, ignores the LOGINDISABLED capability and tries to LOGIN over a plaintext connection.

The major problems I see in this:

1. The user is never notified of the certificate issue.
2. imapsync ignores the --tls switch and sends my authentication plaintext. This should never ever happen.

Host1 connection
Connecting with IO::Socket::INET PeerAddr $HOST1 PeerPort $PORT1 Proto tcp Timeout 120 Debug 1
Connected to $HOST1
Read:   * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS LOGINDISABLED] Dovecot ready.
Host1: * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS LOGINDISABLED] Dovecot ready.
Sending: 1 STARTTLS
Sent 12 bytes
Read:   1 OK Begin TLS negotiation now.
ERROR: Unable to start TLS: SSL connect attempt failed with unknown error error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed at /usr/lib64/perl5/vendor_perl/5.18.1/Mail/IMAPClient.pm line 455.
[...]
Sending: 2 CAPABILITY
Sent 14 bytes
ERROR: socket closed while reading data from server at /usr/lib64/perl5/vendor_perl/5.18.1/Mail/IMAPClient.pm line 1629.
[...]
reconnecting to $HOST1, last error: socket closed while reading data from server
Connecting with IO::Socket::INET PeerAddr $HOST1 PeerPort $PORT1 Proto tcp Timeout 120 Debug 1
Connected to $HOST1
Read:   * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS LOGINDISABLED] Dovecot ready.
reconnect success(1) on try #1/3
Sending: 3 CAPABILITY
Sent 14 bytes
Read:   * CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS LOGINDISABLED
        3 OK Pre-login capabilities listed, post-login capabilities have more.
Host1: $HOST1 says it has NO CAPABILITY for AUTHENTICATE LOGIN
Sending: 4 LOGIN $USER $PASS
Sent 28 bytes
Read:   * BAD [ALERT] Plaintext authentication not allowed without SSL/TLS, but your client did it anyway. If anyone was listening, the password was exposed.
        4 NO [PRIVACYREQUIRED] Plaintext authentication disallowed on non-secure (SSL/TLS) connections.
ERROR: 4 NO [PRIVACYREQUIRED] Plaintext authentication disallowed on non-secure (SSL/TLS) connections. at /usr/lib64/perl5/vendor_perl/5.18.1/Mail/IMAPClient.pm line 1353.
[...]

from imapsync.

Dear Dennis,

The problem seems to be that imapsync cannot verify the server certificate (own CA). After STARTTLS fails locally,
it tries to send CAPABILITY, which likely fails because the server expects the client to finish the STARTTLS sequence instead.

Ok. Bad imapsync.

Afterwards imapsync just reconnects, ignores the LOGINDISABLED capability and tries to LOGIN over a plaintext connection.
The major problems I see in this:

1. The user is never notified of the certificate issue.
2. imapsync ignores the --tls switch and sends my authentication plaintext. This should never ever happen.

Yes, you are twice right.
Shame on me, imapsync does not check the return code of the function starttls() it uses,
so it does not print the error either. It used to do it but let's forget history.
Plus, the automatic reconnect behavior is not a good idea in this scenario and then add exposure.

I'll fix that soon.

Now did you find why you get "SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed"?

Can you try with option --ssl1_SSL_version fixing the ssl version, for example

imapsync ... --ssl1 --ssl1_SSL_version "SSLv3"

Possibilities:
"SSLv3"
"SSLv2"
"SSLv23"
"SSLv23:!SSLv2"

|Host1 connection
Connecting with IO::Socket::INET PeerAddr $HOST1 PeerPort $PORT1 Proto tcp Timeout 120 Debug 1
Connected to $HOST1
Read: * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS LOGINDISABLED] Dovecot ready.
Host1: * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS LOGINDISABLED] Dovecot ready.
Sending: 1 STARTTLS
Sent 12 bytes
Read: 1 OK Begin TLS negotiation now.
ERROR: Unable to start TLS: SSL connect attempt failed with unknown error error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed at /usr/lib64/perl5/vendor_perl/5.18.1/Mail/IMAPClient.pm line 455.
[...]
Sending: 2 CAPABILITY
Sent 14 bytes
ERROR: socket closed while reading data from server at /usr/lib64/perl5/vendor_perl/5.18.1/Mail/IMAPClient.pm line 1629.
[...]
reconnecting to $HOST1, last error: socket closed while reading data from server

Au revoir, 09 51 84 42 42
Gilles Lamiral. France, Baulon (35580) 06 20 79 76 06

from imapsync.

Now did you find why you get "SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed"?

Yes, the CA cert I added to /usr/local/share/ca-certificates was old and expired (i.e. not the one that signed the server's cert), so OpenSSL considered the chain untrusted. Rightly so, as there was a self-signed certificate in it. After I fixed that, imapsync worked like a charm.

from imapsync.

Dear Dennis,

I fixed this ugly bug in imapsync 1.582
It is hard to go in the same conditions as yours.
Now you fixed it on your side I supposed this won't interest you now.
I'll make it officially public later.

from imapsync.

Dear Dennis,

I fixed this ugly bug in imapsync 1.582
It is hard to go in the same conditions as yours.
Now you fixed it on your side I supposed this won't interest you now.
I'll make it officially public later.

On 22/01/2014 00:30, Dennis Schridde wrote:

Now did you find why you get "SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed"?

Yes, the CA cert I added to /usr/local/share/ca-certificates was old and expired (i.e. not the one that signed the server's cert), so OpenSSL considered the chain untrusted. Rightly so, as there was a self-signed certificate in it. After I fixed that, imapsync worked like a charm.

—
Reply to this email directly or view it on GitHub #15 (comment).

Au revoir, 09 51 84 42 42
Gilles Lamiral. France, Baulon (35580) 06 20 79 76 06

from imapsync.

Thanks!

My server is configured to listen only on port 143 and to require starttls before allowing login - if that helps you reproduce it. If you want, I can send you a copy of my dovecot config.

from imapsync.

Hi Dennis,

My server is configured to listen only on port 143 and to require starttls before allowing login -
if that helps you reproduce it. If you want, I can send you a copy of my dovecot config.

I already have tls servers.
What I don't have is tls servers with obsolete/broken certificates and I'm lazy
to build one. Anyway the fix should be ok since the issue was clear, the
output clearly showed the starttls() function detected it and now
imapsync check the exit status of starttls().
So we'll wait users that used to fall into the bug, imapsync working faking tls
on broken certificates, and the next release exiting in that case.

Au revoir, 09 51 84 42 42
Gilles Lamiral. France, Baulon (35580) 06 20 79 76 06

from imapsync.

What I don't have is tls servers with obsolete/broken certificates

It's easier than that. If you have a server with a valid certificate that was signed by a local CA, and you do not put the CA cert in /usr/local/share/ca-certificates, you'll be able to reproduce. CAcert will probably work as well, if it is not automatically trusted by your distribution.

from imapsync.

Hallo Dennis,

In fact I had nothing tp prepare,
Patch sounds working since some previous fake tls tests started to break.

But it more complicated, there's still something weird, since I found that:

--ssl1 --tls2 fails on host2 login with "Unable to start TLS: Cannot determine peer hostname for verification"
--tls1 --tls2 succeeds
--tls2 succeeds
--tls1 --ssl2 succeeds

I search.

2 ll_dev_reconnect_ssl_tls

Transfer started at Tue Jan 28 15:35:55 2014
PID is 13974
$RCSfile: imapsync,v $ $Revision: 1.582 $ $Date: 2014/01/24 01:43:19 $
Here is a [linux] system (Linux mail.cardio-sfc.org 2.6.32-54-generic #116-Ubuntu SMP Tue Nov 12 19:27:09 UTC 2013 i686)
With perl 5.10.1 Mail::IMAPClient 3.35
Command line used:
./imapsync --host1 localhost --ssl1 --user1 tata --passfile1 ../../var/pass/secret.tata --host2 localhost --tls2 --user2 titi --passfile2 ../../var/pass/secret.titi --folder INBOX --useuid --delete2
Temp directory is /tmp
PID file is /tmp/imapsync.pid
Modules version list:
Mail::IMAPClient 3.35
IO::Socket 1.31
IO::Socket::IP ?
IO::Socket::INET 1.31
IO::Socket::SSL 1.31
Net::SSLeay 1.35
Digest::MD5 2.39
Digest::HMAC_MD5 1.01
Digest::HMAC_SHA1 1.01
Term::ReadKey 2.30
Authen::NTLM 1.09
File::Spec 3.31
Time::HiRes 1.9719
URI::Escape 3.29
Data::Uniqid 0.12

Info: will act as --uidexpunge2
Info: turned ON syncinternaldates, will set the internal dates (arrival dates) on host2 same as host1.
Info: will try to use LOGIN authentication on host1
Info: will try to use LOGIN authentication on host2
Info: imap connexions timeout is 120 seconds
Host1: IMAP server [localhost] port [993] user [tata]
Host2: IMAP server [localhost] port [143] user [titi]
Host1: * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE AUTH=PLAIN ACL ACL2=UNION] Courier-IMAP ready. Copyright 1998-2008 Double Precision, Inc. See COPYING for distribution information.
Host1: localhost says it has NO CAPABILITY for AUTHENTICATE LOGIN
Host1: success login on [localhost] with user [tata] auth [LOGIN]
Host2: * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2008 Double Precision, Inc. See COPYING for distribution information.
Can not go to tls encryption on [localhost]:Unable to start TLS: Cannot determine peer hostname for verificationerror:00000000:lib(0):func(0):reason(0)
at ./imapsync line 2370
main::die_clean('Can not go to tls encryption on [localhost]:', 'Unable to start TLS: Cannot determine peer hostname for verif...', '\x{a}') called at ./imapsync line 2091
main::login_imap('localhost', 143, 'titi', undef, 'HUwtEd', undef, 120, 0, undef, ...) called at ./imapsync line 1014

On 24/01/2014 23:19, Dennis Schridde wrote:

What I don't have is tls servers with obsolete/broken certificates

It's easier than that. If you have a server with a valid certificate that was signed by a local CA, and you do not put the CA cert in /usr/local/share/ca-certificates, you'll be able to reproduce. CAcert will probably work as well, if it is not automatically trusted by your distribution.

—
Reply to this email directly or view it on GitHub #15 (comment).

Au revoir, 09 51 84 42 42
Gilles Lamiral. France, Baulon (35580) 06 20 79 76 06

from imapsync.