thunderbird / autoconfig Goto Github PK

Please add the following Github topics to this repository:

• https://github.com/topics/mozilla
• https://github.com/topics/thunderbird
• https://github.com/topics/autoconfig
• https://github.com/topics/ispdb
• https://github.com/topics/imap
• https://github.com/topics/pop3
• https://github.com/topics/smtp

Thanks!

We want to clean up the config files to only include elements that Thunderbird supports. One element that Thunderbird is very unlikely to ever use is webMail. So we'd like to remove it.

But we don't want to break software relying on this information. If you are the author of such a software, please let us know in a comment, so we can work out a removal date that allows you to migrate to a different database or mechanism.

Example of a webMail element:

<webMail>
  <loginPage url="https://accounts.google.com/ServiceLogin?service=mail&amp;continue=http://mail.google.com/mail/" />
  <loginPageInfo
    url="https://accounts.google.com/ServiceLogin?service=mail&amp;continue=http://mail.google.com/mail/">
    <username>%EMAILADDRESS%</username>
    <usernameField id="Email" />
    <passwordField id="Passwd" />
    <loginButton id="signIn" />
  </loginPageInfo>
</webMail>

We should provide some way to automatically check that proposed XML files are valid, and possibly some way to automatically generate one by filling in some form fields somewhere.

Note that this is not a suggestion that we return to the idea of a fully database backed live web app, only that we add/improve some tools.

I was looking at https://support.mozilla.org/en-US/questions/1444845 where it became apparent that standard gmail settings should be what is required, instead of nothing from the auto config. Nothing is what you get when the autoconfiguration runs because there is no response from any server to any of the attempts to detect settings.So even if ALL @nyu email account are not gmail. We will still be improving the odds of a satisfied customer because the current situation is a full manual configuration in all cases.

mail.setup: imap.nyu.edu:993 socketType=3 imap: timeout GuessConfig.sys.mjs:558:21
mail.setup: smtp.nyu.edu:465 socketType=3 smtp: timeout GuessConfig.sys.mjs:558:21
mail.setup: mail.nyu.edu:993 socketType=3 imap: timeout GuessConfig.sys.mjs:558:21
mail.setup: mail.nyu.edu:465 socketType=3 smtp: timeout GuessConfig.sys.mjs:558:21
mail.setup: nyu.edu:993 socketType=3 imap: timeout GuessConfig.sys.mjs:558:21
mail.setup: nyu.edu:465 socketType=3 smtp: timeout GuessConfig.sys.mjs:558:21
mail.setup: imap.nyu.edu:143 socketType=2 imap: timeout GuessConfig.sys.mjs:558:21
mail.setup: smtp.nyu.edu:587 socketType=2 smtp: timeout GuessConfig.sys.mjs:558:21
mail.setup: mail.nyu.edu:143 socketType=2 imap: timeout GuessConfig.sys.mjs:558:21
mail.setup: smtp.nyu.edu:25 socketType=2 smtp: timeout GuessConfig.sys.mjs:558:21
mail.setup: nyu.edu:143 socketType=2 imap: timeout GuessConfig.sys.mjs:558:21
mail.setup: mail.nyu.edu:587 socketType=2 smtp: timeout GuessConfig.sys.mjs:558:21
mail.setup: mail.nyu.edu:25 socketType=2 smtp: timeout GuessConfig.sys.mjs:558:21
mail.setup: pop.nyu.edu:995 socketType=3 pop3: timeout GuessConfig.sys.mjs:558:21
mail.setup: nyu.edu:587 socketType=2 smtp: timeout GuessConfig.sys.mjs:558:21
mail.setup: mail.nyu.edu:995 socketType=3 pop3: timeout GuessConfig.sys.mjs:558:21
mail.setup: nyu.edu:25 socketType=2 smtp: timeout GuessConfig.sys.mjs:558:21
mail.setup: nyu.edu:995 socketType=3 pop3: timeout GuessConfig.sys.mjs:558:21
mail.setup: smtp.nyu.edu:587 socketType=0 smtp: timeout GuessConfig.sys.mjs:558:21
mail.setup: smtp.nyu.edu:25 socketType=0 smtp: timeout GuessConfig.sys.mjs:558:21
mail.setup: pop.nyu.edu:110 socketType=2 pop3: timeout GuessConfig.sys.mjs:558:21
mail.setup: mail.nyu.edu:587 socketType=0 smtp: timeout GuessConfig.sys.mjs:558:21
mail.setup: mail.nyu.edu:110 socketType=2 pop3: timeout GuessConfig.sys.mjs:558:21
mail.setup: mail.nyu.edu:25 socketType=0 smtp: timeout GuessConfig.sys.mjs:558:21
mail.setup: nyu.edu:110 socketType=2 pop3: timeout GuessConfig.sys.mjs:558:21
mail.setup: nyu.edu:587 socketType=0 smtp: timeout GuessConfig.sys.mjs:558:21
mail.setup: imap.nyu.edu:143 socketType=0 imap: timeout GuessConfig.sys.mjs:558:21
mail.setup: nyu.edu:25 socketType=0 smtp: timeout GuessConfig.sys.mjs:558:21
mail.setup: guessConfig failed: Can't find a server accountSetup.js:877:29
mail.setup:
Object { _message: "Hostname is empty or contains forbidden characters. Only letters, numbers, - and . are allowed.", stack: "Exception@resource:///modules/accountcreation/AccountCreationUtils.sys.mjs:114:3\nMalformedException@resource:///modules/accountcreation/Sanitizer.sys.mjs:241:34\nhostname@resource:///modules/accountcreation/Sanitizer.sys.mjs:115:13\ngetUserConfig@chrome://messenger/content/accountcreation/accountSetup.js:1618:44\nvalidateManualEditComplete@chrome://messenger/content/accountcreation/accountSetup.js:2086:31\neditConfigDetails@chrome://messenger/content/accountcreation/accountSetup.js:1731:10\n_guessConfig/self._abortable<@chrome://messenger/content/accountcreation/accountSetup.js:879:14\ncheckDone@resource:///modules/accountcreation/GuessConfig.sys.mjs:157:22\noutgoingError@resource:///modules/accountcreation/GuessConfig.sys.mjs:272:5\n_checkFinished@resource:///modules/accountcreation/GuessConfig.sys.mjs:674:12\n_tryAll/</thisTry.abortable<@resource:///modules/accountcreation/GuessConfig.sys.mjs:560:16\n_error@resource:///modules/accountcreation/GuessConfig.sys.mjs:1133:18\ntimeoutFunc@resource:///modules/accountcreation/GuessConfig.sys.mjs:1138:13\nnotify@resource://gre/modules/Timer.sys.mjs:49:17\nnsIPrompt_promptPassword@resource:///modules/MsgAsyncPrompter.sys.mjs:593:15\npromptPassword@resource:///modules/MsgAsyncPrompter.sys.mjs:377:40\n_promiseAuthPrompt/<@resource:///modules/MsgAsyncPrompter.sys.mjs:42:18\n_promiseAuthPrompt@resource:///modules/MsgAsyncPrompter.sys.mjs:40:12\nrun@resource:///modules/MsgAsyncPrompter.sys.mjs:65:23\n" }
accountSetup.js:1621:27
mail.setup:
Object { _message: "Hostname is empty or contains forbidden characters. Only letters, numbers, - and . are allowed.", stack: "Exception@resource:///modules/accountcreation/AccountCreationUtils.sys.mjs:114:3\nMalformedException@resource:///modules/accountcreation/Sanitizer.sys.mjs:241:34\nhostname@resource:///modules/accountcreation/Sanitizer.sys.mjs:115:13\ngetUserConfig@chrome://messenger/content/accountcreation/accountSetup.js:1665:44\nvalidateManualEditComplete@chrome://messenger/content/accountcreation/accountSetup.js:2086:31\neditConfigDetails@chrome://messenger/content/accountcreation/accountSetup.js:1731:10\n_guessConfig/self._abortable<@chrome://messenger/content/accountcreation/accountSetup.js:879:14\ncheckDone@resource:///modules/accountcreation/GuessConfig.sys.mjs:157:22\noutgoingError@resource:///modules/accountcreation/GuessConfig.sys.mjs:272:5\n_checkFinished@resource:///modules/accountcreation/GuessConfig.sys.mjs:674:12\n_tryAll/</thisTry.abortable<@resource:///modules/accountcreation/GuessConfig.sys.mjs:560:16\n_error@resource:///modules/accountcreation/GuessConfig.sys.mjs:1133:18\ntimeoutFunc@resource:///modules/accountcreation/GuessConfig.sys.mjs:1138:13\nnotify@resource://gre/modules/Timer.sys.mjs:49:17\nnsIPrompt_promptPassword@resource:///modules/MsgAsyncPrompter.sys.mjs:593:15\npromptPassword@resource:///modules/MsgAsyncPrompter.sys.mjs:377:40\n_promiseAuthPrompt/<@resource:///modules/MsgAsyncPrompter.sys.mjs:42:18\n_promiseAuthPrompt@resource:///modules/MsgAsyncPrompter.sys.mjs:40:12\nrun@resource:///modules/MsgAsyncPrompter.sys.mjs:65:23\n" }
accountSetup.js:1668:27

The enable info is specific to the email configs, so it should be inside that email element.
The configs here already have it correctly, but as #84 points out, the spec disagrees with the configs. In this case, the configs are correct and the spec was wrong.

This issue is about correcting the spec.

Thunderbird is currently hardcoding all the parameters needed to use OAuth2, including all server URLs.

We should move these OAuth2 parameters to the config files here. The config file format supports these now.

3 steps:

• 1. Add the parameters to the official specification (DONE)
• 2. Add the parameters the config files in the ISPDB, for the ISPs that support OAuth2. (This issue, now DONE.)
• 3. Make Thunderbird use these parameters, and remove the hardcoded values from Thunderbird. ¹ ² (Bugzilla bug to be filed, TODO)

¹ Unfortunately, Thunderbird nonetheless needs to hardcode the client key.
² Automatic client key retrieval is not supported by the large mail providers.

Migrated from Bugzilla (https://bugzilla.mozilla.org/show_bug.cgi?id=1884549)

Initial message from the reporter:

Cox are currently in a migration of all their email account to Yahoo.
While information beyond that linked to is in short supply, the information in the ISP database is already out of date for some and will soon be out of date for all the Cox email users.

I have not seen the email from Cox, but previous experience would indicate that yahoo will be advising folk to use the new account wizard for the transition, ad that sounds like what occured here https://support.mozilla.org/en-US/questions/1441949. Creating new account is what they have advised in the past for similar transitions and from my observations in bug 1884548 modifying your account settings is not a simple process that I would be prepared to recommend any longer.

I have asked the user in the support topic for a copy of the email they received to initiate their changes, if I receive it I will attach the content to this bug for informational purposes.

We currently host a config for Cox here: https://github.com/thunderbird/autoconfig/blob/f254b7d2b043a2b91b99e22633e56284ffe5a4fd/ispdb/cox.net.xml

Information on the migration from Cox's website: https://www.cox.com/residential/support/coxnet-email-transitioning-to-yahoo.html

My understanding is that the migration is still ongoing, so unsure what our timings should be with regards to updating existing configs - we might want to wait till it's fully done. But then, I don't believe Cox has published any information regarding a deadline for their users to migrate their accounts.

Hello @benbucksch and others,

Recently I came across autoconfig because Thunderbird uses it. I think this is very useful protocol to set email easily. However, at the moment the protocol isn't standardized yet.

I would like to see autoconfig standardized. This means that autoconfig will be registered at https://www.iana.org/assignments/well-known-uris/well-known-uris.xhtml and the XML format (maybe also the old format) and the used HTTP URLs (maybe also the old paths) will be known by both email client implementers and email server hosters. It also will cause the protocol to be more mainstream and more used.

If I can help writing this standard (I think making an RFC is the easiest), I would like to do that.

Thanks in advance

Thanks to the awesome support by Ryan @ryanleesipes I'm quite sure that an issue I have with a German email hosting provider might be fixed by changing its entry in the Thunderbird ISPDB.

When I was looking for the source repository of the ISPDB (https://autoconfig.thunderbird.net/v1.1/) I previously found the repository at https://github.com/mozilla/ispdb/ . But given the fact that its last commit was in August 2012 I assumed that the ISPDB is not based on the repo at https://github.com/mozilla/ispdb/ .

Maybe a small explanation and link can be added to the README file at https://github.com/mozilla/ispdb/blob/master/README.md to guide people like me to the correct repo at https://github.com/thundernest/autoconfig ?

It would be nice if the autoconfig would also be able to store infos about SMTP relay servers :)

The ptd.net configuration is missing outgoing servers for port 465/SSL and port 25/STARTTLS. Because of changes to the www.ptd.net website, the documentation URL should be updated.

@Sancus

Hello!

Here is the "sapo.pt" configuration file.

Remove the ".txt" from the filename.

When will it be available in Thunderbird for me to test?

Thanks!

sapo.pt.txt

It would be nice to have a way to add default names for certain folder types.
I experienced (in the past) how thunderbird didn't recognize the junk folder and created it's own for some servers. (not sure whether this is fixed, but this is something mail clients will experience from time to time)

how should we treat loginButtons without id nor name attributes?
example: https://www.1und1.de/login
<loginButton /> like this and the application has to figure out how to submit the form?

Looking at the page it actually duplicates IDs like "collapse", urgh..

• I created a new hotmail account and autoconfig set IMAP to Normal password
• This resulted in Hotmail not receiving email in TB unless I changed it to OAuth2
• I think the file to change is: https://autoconfig.thunderbird.net/v1.1/hotmail.com
• Until this is changed, we need to document it: thunderbird/knowledgebase-issues#37

In the README file at https://github.com/thundernest/autoconfig/blob/master/README.md there is this list:

Be sure to read the docs below before changing anything:

• Autoconfig mechanism documentation, including file format definition and explanations.
• How to configure your domain without being added to the ISPDB here. This is useful for small ISPs and domains that are not open to the public.

Unfortunately both pages return a 404 error (Page not found).

Hello.

We need to change the following XML file for umich.edu in the ISP Database:
https://autoconfig.thunderbird.net/v1.1/umich.edu

The new version needs to be as follows:

<?xml version="1.0" encoding="UTF-8"?>
<clientConfig version="1.1">
	<emailProvider id="umich.edu">
		<domain>umich.edu</domain>
		<displayName>University of Michigan</displayName>
		<displayShortName>umich.edu</displayShortName>
		<incomingServer type="imap">
			<hostname>imap.gmail.com</hostname>
			<port>993</port>
			<socketType>SSL</socketType>
			<username>%EMAILADDRESS%</username>
			<authentication>OAuth2</authentication>
			<authentication>password-cleartext</authentication>
		</incomingServer>
		<outgoingServer type="smtp">
			<hostname>smtp.gmail.com</hostname>
			<port>465</port>
			<socketType>SSL</socketType>
			<username>%EMAILADDRESS%</username>
			<authentication>OAuth2</authentication>
			<authentication>password-cleartext</authentication>
		</outgoingServer>
	</emailProvider>

	<oAuth2>
		<issuer>accounts.google.com</issuer>
		<!--  https://developers.google.com/identity/protocols/oauth2/scopes  -->
			<scope>https://mail.google.com/ https://www.googleapis.com/auth/contacts https://www.googleapis.com/auth/calendar https://www.googleapis.com/auth/carddav</scope>
			<authURL>https://accounts.google.com/o/oauth2/auth</authURL>
			<tokenURL>https://www.googleapis.com/oauth2/v3/token</tokenURL>
	</oAuth2>
</clientConfig>

I believe I have submitted the needed pull request here; can someone please confirm this will be reviewed and changed, if all is in order? #36

Best regards,
[b]

brian cors (he / him)
University of Michigan | Information and Technology Services
Collaboration Services Manager

How is this attribute supposed to be interpreted/what is it used for? :)

I can imagine, but it would still be good to have a sentence explaining it in the specs:
https://www.bucksch.org/1/projects/thunderbird/autoconfiguration/config-file-format.html

example:
https://github.com/thunderbird/autoconfig/blob/master/ispdb/t-online.de.xml#L49

Before I open a pull request I wanted to discuss my planned change in general.

I think SSL/TLS is more secure and therefore preferred over STARTTLS when transferring emails, especially in regards to SMTP submission.

To cite RFC 8314:

RFC 8314, Section 3: Implicit TLS
With STARTTLS, the client establishes a cleartext application session and determines whether to issue a STARTTLS command based on server capabilities and client configuration. If the client issues a STARTTLS command, a TLS handshake follows that can upgrade the connection. Although this mechanism has been deployed, an alternate mechanism where TLS is negotiated immediately at connection start on a separate port (referred to in this document as "Implicit TLS") has been deployed more successfully. To encourage more widespread use of TLS and to also encourage greater consistency regarding how TLS is used, this specification now recommends the use of Implicit TLS for POP, IMAP, SMTP Submission, and all other protocols used between an MUA and an MSP.

I identified a German email hosting provider that supports both SSL/TLS and STARTTLS. Given the cited paragraph above I would like to change it's entry in the ISPDB from STARTTLSS to SSL/TLS.

The latest official spec of the config file format supports adding CalDAV and CardDAV setup parameters, so that a client can automatically configure the calendard and address book sync.

This adds the server URLs for the major ISPs that provide CalDAV and CardDAV services for their freemail users.

Note: In order to add it to the configuration, all email users of the ISP should have access to these features. We should only add it, if most of the users of that ISP have access to the CalDAV and CardDAV services.

We should have a documented step by step review policy for adding XML files to the database, likely via Pull Request.

There is some helpful information in the official docs, but not a real review guide or policy on what's supposed to be allowed and not.

This would likely work in combination with #4 to make submission easier.

I'd love to use your repository as it contains useful configurations - however without license file the default license is "all rights are reserved".
Is it this on purpose? If not, could you publish it under e.g. MIT license?

Thanks for the great work!

I'm sorry. I made a mistake with this comment.