Support for X-GM-LABELS extension about imapsync HOT 20 OPEN

That would be a great feature

from imapsync.

I want to use X-GM-LABELS to port existing tags to Gmail (so both folders and tags turn into labels). I now have a (very simple) prototype that appears to work (patch below). I'm migrating from Zimbra to GMail, and Zimbra stores tags as IMAP keywords (flags that don't start with '' per RFC 3501), so I simply filter the flags and then issue an additional STORE +X-GM-LABELS command. But this prototype probably has a long way to go before it can be generally accepted (and isn't exactly the same as the original request).

diff --git a/imapsync b/imapsync
index 8df547c..c6591c2 100755
--- a/imapsync
+++ b/imapsync
@@ -5401,6 +5401,16 @@ EOF
 }


+sub maybe_add_gmail_labels {
+    my( $id, $h1_flags ) = @_;
+
+    # Create a space delimited string of flags from the previous host that don't start with \
+    myprint("gmail labels: starting with $h1_flags\n");
+    my $labels = join(' ', grep(!/^\\/, split(/\s/, $h1_flags)));
+    myprint("gmail labels: labels are '$labels'\n");
+    $imap2->store($id, "+X-GM-LABELS ($labels)") if $labels;
+}
+
 # GlobVar
 # $dry
 # $max_msg_size_in_bytes
@@ -5450,6 +5460,8 @@ sub append_message_on_host2 {
                        $nb_msg_transferred += 1 ;
                         $h1_nb_msg_processed +=1 ;

+                       maybe_add_gmail_labels($new_id, $h1_flags);
+
                         my $time_spent = timesince( $begin_transfer_time ) ;
                         my $rate = bytes_display_string( $total_bytes_transferred / $time_spent ) ;
                         my $eta = eta( $time_spent,

from imapsync.

Hi SCbash,

Since I also want this feature I started digging into it today. I
now have a (very simple) prototype that appears to work (patch
below). I'm migrating from Zimbra to GMail, and Zimbra stores tags
as IMAP keywords (flags that don't start with '' per RFC 3501), so
I simply filter the flags and then issue an additional STORE
+X-GM-LABELS command.

Very good!

But this prototype probably has a long way to go before it can be generally accepted.

Generally for imapsync means often only me so the way is not long.
I'm kidding, generally accepted is easy as long as it is not a default
before so adding this good idea as an option will definitively ensure
its acceptance.

+sub maybe_add_gmail_labels

I suggest to fing another name. Suggestions

• --flagstogmaillabels
• --addgmaillabels
• --gmaillabels

Au revoir,
Gilles Lamiral. France, Baulon (35580)
mob 06 19 22 03 54
tel 09 51 84 42 42

from imapsync.

Thanks Gilles! I'll fork and start working on a real implementation. Hopefully a PR will be ready in a day or two.

+sub maybe_add_gmail_labels

I suggest to fing another name. Suggestions

• --flagstogmaillabels
• --addgmaillabels
• --gmaillabels

Yeah, this was a very quick prototype so I wasn't thinking about naming :) --flagstogmaillabels feels a little long, but I like how descriptive it is. Maybe --keywordstogmaillabels since the IMAP spec (and the Thunderbird documentation... ah, and Wikipedia) uses "keywords" (but wow that's even longer...).

from imapsync.

--xgmlabels

from imapsync.

I'm almost done with a working version that uses --porttagstogmail and --excludetags (since in the process of sync'ing more mail I found Zimbra uses some IMAP keywords internally, e.g. $Forwarded). But we can have more conversations on the PR I'm about to publish...

from imapsync.

Hi Gregory,

--xgmlabels

Then,

• what name for the reverse xgmlabels to keywords?
• what name for xgmlabels to xgmlabels?

Au revoir,
Gilles Lamiral. France, Baulon (35580)
mob 06 19 22 03 54
tel 09 51 84 42 42

from imapsync.

Hi Gilles,

just had a stupid moment and wanted to suggest something simple and silly. Not sure about reverse. Perhaps --xgm2key? ...and --xgm2xgm?

from imapsync.

Hi,

There is nothing stupid at the time of suggestions!
I do agree that simple names are a good aim.

Le 04/11/2016 à 09:27, Gregory Netsas a écrit :

Hi Gilles,

just had a stupid moment and wanted to suggest something simple and silly. Not sure about reverse. Perhaps |--xgm2key|? ...and |--xgm2xgm|?

Good too!

In fact, for the simplicity I'm thinking of --gmail1 and --gmail2 options
that would trigger all the good options for gmail syncs.

And I'm also thinking about detecting gmail account and
trigger --gmail1 or --gmail2 automagically.

So we're still good with long and explicit names.

Au revoir,
Gilles Lamiral. France, Baulon (35580)
mob 06 19 22 03 54
tel 09 51 84 42 42

from imapsync.

Hi Stephen,

I'm almost done with a working version that uses |--porttagstogmail| and |--excludetags|
(since in the process of sync'ing more mail I found Zimbra uses some IMAP keywords internally, e.g. $Forwarded).
But we can have more conversations on the PR I'm about to publish...

For the names I prefer the trio:

--tagstogmail
--tagstogmailnotmatching regex # instead of --excludetags
--tagstogmailmatching regex

With --tagstogmailmatching or --tagstogmailnotmatching regex
that will automatically activate --tagstogmail

Same behavior than
--delete2folders
--delete2foldersonly regex
--delete2foldersbutnot regex

Only if you have time and agree!

On my side I will add

--delete2foldersmatching regex
--delete2foldersnotmatching regex

in order to remove the very unsafe --delete2foldersonly and --delete2foldersbutnot,
especially in online context ( powerful but unsafe eval "regex" form used for both).

Au revoir,
Gilles Lamiral. France, Baulon (35580)
mob 06 19 22 03 54
tel 09 51 84 42 42

from imapsync.

Sounds good Gilles. I've converted the names of the existing arguments, and I'm investigating if I tell GetOpt to make --tagstogmailmatching and --tagstogmailnotmatching mutual exclusive... I'll try to get --tagstogmailmatching fully implemented this week, but I can't make any guarantees.

I did use my existing code to port almost 24k messages to Gmail over the weekend :)

from imapsync.

Hi Stephen,

Sounds good Gilles. I've converted the names of the existing arguments,

Good.
I wonder if label in the name would be more talkative,

--tagstogmlabels

Don't bother, I'll decide later.

and I'm investigating if I tell GetOpt to make |--tagstogmailmatching| and |--tagstogmailnotmatching| mutual exclusive...

Don't make them mutual exclusive, it's useless behavior,
make them usefully complementary.

In fact, don't code what you don't need, so forget --tagstogmailmatching

I did use my existing code to port almost 24k messages to Gmail over the weekend :)

Good!
Didn't reach Gmail limits?

Au revoir,
Gilles Lamiral. France, Baulon (35580)
mob 06 19 22 03 54
tel 09 51 84 42 42

from imapsync.

In fact, don't code what you don't need, so forget --tagstogmailmatching

Okay, I'll take another look at what I have, and run a quick test, but I think it's pretty close to done. I can optionally squash the rename commits.

I did use my existing code to port almost 24k messages to Gmail over the weekend :)

Good!
Didn't reach Gmail limits?

Apparently not? There was at least one time one of the servers dropped connection, but imapsync reestablished the connection and kept going.

from imapsync.

In fact, don't code what you don't need, so forget --tagstogmailmatching

Okay, I'll take another look at what I have, and run a quick test, but I think it's pretty close to done.

From the standpoint of "don't code what you don't need", I think I'm all finished, but I'm happy to address further tweaks/fixes if necessary. To complete the success story, my boss (CTO) is starting to use my modifications to migrate about 15 more people from Zimbra to Gmail.

Thanks!

from imapsync.

Hi Stephen!
Any code to share?

from imapsync.

Huh, I thought I'd created a pull request for this back when we talked about it... (sorry for the delay in responding, we're currently moving cross county and I'm settling into a new job)

Oh yes, there it is #82. I doubt I'll find time to do more work on the branch, but if there are small changes I might be able to find an hour or two to clean up the branch for merging. Thoughts?

from imapsync.

Yes I found your code after exploring the imapsync github repository.
For now I've coded the sync of X-GM-LABELS for a faster sync from Gmail to Gmail. It looks to work well.
I let this issue open until I add your contribution. The default behavior of imapsync is to resync flags as well so it will take more effort than just copy your code.
Thanks!

from imapsync.

any news on this? (adding an additional tag to all synced mail)

from imapsync.

or less gmail specific: adding an additional folder to all synced mail

from imapsync.

The --synclabels and --resynclabels options are coded and available.

Option --synclabels is on with --gmail1 --gmail2 but I've just discovered that --resynclabels was not!

There is no --addlabel yet nor --regexlabel option.

A good feature supported by imapsync is that --subfolder2 is ok with --synclabels and --resynclabels It's not what you ask but it's very powerful way to backup several gmail accounts to a single gmail account and restore them later with --subfolder1

from imapsync.