adb-sync doesn't copy files properly if names are same on local and remote about adb-sync HOT 5 OPEN

I'm trying this small change to DiffLists. The issue here is that comparing mod times between OSX files and Android device files doesn't line up. I'm having to munge the times to get the comparison to skip files already copied over previously. I tried snapping to minutes and then adding 16. So how can this sync ever work, if the timestamps don't have even a remote correspondence. I tried with --times and without.

    if a_item[0] == b_item[0]:
      # if a.mtime is greater, then want in a_only to do the copy
      if a_item[1].st_mtime > b_item[1].st_mtime:
        a_only.append(a_item)
      else:
        both.append((a_item[0], a_item[1], b_item[1]))
      a_revlist.pop()
      b_revlist.pop()

Modification timestamps seems to be an underlying flaw in the Android file system.
https://issuetracker.google.com/issues/36930892#c128

And some circular references back to that unresolved issue, but more of a description of the problem.
https://www.xda-developers.com/diving-into-sdcardfs-how-googles-fuse-replacement-will-reduce-io-overhead/

It's clear from --times still being broken in the release buld that timestamp comparisons in adb-sync were never tested/used.

from adb-sync.

Also seems like there's an "adb shell stat -c "%y" filename" command that might return more precise timings than "ls -al" which only shows timings to minutes. But this would have to get called for each filename. At least here the seconds are available.

2020-11-25 00:16:05.000000000

There's also "ls -ll" to get more accurate timestamps so they don't need to be rounded, so I'm going to try using that.

adb shell ls -all

drwxrwx--x 14 u0_a184 sdcard_rw 3488 2020-11-25 08:57:14.000000000 -0800 .

from adb-sync.

I rewrote the timestamp parsing to use microseconds by cropping the nanos returned by ls -ll, and also using the UTC timestamp returned. I stoped using --times since it only sets low resolution timings via "adb touch" and it's slow, and the "adb push" uses to emulate copy already preserves modstamps.

Then I added the modstamp tests on non-directories mentioned above, and fixed the "both" logic to skip copies off a newer modstamp. This all finally got adb-sync to sync files properly to the device.

from adb-sync.

Since it's an underpinning of adb-sync, I looked into the behavior of adb push by itself on and Android 9 device and macOS. My understanding is that macOS emulates the Android file system where Linux/Win have native implementations.

Looks like "adb push" if you pull the cable to the device in the middle of any push leaves a partial file with the current modstamp set as well. This means you have a partial copy of the file on the device, and since adb sync only checks modstamps, then it breaks our builds.

Also if you push, touch a file, push again and cancel the push then the file is deleted. Wasn't expecting that either. Thought that the file system would still have that original file. push and cancel seems to not leave a file on the device either.

Considering also adding a size check since the "ls -ll" command includes those too. At least it may be a checksum if there is correspondence between file sizes, and they're not thrown off by pagesize on the two file systems.

from adb-sync.

We ended up dumping adb-sync except for excruciatingly slow deletes that are done one file at a time even if all the files under a folder are deleted. The general idea is as follows, and this is crazy fast and more safe than adb-sync.

	# First use "adb sync" to copy newer files to the device.
	"adb push --sync folder/* /sdcard/Android/data/com.appdomain/files/"

	# Now use python script "adb-sync" to delete files not present in current build.  no-clobber so that it doesn't copy any files.  Strip this python script down to just handling deletes.
	"adb-sync --delete --no-clobber folder/ "/sdcard/Android/data/com.appdomain/files/"

from adb-sync.