odeke-em / drive Goto Github PK

From rakyll#59

Do NOT use this application when you're working with classmates on a project. The risk of losing data in files is way too damn high. Here's why:

The merging mechanism isn't just stupid, it's nonexistent. Have this situation:

a/
   b.txt
   c.txt

Have one example. It's the absolute worst one:

1. Edit b.txt
2. drive pull a
3. b.txt will show an "M", stating it's modified. If you proceed with pulling, b.txt will be overwritten with an older file.

Why would you want to pull the folder? Well, your teammates might have edit c.txt, or maybe they have even edited b.txt. When you see that b.txt was modified, you're led to believe that your teammates modified it. The fact is that it could mean that, but it's probably because you edited b.txt yourself. There is no way to tell.

In a project I handed in today I was very careful to push and pull per file. Luckily no one else edited files at the same time I did, that would have put me in deep trouble. That or maybe that did happen, drive wouldn't have told me so there would be no way for me to find out unless they actually told me. Despite the per file push/pulling I still lost data, most likely because I pulled the folder thinking I had pushed my per file changes.

Files just get overwritten. Both on push and on pull. No merging, no conflict mechanism, no warnings that you're overwriting unsaved changes (or haven't merged with the latest version yet) and a downright misleading status message. This tool is not safe for projects in which people cooperate.

Here's the example of what I'm talking about:

Tested on the AUR version of drive (0.0.8-1 (Thu Feb 5 18:57:44 CET 2015))

Evey option I try with the list command fails with 'no gd context is found; use gd init'. It did work earlier in the day, pull and push both work, I have tried init several times and there is no difference. Is some magic required ?

regards CB

Support for uploads if they fail or get paused.

$ drive --help
Usage of drive:

version 0.0.5

It would be nice to do a:

$ drive push Pictures/My* and it would push everything that was 'My Pics' or 'My Stuff'.

The default file list is sorted by directory then reverse chronological order, this may have its uses but it not the way most people think. Could there be (or is there) an option to sort the file names in alphabetical order.

Regards CB

This is the opposite of the NoClobber option.
With force, one is requesting an overwrite(an Add) and ignoring of modification changes.

Should figure out how much free space is left for uploads and then report to the user before even the upload is made.

I occasionally get this error when pushing a directory with subdirectories that have to be created. At least, that seems to be in common. I'm not sure if anything bad actually happens as a result as I'm doing uploads with lots of files and there's no indicator of what remote path doesn't exist. Maybe something is slow at taking note of a newly created directory at times?

Great software, very useful!

I have just started using drive and I am trying to push a directory but I see <nil> remote path doesn't exist for every file that it tries to push. Is there something I need to do before I push?

The full output is here https://gist.github.com/bausmeier/1178ab3d857d4f1e0920.

I just sync'ed my Drive on my local filesystem. All documents created on Google Drive are now zero byte on my local disk. Is this a feature that has not been implemented?

I installed the program using go get. Do I need to switch to some stable release version?

Hi,

We created a php script with an exec function to execute the drive client in a distinct way and then added the php script to cron. However we are getting issues in running the drive client in cron. cron uses sh / bash to execute commands and it cannot execute drive. The error is: cannot execute binary file

I'm not sure on how to go about this issue. Please help.

Reported by @ikwyl6 with such reports as http://paste2.org/IA1LhYCF
When working on big dirs

Hi ,
I am trying to "install" drive, but I get this error...

root@debian-amd64:/home/user# go get -u github.com/odeke-em/drive/cmd/drive
# golang.org/x/net/context
/usr/lib/go/src/pkg/golang.org/x/net/context/context.go:250: function ends without a return statement

Not sure how to call or report this issue. I am trying to use drive for the first time, and it doesn't really behave in a confidence-inducing way:

I call drive init followed by drive pull. So far so good, I end up with a directory that holds all my files. If I now call drive push without changing anything, I get a long list of modified and deleted files! Some files are even listed twice, once as modified and once as deleted. Obviously, I am very reluctant to call drive push as I fear it will delete the listed files (more than 100) irrevocably from by google drive.

What is going wrong here?

When I attempt to run "drive init," the program Segmentation Faults

Enable a copy command for copying remote files and directories between directories on Google Drive.

From rakyll#53

Would be nice to show, before the 'Proceed with changes', some overall details of the changes before committing:

1. Overall how many files are + and - at the end of the remote/source file comparison.
2. Number of files to be changed, shown by 1st level dirs under the 'root' dir
3. After user commits to proceeding, add push/pull kB or MB rate and total file size on the same line as:

112 / 1614 [======>----------------------------------------------------------------------------------------] 6.94 % 19h43m9s

Off the top of my head the simplest idea is to allow a user's credentials to be saved to a location of their choice just like you'd do after ssh-keygen, or even allow for credentials encrypting and decryption on every usage. This is an issue brought to my attention by noticing that doing a pull/push with -hidden enabled affects the credentials files. Also @bh4017 raised a scenario in an offline email in which the contents of a mounted Google drive saved on removable media if acquired by anyone would give them access to the victim's drive.
Please feel free to pitch in your thoughts.

I've created an aur pacakge based on the release snapshots. https://aur.archlinux.org/packages/drive/
It's not perfect nor am I... But for anyone using archlinux your welcome to install it and enjoy drive.

This is based on the releases, not git version.

Uploading/pushing a 54MB file to google drive shows no progress.

The force option will transform 'no change' into either an addition or modification of the destination file depending on whether the file already exists or not. This option is useful for example when doing backups and exports yet files are in sync.

Implement a method to allow for exploration of the contents on the remote drive.

With ther frequent versions available I was trying to do a clean test with no historical baggage. The drive init command does not forget old directorires. Could the wiki and/or doc files include a list of files (including the hidden ones) to delete to ensure a clean start.

regards CB

If a request for a drive initialization is made, then aborted the state of the directory should be what it was before the init.

If we had a path e.g '/documents/https://foo.com/curation.png'
On pulling we get:

 |- /documents

       |- https:

            |- foo.com

                  |- curation.png

It should be:

 |- /documents

       |- https://foo.com/curation.png

Implement the methods for trash handling.

The current diff-ing library trips out at about > 5MB or so with a panic due to a slice out of bounds error.

It would be really great to be able to drive push a data stream read from stdin.

I hope this could work like this:

echo "testing123" > test.txt
cat test.txt | drive push --stdin stdinTest.txt

Then you'd have a file in your Google Drive called stdinTest.txt which contains the string "testing123"

What do you think?

Hello

I've looked through the docs and command line options but I can't see where to specify the destination path. I'd like to backup some folders from the local machine but I don't want them in the root of my Google Drive. Am running 0.0.4.

Many thanks!

Seems function "checksumDiffers" in types.go calculates the checksum of a local file even if it differs in size from the remote file. This seem unnecessary and increases resolving time if many files have been changed.

Edit: Enhancement

I have a screenshot of the problem here:

Sorry for the large size, I have a High DPI screen.

The files in the screenshot come from a collaboration project. I can download them through drive, but drive never acknowledges that they're there. The diff states that the files either don't exist locally (which they do) or that the modified dates are different. It will say the same thing right after pulling. The files don't seem to be corrupt. I also didn't touch them, so the fact that the modified dates are different is a mystery to me.

This happened in a huge folder containing subfolders of files. Most files do not suffer this issue. It really is only this set of files. I have already tried removing those files and redownload them, but that doesn't solve it.

I'm having this issue, I can't easily get rid of it, but I have no clue on how to fix it. It might have something to do with Visio and Visio temp files.

I see that c281591 was tagged as 0.0.7 (10 days ago), after which 6649a6c was tagged as 0.0.7-rc.1 (5 days ago), after which daa0e17 was tagged as 0.0.7-rc.2 (2 days ago). How can the 0.0.7 release come before its release candidates?

Maybe 0.0.7-rc.1 and 0.0.7-rc.2 should have been tagged as release candidates for 0.0.8?

It would be nice if drive did not change modification times. If it is necessary to keep the modification times in sync then it should send the modification time along with the file.

[mlbaker@localhost ~]$ drive version
drive version 0.0.4
[mlbaker@localhost ~]$

I was suspicious that a push of a new directory tree was missing some files so here is a detailed test.
I deleted all files and emptied the trash of my Google drive and deleted all files from my local path and made a new tree with a single top level and some sub directories. When the first push was complete I made a remote listing and then pushed the same folders again and 3 more files were required. Examination of the logs and listing showed the 3 files were not included in the first push logs.

All the details here... http://pastebin.com/taC9qkxi

Drive 0.0.4b from the Ubuntu ppa on Linux Mint 17.1 64bit.

Not sure if this is a bug or a feature request.

I have a fileserver mounted via SSHFS that I try to sync to Drive. The first upload using Drive Push works fine; however on the second Push and thereafter the program hangs at "Resolving...". If I leave it long enough (several minutes) it sometimes spits out some additions to make, but some of the files it lists are unmodified and already uploaded, and it misses others.

Let's say I have a 1000MB file locally and I push it up to my Google drive. Let's say the 'drive push' command takes 100 seconds to complete (I guess this is roughly the upload time). Now let's say I locally modify 100MB in that 1000MB file and run 'drive push' again. I expect this operation take 10 seconds to complete. My testing seems to show that this second push takes 100 seconds to complete.

Is my expectation unreasonable?
Can a delta upload feature be added easily?

Tested against 0.0.8rc2
$ cat hello_world > a file
$ drive push a_file
(pushes)
$ drive push a_file
(everythign is up to date

However, do the same with a Jpg in a different directory

$ drive push head.jpg
(pushes)
$ drive push head.jpg
(pushes again)

UPDATE
multple copies of the folder structure and file have been created in the drive

$ drive push Pictures

I get a crash when I try to push a directory that has a large number of files:

http://paste2.org/7e2OyDxZ

number of files (8053):

$ find Pictures/ -type f |wc
   8053   60964  750553

number of dirs (124):

$ find Pictures/ -type d |wc
    124    1014    9827

Allowing multipart ie chunking files for faster uploading.

To address racing conditions if remote is being modified while we're trying to update the file. Google Drive provides resource versioning with ETags, use Etags to avoid racy cases.

1. I did a previous initial push using rakyll's drive. I do a push on the same dir using your drive and it seems to want to do the same initial push (I can't tell because there are a lot of files and I can't scroll back ijn history).
2. As well, if I do a pull on a remote dir (previously using rakyll) to check that all files are synced and your drive tells me that it should be pulled using the 'M' flag where this file hasn't been changed in months locally and/or remotely. I understand the 'Last Modified' date on google drive will be when the rakyll push was pushed, but this should not be the case for an initial push as per comments on rakyll#50. This should be the same as cp -p when copying files.
3. I have a remote dir which exists but doing pull commands keep asking for the same files over and over. I'm not sure why as the files havent changed and I'm not accessing them in any way.

I'm interested in verifying the integrity of a file on my Google Drive. Is there some way to ask the server to return a hash (sha256, md5 or whatever)? If not, can this feature be added to drive?

I see that drive stat shows a FileId. Can I somehow use that to verify the integrity of the remote file?

Support recursive diffing.

I was using the list command to test duplicate file issues and with over 1000 files involved the pagination became a pain. Is there something like a -no-more option so if the output is redirected to a file then the list is simply sent all at once.

Some examples of using pretty printing options would be educational as well.

Regards CB

Implement trash as requested in rakyll#58

I'm having trouble sending my backup files from my CentOS 6 VPS into Google Drive.

The smaller files seem to be uploading just fine but when it comes to the larger WAV files I'm getting "Error 401: Invalid Credentials, authError". It's also taking an extremely long time to process the WAV files (hours for a 120mb file).

Here's a screen shot of drive which has been running for most of the day:

I'm not doing anything special for api access that I'm aware of. I started with drive init which then gave me a url to visit to get a code etc.

[root@infuser conf.d]# uname -a
Linux infuser.crystaldreamsmusic.net 2.6.32-042stab094.7 #1 SMP Wed Oct 22 12:43:21 MSK 2014 x86_64 x86_64 x86_64 GNU/Linux
[root@infuser conf.d]# go version
go version go1.3.3 linux/amd64
[root@infuser conf.d]# 

Any thoughts would be greatly appreciated.

Thanks, Joel