A simple Class for uploading and downloading files and handling file versions from Google drive
It was made essentially to make data sharing easier between Google Colaboratory and Google Drive so I can make sure my model is saved properly after a few hours of training even if the process is being killed for inactivity.
!pip install - U - q PyDrive
Gd = Google_drive()
Call main class
Gd.upload_version(local_file, drive_dir_ID)
This method uploads the file called by os.path.basename(local_file) available inside the local directory path local_file to the Google Drive directory provided by its ID drive_dir_ID*. Basename of local_file must be the same as used in load_version().
Gd.load_version(local_file, drive_dir_ID, specific_version=None)
This method import file called by os.path.basename(local_file) (if possible) available inside the Google Drive directory provided by its ID drive_dir_ID* and save it under the path local_file. Basename of local_file must be the same as used in upload_version().
If specific_version is None, default version is the highest version.
Gd.print_all_versions(filename, drive_dir_ID)
This method prints all current versions of a file called filename available inside the Google Drive directory provided by its ID drive_dir_ID*.
Gd.load_all(local_dir, drive_dir_ID, force=False)
This method import all files available inside the Google Drive directory provided by its ID drive_dir_ID* to the local directory path. Also, local_dir. force allows overwriting. The Google Drive directory must exclusively gather files.
Gd.load_file(local_dir, file_ID)
This method import file provided by its ID file_ID* into the the local directory path local_dir.
Gd.upload_file(local_path, drive_dir_ID)
This method upload a file available inside the local path local_file into the Google Drive directory provided by its ID drive_dir_ID*.
* The shared link of google directory/file provided by its ID must be activated.
loadURL(local_dir, URL)
This method download a file from arbitrary URL inside the local directory path local_dir. If the file is zipped, it will unzip it (based on URL basename).
!pip install - U - q PyDrive
import os
os.chdir('/content/')
!git clone https://github.com/ruelj2/Google_drive.git
from Google_drive.handle import *
Gd = Google_drive()
#This is a Google directory link: https://drive.google.com/drive/folders/1TcMSKVAgRXZXymxfBv3WouPkauSIIpDX
#This is its ID: '1TcMSKVAgRXZXymxfBv3WouPkauSIIpDX'
Gd.load_all('/content/projet/dataset', '1TcMSKVAgRXZXymxfBv3WouPkauSIIpDX')
local_file = 'path/to/local/file'
drive_dir_ID = '1TcMSKVAgRXZXymxfBv3WouPkauSIIpDX'
Gd.load_version(local_file, drive_dir_ID)
'''
Loading Model from local file;
Training Model;
Saving Model to local file.
'''
Gd.upload_version(local_file, drive_dir_ID)
For a local machine, colabtools repo must be installed
$ git clone https://github.com/googlecolab/colabtools.git
$ cd colabtools
$ python setup.py install
To learn more: https://stackoverflow.com/questions/50194637/colaboratory-how-to-install-and-use-on-local-machine