Giter VIP home page Giter VIP logo

drive's Introduction

Google_drive

A simple Class for uploading and downloading files and handling file versions from Google drive

It was made essentially to make data sharing easier between Google Colaboratory and Google Drive so I can make sure my model is saved properly after a few hours of training even if the process is being killed for inactivity.

Requirements

!pip install - U - q PyDrive

methods

Gd = Google_drive()
Call main class

Gd.upload_version(local_file, drive_dir_ID)
This method uploads the file called by os.path.basename(local_file) available inside the local directory path local_file to the Google Drive directory provided by its ID drive_dir_ID*. Basename of local_file must be the same as used in load_version().

Gd.load_version(local_file, drive_dir_ID, specific_version=None)
This method import file called by os.path.basename(local_file) (if possible) available inside the Google Drive directory provided by its ID drive_dir_ID* and save it under the path local_file. Basename of local_file must be the same as used in upload_version(). If specific_version is None, default version is the highest version.

Gd.print_all_versions(filename, drive_dir_ID)
This method prints all current versions of a file called filename available inside the Google Drive directory provided by its ID drive_dir_ID*.

Gd.load_all(local_dir, drive_dir_ID, force=False)
This method import all files available inside the Google Drive directory provided by its ID drive_dir_ID* to the local directory path. Also, local_dir. force allows overwriting. The Google Drive directory must exclusively gather files.

Gd.load_file(local_dir, file_ID)
This method import file provided by its ID file_ID* into the the local directory path local_dir.

Gd.upload_file(local_path, drive_dir_ID)
This method upload a file available inside the local path local_file into the Google Drive directory provided by its ID drive_dir_ID*.

* The shared link of google directory/file provided by its ID must be activated.

featured function

loadURL(local_dir, URL)
This method download a file from arbitrary URL inside the local directory path local_dir. If the file is zipped, it will unzip it (based on URL basename).

Exemple for Google colaboratory

!pip install - U - q PyDrive  
import os  
os.chdir('/content/')  
!git clone https://github.com/ruelj2/Google_drive.git  
  
from Google_drive.handle import *
Gd = Google_drive()  
  
#This is a Google directory link: https://drive.google.com/drive/folders/1TcMSKVAgRXZXymxfBv3WouPkauSIIpDX  
#This is its ID: '1TcMSKVAgRXZXymxfBv3WouPkauSIIpDX'  
Gd.load_all('/content/projet/dataset', '1TcMSKVAgRXZXymxfBv3WouPkauSIIpDX')  
  
local_file = 'path/to/local/file'
drive_dir_ID = '1TcMSKVAgRXZXymxfBv3WouPkauSIIpDX'  
Gd.load_version(local_file, drive_dir_ID)  
  
'''
Loading Model from local file;
Training Model;
Saving Model to local file.
'''
  
Gd.upload_version(local_file, drive_dir_ID)  

Use on local machine:

For a local machine, colabtools repo must be installed

$ git clone https://github.com/googlecolab/colabtools.git  
$ cd colabtools  
$ python setup.py install  

To learn more: https://stackoverflow.com/questions/50194637/colaboratory-how-to-install-and-use-on-local-machine

drive's People

Contributors

jeanchristopheruel avatar

Watchers

 avatar

Forkers

dkr1818

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.