iwoz / file_sync Goto Github PK

View Code? Open in Web Editor NEW

140.0 11.0 30.0 311 KB

Detect file modification in background and push it to git automatically.

License: MIT License

Python 100.00%

file_sync's Introduction

file_sync

Synchronize your files to a git repository automatically.

When you have modified your files, files will be pushed to a remote git repository.

With git, you can know every change with your files by commits.

Setup

Install git
Install Python

Install watchdog

in OS X or *nix:

 pip install watchdog

in Windows:

 python -m pip install watchdog

How to use

Make a new git repository for synchronize your files

I made a new git repository named configs in Github.

Setup your git repository locally

cd ~/Documents ## REPLACE ~/Documents TO WHATEVER DIR YOU LIKE

mkdir configs
cd configs

git init

## REPLACE [email protected]:iWoz/configs.git TO YOUR GIT REPOSITORY CREATED BEFORE
git remote add origin [email protected]:iWoz/configs.git 

## add submodule
git submodule add [email protected]:iWoz/file_sync.git

## stage all, commit and push to remote
git add -A
git commit -m "First commit."
git push -u origin master
## make sure you pushed successfully before you start the  next step

Setup your file_list.txt for files to synchronize

Make a new file name file_list.txt in root of your git repository.

For example, contents in my file_list.txt are :

/Users/Tim/.zshrc
/Users/Tim/Library/Application Support/Sublime Text 2/Packages/User/Default (OSX).sublime-keymap
/Users/Tim/Library/Application Support/Sublime Text 2/Packages/User/Preferences.sublime-settings

Make sure one line one file in its absolute path.

Test

Run file_sync.py in submodule file_sync

cd file_sync
python file_sync.py

Now modify one of your files mentioned in file_list.txt.

For example, I modified .zshrc, and then logs are:

And when I check the remote git repository, all worked ok:

Auto run when system startup

OS X

make a file named com.wuzhiwei.filesync.plist in ~/Library/LaunchAgents/

fill the plist with content below

 <?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE plist PUBLIC -//Apple Computer//DTD PLIST 1.0//EN http://www.apple.com/DTDs/PropertyList-1.0.dtd >
 <plist version="1.0">
   <dict>
     <key>Label</key>
     <string>com.wuzhiwei.filesync</string>
     <key>Program</key>
     <!-- REPLACE BELOW TO YOUR file_sync.py PATH -->
     <string>/Users/Tim/Documents/configs/file_sync/file_sync.py</string>
     <key>RunAtLoad</key>
     <true/>
     <key>KeepAlive</key>
     <true/>
     <key>UserName</key>
     <!-- REPLACE BELOW TO YOUR USERNAME -->
     <string>Tim</string>
     <key>StandardErrorPath</key>
     <!-- REPLACE BELOW TO WHAT EVER PATH YOU LIKE -->
     <string>/Users/Tim/Documents/configs/file_sync/file_sync_error.log</string>
     <key>StandardOutPath</key>
     <!-- REPLACE BELOW TO WHAT EVER PATH YOU LIKE -->
     <string>/Users/Tim/Documents/configs/file_sync/file_sync_output.log</string>
   </dict>
 </plist>

replace to your real path or username in lines below REPLACE BELOW TO ...
load the plist to system service: launchctl load ~/Library/LaunchAgents/com.wuzhiwei.filesync.plist
if you want to unload the autorun service: launchctl unload ~/Library/LaunchAgents/com.wuzhiwei.filesync.plist
check if the service is run in back: launchctl list | grep com.wuzhiwei.filesync

Windows

I rarely use Windows, there is an ugly way to do this:
- make a bat file named file_sync.bat
- write pythonw __YOUR_DIR__\file_sync.py in this bat
- put this bat to your startup folder C:\Documents and Settings\All Users\Start Menu\Programs\Startup
- close the console because we use pythonw
- more infomation can be found in here
*nix

I'm pretty sure you can make it happen by yourself when you use *nix :)

file_sync's People

Contributors

Stargazers

Watchers

file_sync's Issues

支持文件夹同步

感谢思路，改进为文件夹同步，设置为3s内最多同步一次

#!/usr/bin/python
# -*- coding: utf-8 -*-

from sqlite3 import Timestamp
import sys
import time
import ntpath
import os
import re
import platform

from subprocess import call
from shutil import copy
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
import threading

# git root path for files to push to remote
DIR_FOR_GIT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

# folders to synchronize
SYNC_FOLDER_LIST = []
f = open(os.path.join(DIR_FOR_GIT, "file_list.txt"), "r")
try:
    SYNC_FOLDER_LIST = [line.strip().replace('\\','/') for line in f] # if os.path.isfile(line.strip().replace('\\','/'))]
except Exception as e:
    raise e
finally:
    f.close()

class FileChangeHandler(FileSystemEventHandler):
    timer = None
    def on_any_event(self, event):
        if self.timer:
            self.timer.cancel()
        self.timer = threading.Timer(3, self.checkSnapshot) #延后
        self.last_event = event
        self.timer.start()
    def checkSnapshot(self):
        # print(self.last_event.event_type, self.last_event.src_path)
        src_path = self.last_event.src_path.replace('\\','/')
        for SYNC_PATH in SYNC_FOLDER_LIST:
            if SYNC_PATH in src_path and ".git" not in src_path: #排除.git
                os.chdir(SYNC_PATH)
                git_add_cmd = "git add -A"
                git_commit_cmd = "git commit -m " + re.escape("Update "+os.path.basename(src_path))
                if platform.system() == "Windows":
                    git_commit_cmd = "git commit -m Update."
                git_pull_cmd = "git pull origin master"
                git_push_cmd = "git push origin master"
                call(
                    git_add_cmd + "&&" +
                    git_commit_cmd + "&&" +
                    git_pull_cmd + "&&" +
                    git_push_cmd,
                    shell=True
                )
                break

if __name__ == "__main__":
    observer = Observer()
    event_handler = FileChangeHandler()

    for file_path in SYNC_FOLDER_LIST:
        observer.schedule(event_handler, path=os.path.dirname(os.path.realpath(file_path)), recursive=True)

    observer.start()

    try:
        while True:
            time.sleep(1)
    except KeyboardInterrupt:
        observer.stop()
    observer.join()

SameFileError 异常

Hi，iWoz,我在使用python file_sync的时候发现把config目录下的file_list.txt和README.md文件都加入到file_list.txt里面，在执行代码里面的copy方法复制的时候会抛出SameFileError,这种情况怎么解决呀?

H:\config\file_sync>python file_sync.py
Traceback (most recent call last):
  File "file_sync.py", line 54, in <module>
    copy(file_path, DIR_FOR_GIT)
  File "C:\Program Files\Anaconda3\lib\shutil.py", line 241, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "C:\Program Files\Anaconda3\lib\shutil.py", line 104, in copyfile
    raise SameFileError("{!r} and {!r} are the same file".format(src, dst))
shutil.SameFileError: 'H:/config/README.md' and 'H:\\config\\README.md' are the same file

H:\config\file_sync>python file_sync.py
Traceback (most recent call last):
  File "file_sync.py", line 54, in <module>
    copy(file_path, DIR_FOR_GIT)
  File "C:\Program Files\Anaconda3\lib\shutil.py", line 241, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "C:\Program Files\Anaconda3\lib\shutil.py", line 104, in copyfile
    raise SameFileError("{!r} and {!r} are the same file".format(src, dst))
shutil.SameFileError: 'H:/config/README.md' and

Run file_sync.py error

Hi.

@iWoz

can sync between two local computers without through github

Hi,
Is that possible I can sync between two local computers without go through github.
I have a home server, that all files backup, how can I push the diff to home server from other computer?

项目功能不是很适用

1.研究了下项目，运行环境需要Python和一个Python的库，对于开发人员来说还好，不是很难。
2.支持的文件更新同步需要全部配置在file_list.txt里面，这个局限性就很大了，很多时候希望同步的都是一个文件夹，而且文件夹里面增删改都会会有的。
3.自动同步和拉取通过脚本也是可以的。
该项目其实蛮有借鉴意义的，思路非常好。