Giter VIP home page Giter VIP logo

go-fswatch's Introduction

fswatch

fswatch is a go library for recursively watching file system changes to does not depend on inotify and therefore is not limit by the ulimit of your operating system.

Motivation

Why not use inotify? Even though there are great libraries like fsnotify that offer cross platform file system change notifications - the approach breaks when you want to watch a lot of files or folder.

For example the default ulimit for Mac OS is set to 512. If you need to watch more files you have to increase the ulimit for open files per process. And this sucks.

Usage

Watching a single file

If you want to watch a single file use the NewFileWatcher function to create a new file watcher:

go func() {
	checkIntervalInSeconds := 2
	fileWatcher := fswatch.NewFileWatcher("Some-file", checkIntervalInSeconds).Start()

	for fileWatcher.IsRunning() {

		select {
		case <-fileWatcher.Modified:

			go func() {
				// file changed. do something.
			}()

		case <-fileWatcher.Moved:

			go func() {
				// file moved. do something.
			}()
		}

	}
}()

Watching a folder

To watch a whole folder (and all of its child directories) for new, modified or deleted files you can use the NewFolderWatcher function.

Parameters:

  1. The path to the directory you want to monitor
  2. A flag indicating whether the folder shall be watched recursively
  3. An expression which decides which files are skipped
  4. The check interval in seconds (1 - n seconds)
go func() {

	recurse := true // include all sub directories

	skipDotFilesAndFolders := func(path string) bool {
		return strings.HasPrefix(filepath.Base(path), ".")
	}

	checkIntervalInSeconds := 2

	folderWatcher := fswatch.NewFolderWatcher(
		"some-directory",
		recurse,
		skipDotFilesAndFolders,
		checkIntervalInSeconds
	)
	
	folderWatcher.Start()

	for folderWatcher.IsRunning() {

		select {

		case <-folderWatcher.Modified():
			fmt.Println("New or modified items detected")

		case <-folderWatcher.Moved():
			fmt.Println("Items have been moved")

		case changes := <-folderWatcher.ChangeDetails():

			fmt.Printf("%s\n", changes.String())
			fmt.Printf("New: %#v\n", changes.New())
			fmt.Printf("Modified: %#v\n", changes.Modified())
			fmt.Printf("Moved: %#v\n", changes.Moved())

		}
	}

}()

go-fswatch in action

You can see go-fswatch in action in the live-reload feature of my markdown webserver "allmark".

See: github.com/andreaskoch/allmark/blob/master/src/allmark.io/modules/dataaccess/filesystem/watcher.go

I would still prefer using inotify, but go-fswatch has been doing it's job in allmark pretty well and works easily with relatively large folder structures.

Build Status

Build Status

Contribute

All contributions are welcome. Especially if you have an idea

  • how to reliably increase the limit for the maximum number of open files from within an application so we can use inotify for large folder structures.
  • how to overcome the limitations of inotify without having to resort to checking the files for changes over and over again
  • or how to make the existing code more efficient

please send me a message or a pull request.

go-fswatch's People

Contributors

andreaskoch avatar bradwright avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

go-fswatch's Issues

Insecure increment of numberOfFileWatchers & numberOfFolderWatchers++

Hi the increments should be surrounded with an Mutex to synchronize the write access to the variable. Because of concurrent go routines.

go-fswatch/file.go:
87
88 // increment watcher count
89: numberOfFileWatchers++
90
91 var modTime time.Time

go-fswatch/folder.go:
119
120 // increment watcher count
121: numberOfFolderWatchers++
122
123 for folderWatcher.wasStopped == false {

FolderWatcher uses checkIntervalInSeconds as nanoseconds

Maybe I'm just dumb, but my debugger brought to my attention that the checkInterval that I'm using (assumed to tbe 1s) is in reality 1ns.
Checking the code, I found that the int parameter checkIntervalInSeconds in NewFolderWatcher is used unchanged to initalize a time.Duration here. This leads to a duration in nanoseconds.
Same issue also applies to NewFileWatcher here

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.