Giter VIP home page Giter VIP logo

fsevents's Introduction

fsnotify is a Go library to provide cross-platform filesystem notifications on Windows, Linux, macOS, BSD, and illumos.

Go 1.17 or newer is required; the full documentation is at https://pkg.go.dev/github.com/fsnotify/fsnotify


Platform support:

Backend OS Status
inotify Linux Supported
kqueue BSD, macOS Supported
ReadDirectoryChangesW Windows Supported
FEN illumos Supported
fanotify Linux 5.9+ Not yet
AHAFS AIX aix branch; experimental due to lack of maintainer and test environment
FSEvents macOS Needs support in x/sys/unix
USN Journals Windows Needs support in x/sys/windows
Polling All Not yet

Linux and illumos should include Android and Solaris, but these are currently untested.

Usage

A basic example:

package main

import (
    "log"

    "github.com/fsnotify/fsnotify"
)

func main() {
    // Create new watcher.
    watcher, err := fsnotify.NewWatcher()
    if err != nil {
        log.Fatal(err)
    }
    defer watcher.Close()

    // Start listening for events.
    go func() {
        for {
            select {
            case event, ok := <-watcher.Events:
                if !ok {
                    return
                }
                log.Println("event:", event)
                if event.Has(fsnotify.Write) {
                    log.Println("modified file:", event.Name)
                }
            case err, ok := <-watcher.Errors:
                if !ok {
                    return
                }
                log.Println("error:", err)
            }
        }
    }()

    // Add a path.
    err = watcher.Add("/tmp")
    if err != nil {
        log.Fatal(err)
    }

    // Block main goroutine forever.
    <-make(chan struct{})
}

Some more examples can be found in cmd/fsnotify, which can be run with:

% go run ./cmd/fsnotify

Further detailed documentation can be found in godoc: https://pkg.go.dev/github.com/fsnotify/fsnotify

FAQ

Will a file still be watched when it's moved to another directory?

No, not unless you are watching the location it was moved to.

Are subdirectories watched?

No, you must add watches for any directory you want to watch (a recursive watcher is on the roadmap: #18).

Do I have to watch the Error and Event channels in a goroutine?

Yes. You can read both channels in the same goroutine using select (you don't need a separate goroutine for both channels; see the example).

Why don't notifications work with NFS, SMB, FUSE, /proc, or /sys?

fsnotify requires support from underlying OS to work. The current NFS and SMB protocols does not provide network level support for file notifications, and neither do the /proc and /sys virtual filesystems.

This could be fixed with a polling watcher (#9), but it's not yet implemented.

Why do I get many Chmod events?

Some programs may generate a lot of attribute changes; for example Spotlight on macOS, anti-virus programs, backup applications, and some others are known to do this. As a rule, it's typically best to ignore Chmod events. They're often not useful, and tend to cause problems.

Spotlight indexing on macOS can result in multiple events (see #15). A temporary workaround is to add your folder(s) to the Spotlight Privacy settings until we have a native FSEvents implementation (see #11).

Watching a file doesn't work well

Watching individual files (rather than directories) is generally not recommended as many programs (especially editors) update files atomically: it will write to a temporary file which is then moved to to destination, overwriting the original (or some variant thereof). The watcher on the original file is now lost, as that no longer exists.

The upshot of this is that a power failure or crash won't leave a half-written file.

Watch the parent directory and use Event.Name to filter out files you're not interested in. There is an example of this in cmd/fsnotify/file.go.

Platform-specific notes

Linux

When a file is removed a REMOVE event won't be emitted until all file descriptors are closed; it will emit a CHMOD instead:

fp := os.Open("file")
os.Remove("file")        // CHMOD
fp.Close()               // REMOVE

This is the event that inotify sends, so not much can be changed about this.

The fs.inotify.max_user_watches sysctl variable specifies the upper limit for the number of watches per user, and fs.inotify.max_user_instances specifies the maximum number of inotify instances per user. Every Watcher you create is an "instance", and every path you add is a "watch".

These are also exposed in /proc as /proc/sys/fs/inotify/max_user_watches and /proc/sys/fs/inotify/max_user_instances

To increase them you can use sysctl or write the value to proc file:

# The default values on Linux 5.18
sysctl fs.inotify.max_user_watches=124983
sysctl fs.inotify.max_user_instances=128

To make the changes persist on reboot edit /etc/sysctl.conf or /usr/lib/sysctl.d/50-default.conf (details differ per Linux distro; check your distro's documentation):

fs.inotify.max_user_watches=124983
fs.inotify.max_user_instances=128

Reaching the limit will result in a "no space left on device" or "too many open files" error.

kqueue (macOS, all BSD systems)

kqueue requires opening a file descriptor for every file that's being watched; so if you're watching a directory with five files then that's six file descriptors. You will run in to your system's "max open files" limit faster on these platforms.

The sysctl variables kern.maxfiles and kern.maxfilesperproc can be used to control the maximum number of open files.

fsevents's People

Contributors

adriansr avatar andrew-stripe avatar arp242 avatar drewwells avatar emanuel-skrenkovic avatar lox avatar nathany avatar pbnjay avatar rogpeppe avatar salbertson avatar samjacobson avatar shogo82148 avatar wadey avatar zchee avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

fsevents's Issues

Support build on darwin_arm64 with cgo

@nathany I'm trying to use fsevents as a dependency of a go binary that I want to build for darwin_arm64 with cgo from a darwin_amd64 host. However I got the following

external/go_sdk/pkg/tool/darwin_amd64/link: running external/local_config_cc/cc_wrapper.sh failed: exit status 134
ld: warning: ignoring file /var/folders/bz/z6hcm0ys1c922kw4v137vh9c0000gn/T/go-link-3522633666/000005.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
ld: warning: ignoring file /var/folders/bz/z6hcm0ys1c922kw4v137vh9c0000gn/T/go-link-3522633666/000006.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
ld: warning: ignoring file /var/folders/bz/z6hcm0ys1c922kw4v137vh9c0000gn/T/go-link-3522633666/000007.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
Undefined symbols for architecture arm64:
  "__cgo_45b9ffab6b40_Cfunc_ArrayCreateMutable", referenced from:
      _github.com/fsnotify/fsevents._cgo_45b9ffab6b40_Cfunc_ArrayCreateMutable in go.o
  "__cgo_45b9ffab6b40_Cfunc_CFArrayAppendValue", referenced from:
      _github.com/fsnotify/fsevents._cgo_45b9ffab6b40_Cfunc_CFArrayAppendValue in go.o
  "__cgo_45b9ffab6b40_Cfunc_CFRelease", referenced from:
      _github.com/fsnotify/fsevents._cgo_45b9ffab6b40_Cfunc_CFRelease in go.o
  "__cgo_45b9ffab6b40_Cfunc_CFRetain", referenced from:
      _github.com/fsnotify/fsevents._cgo_45b9ffab6b40_Cfunc_CFRetain in go.o
  "__cgo_45b9ffab6b40_Cfunc_CFRunLoopGetCurrent", referenced from:
      _github.com/fsnotify/fsevents._cgo_45b9ffab6b40_Cfunc_CFRunLoopGetCurrent in go.o
  "__cgo_45b9ffab6b40_Cfunc_CFRunLoopRun", referenced from:
      _github.com/fsnotify/fsevents._cgo_45b9ffab6b40_Cfunc_CFRunLoopRun in go.o
  "__cgo_45b9ffab6b40_Cfunc_CFRunLoopStop", referenced from:
      _github.com/fsnotify/fsevents._cgo_45b9ffab6b40_Cfunc_CFRunLoopStop in go.o
  "__cgo_45b9ffab6b40_Cfunc_CFStringCreateCopy", referenced from:
      _github.com/fsnotify/fsevents._cgo_45b9ffab6b40_Cfunc_CFStringCreateCopy in go.o
  "__cgo_45b9ffab6b40_Cfunc_CFStringCreateWithCString", referenced from:
      _github.com/fsnotify/fsevents._cgo_45b9ffab6b40_Cfunc_CFStringCreateWithCString in go.o
  "__cgo_45b9ffab6b40_Cfunc_CFStringGetBytes", referenced from:
      _github.com/fsnotify/fsevents._cgo_45b9ffab6b40_Cfunc_CFStringGetBytes in go.o
  "__cgo_45b9ffab6b40_Cfunc_CFStringGetLength", referenced from:
      _github.com/fsnotify/fsevents._cgo_45b9ffab6b40_Cfunc_CFStringGetLength in go.o
  "__cgo_45b9ffab6b40_Cfunc_CFUUIDCreateString", referenced from:
      _github.com/fsnotify/fsevents._cgo_45b9ffab6b40_Cfunc_CFUUIDCreateString in go.o
  "__cgo_45b9ffab6b40_Cfunc_EventStreamCreate", referenced from:
      _github.com/fsnotify/fsevents._cgo_45b9ffab6b40_Cfunc_EventStreamCreate in go.o
  "__cgo_45b9ffab6b40_Cfunc_EventStreamCreateRelativeToDevice", referenced from:
      _github.com/fsnotify/fsevents._cgo_45b9ffab6b40_Cfunc_EventStreamCreateRelativeToDevice in go.o
  "__cgo_45b9ffab6b40_Cfunc_FSEventStreamFlushAsync", referenced from:
      _github.com/fsnotify/fsevents._cgo_45b9ffab6b40_Cfunc_FSEventStreamFlushAsync in go.o
  "__cgo_45b9ffab6b40_Cfunc_FSEventStreamFlushSync", referenced from:
      _github.com/fsnotify/fsevents._cgo_45b9ffab6b40_Cfunc_FSEventStreamFlushSync in go.o
  "__cgo_45b9ffab6b40_Cfunc_FSEventStreamInvalidate", referenced from:
      _github.com/fsnotify/fsevents._cgo_45b9ffab6b40_Cfunc_FSEventStreamInvalidate in go.o
  "__cgo_45b9ffab6b40_Cfunc_FSEventStreamRelease", referenced from:
      _github.com/fsnotify/fsevents._cgo_45b9ffab6b40_Cfunc_FSEventStreamRelease in go.o
  "__cgo_45b9ffab6b40_Cfunc_FSEventStreamScheduleWithRunLoop", referenced from:
      _github.com/fsnotify/fsevents._cgo_45b9ffab6b40_Cfunc_FSEventStreamScheduleWithRunLoop in go.o
  "__cgo_45b9ffab6b40_Cfunc_FSEventStreamStart", referenced from:
      _github.com/fsnotify/fsevents._cgo_45b9ffab6b40_Cfunc_FSEventStreamStart in go.o
  "__cgo_45b9ffab6b40_Cfunc_FSEventStreamStop", referenced from:
      _github.com/fsnotify/fsevents._cgo_45b9ffab6b40_Cfunc_FSEventStreamStop in go.o
  "__cgo_45b9ffab6b40_Cfunc_FSEventsCopyUUIDForDevice", referenced from:
      _github.com/fsnotify/fsevents._cgo_45b9ffab6b40_Cfunc_FSEventsCopyUUIDForDevice in go.o
  "__cgo_45b9ffab6b40_Cfunc__Cmalloc", referenced from:
      _github.com/fsnotify/fsevents._cgo_45b9ffab6b40_Cfunc__Cmalloc in go.o
  "__cgo_45b9ffab6b40_Cfunc_free", referenced from:
      _github.com/fsnotify/fsevents._cgo_45b9ffab6b40_Cfunc_free in go.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Error in child process '/usr/bin/xcrun'. 1
external/local_config_cc/cc_wrapper.sh: line 69: 69609 Abort trap: 6           "$(/usr/bin/dirname "$0")"/wrapped_clang "$@"

I think this is coming from https://github.com/fsnotify/fsevents/blob/master/wrap.go#L6

Could you please give an help here to fix the problem?

For reference what I'm trying to build is https://github.com/bazelbuild/bazel-watcher which depends on fsnotify/fsevents

Data race detected

Using the latest fsevents, 3ceee05

Which version of macOS are you using?

ProductName:	Mac OS X
ProductVersion:	10.12.6
BuildVersion:	16G29

Please describe the issue that occurred.

When running a test with -race enabled, a data race shows up:

==================
WARNING: DATA RACE
Read at 0x00c4200a2130 by goroutine 8:
  github.com/fsnotify/fsevents.fsevtCallback()
      /Users/lachlan/go/src/github.com/fsnotify/fsevents/wrap.go:72 +0x2d8
  github.com/fsnotify/fsevents._cgoexpwrap_56954eb97524_fsevtCallback()
      github.com/fsnotify/fsevents/_obj/_cgo_gotypes.go:588 +0x6a
  runtime.call64()
      /usr/local/opt/go/libexec/src/runtime/asm_amd64.s:510 +0x3a
  github.com/fsnotify/fsevents.(*EventStream).start.func1()
      /Users/lachlan/go/src/github.com/fsnotify/fsevents/wrap.go:235 +0x111

Previous write at 0x00c4200a2130 by goroutine 7:
  github.com/fsnotify/fsevents.(*EventStream).Start()
      /Users/lachlan/go/src/github.com/fsnotify/fsevents/fsevents.go:161 +0x18d
  github.com/lox/binarystar/filetree.TestWatchOnMacos()
      /Users/lachlan/go/src/github.com/lox/binarystar/filetree/watch_darwin_test.go:31 +0x324
  testing.tRunner()
      /usr/local/opt/go/libexec/src/testing/testing.go:746 +0x16c

Goroutine 8 (running) created at:
  github.com/fsnotify/fsevents.(*EventStream).start()
      /Users/lachlan/go/src/github.com/fsnotify/fsevents/wrap.go:229 +0x16c
  github.com/fsnotify/fsevents.(*EventStream).Start()
      /Users/lachlan/go/src/github.com/fsnotify/fsevents/fsevents.go:159 +0x122
  github.com/lox/binarystar/filetree.TestWatchOnMacos()
      /Users/lachlan/go/src/github.com/lox/binarystar/filetree/watch_darwin_test.go:31 +0x324
  testing.tRunner()
      /usr/local/opt/go/libexec/src/testing/testing.go:746 +0x16c

Goroutine 7 (finished) created at:
  testing.(*T).Run()
      /usr/local/opt/go/libexec/src/testing/testing.go:789 +0x568
  testing.runTests.func1()
      /usr/local/opt/go/libexec/src/testing/testing.go:1004 +0xa7
  testing.tRunner()
      /usr/local/opt/go/libexec/src/testing/testing.go:746 +0x16c
  testing.runTests()
      /usr/local/opt/go/libexec/src/testing/testing.go:1002 +0x521
  testing.(*M).Run()
      /usr/local/opt/go/libexec/src/testing/testing.go:921 +0x206
  main.main()
      github.com/lox/binarystar/filetree/_test/_testmain.go:56 +0x1d3
==================
Found 1 data race(s)

Are you able to reproduce the issue? Please provide steps to reproduce and a code sample if possible.

// +build darwin

package filetree

import (
	"io/ioutil"
	"testing"
	"time"

	"github.com/fsnotify/fsevents"
)

func TestWatchOnMacos(t *testing.T) {
	path, err := ioutil.TempDir("", "fsexample")
	if err != nil {
		t.Fatal(err)
	}

	dev, err := fsevents.DeviceForPath(path)
	if err != nil {
		t.Fatal(err)
	}

	es := &fsevents.EventStream{
		Paths:   []string{path},
		Latency: 500 * time.Millisecond,
		Device:  dev,
		Flags:   fsevents.FileEvents,
	}

	es.Start()
}

wrap.go ignores failure return value from FSEventStreamStart

While investigating #36, I noticed that this package ignores the failure return value from FSEventStreamStart

https://developer.apple.com/documentation/coreservices/1448000-fseventstreamstart

Return Value

True if it succeeds, otherwise False if it fails. It ought to always succeed, but in the event it does not then your code should fall back to performing recursive scans of the directories of interest as appropriate.

The linked bug is an example of a case where it returns false. This failure should be propagated to the caller somehow.

Error vendor/github.com/fsnotify/fsevents/wrap.go:38: constant 36893488147419103231 overflows uint64

Before reporting an issue, please ensure you are using the latest release of fsnotify.

Which version of macOS are you using?

ProductName: Mac OS X
ProductVersion: 10.13.1
BuildVersion: 17B48

Please describe the issue that occurred.

While trying to run or buuld the example/main.go (copied to it´s own file) I get

vendor/github.com/fsnotify/fsevents/wrap.go:38: constant 36893488147419103231 overflows uint64

Are you able to reproduce the issue? Please provide steps to reproduce and a code sample if possible.

  1. mkdir test
  2. cd test
  3. make a file main.go with content of example/main.go
  4. dep init
  5. go run main.go

Can't watch multiple paths in fsevents

Which version of macOS are you using?

ProductName:	Mac OS X
ProductVersion:	10.13.4
BuildVersion:	17E199

Please describe the issue that occurred.

When there are multiple paths in the Path slice in the EventStream struct calling Start() causes an error to be printed from fsevents itself:

2018-04-30 13:55 main[37509] (FSEvents.framework) FSEventStreamStart: register_with_server: ERROR: f2d_register_rpc() => (null) (-21)

Are you able to reproduce the issue? Please provide steps to reproduce and a code sample if possible.

Here's a version of example/main.go which illustrates this problem:

// +build darwin

package main

import (
	"bufio"
	"fmt"
	"io/ioutil"
	"log"
	"os"
	"runtime"
	"time"

	"github.com/fsnotify/fsevents"
)

func main() {
	path, err := ioutil.TempDir("", "fsexample")
	if err != nil {
		log.Fatalf("Failed to create TempDir: %v", err)
	}
	fmt.Println(path)
	path2, err := ioutil.TempDir("", "fsexample2")
	if err != nil {
		log.Fatalf("Failed to create TempDir: %v", err)
	}
	fmt.Println(path2)
	dev, err := fsevents.DeviceForPath(path)
	if err != nil {
		log.Fatalf("Failed to retrieve device for path: %v", err)
	}
	log.Print(dev)
	log.Println(fsevents.EventIDForDeviceBeforeTime(dev, time.Now()))

	es := &fsevents.EventStream{
		Paths:   []string{path, path2},
		Latency: 500 * time.Millisecond,
		Device:  dev,
		Flags:   fsevents.FileEvents | fsevents.WatchRoot}
	es.Start()
	ec := es.Events

	log.Println("Device UUID", fsevents.GetDeviceUUID(dev))

	go func() {
		for msg := range ec {
			for _, event := range msg {
				logEvent(event)
			}
		}
	}()

	in := bufio.NewReader(os.Stdin)

	if false {
		log.Print("Started, press enter to GC")
		in.ReadString('\n')
		runtime.GC()
		log.Print("GC'd, press enter to quit")
		in.ReadString('\n')
	} else {
		log.Print("Started, press enter to stop")
		in.ReadString('\n')
		es.Stop()

		log.Print("Stopped, press enter to restart")
		in.ReadString('\n')
		es.Resume = true
		es.Start()

		log.Print("Restarted, press enter to quit")
		in.ReadString('\n')
		es.Stop()
	}
}

var noteDescription = map[fsevents.EventFlags]string{
	fsevents.MustScanSubDirs: "MustScanSubdirs",
	fsevents.UserDropped:     "UserDropped",
	fsevents.KernelDropped:   "KernelDropped",
	fsevents.EventIDsWrapped: "EventIDsWrapped",
	fsevents.HistoryDone:     "HistoryDone",
	fsevents.RootChanged:     "RootChanged",
	fsevents.Mount:           "Mount",
	fsevents.Unmount:         "Unmount",

	fsevents.ItemCreated:       "Created",
	fsevents.ItemRemoved:       "Removed",
	fsevents.ItemInodeMetaMod:  "InodeMetaMod",
	fsevents.ItemRenamed:       "Renamed",
	fsevents.ItemModified:      "Modified",
	fsevents.ItemFinderInfoMod: "FinderInfoMod",
	fsevents.ItemChangeOwner:   "ChangeOwner",
	fsevents.ItemXattrMod:      "XAttrMod",
	fsevents.ItemIsFile:        "IsFile",
	fsevents.ItemIsDir:         "IsDir",
	fsevents.ItemIsSymlink:     "IsSymLink",
}

func logEvent(event fsevents.Event) {
	note := ""
	for bit, description := range noteDescription {
		if event.Flags&bit == bit {
			note += description + " "
		}
	}
	log.Printf("EventID: %d Path: %s Flags: %s", event.ID, event.Path, note)
}

Running this results in:

[~/go/src/github.com/fsnotify/fsevents/example]> ./main
/var/folders/6p/7hwgf0d92_3bckf61qbwz1n80000gp/T/fsexample985416144
/var/folders/6p/7hwgf0d92_3bckf61qbwz1n80000gp/T/fsexample2184687343
2018/04/30 14:03:40 16777224
2018/04/30 14:03:40 69444979
2018-04-30 14:03 main[38203] (FSEvents.framework) FSEventStreamStart: register_with_server: ERROR: f2d_register_rpc() => (null) (-21)
2018/04/30 14:03:40 Device UUID 32F97B79-2B30-4389-A5C1-C2586D6EB0E6
2018/04/30 14:03:40 Started, press enter to stop

2018/04/30 14:03:41 Stopped, press enter to restart

2018-04-30 14:03 main[38203] (FSEvents.framework) FSEventStreamStart: register_with_server: ERROR: f2d_register_rpc() => (null) (-21)
2018/04/30 14:03:42 Restarted, press enter to quit

Any events produced in either of these directories are not reported like they are if you're only watching one directory.

I'm running this on an iMac with a fusion drive, which is the only machine-specific thing I can think of which is causing this. I imagine though that a fusion drive appears as one device as far as FSEvents is concerned?

I'm also unsure which path you are supposed to use to look up the device.

next steps

It looks like the next steps here are:

  • review, revise, and merge #18 (thanks @drewwells for all the work on testing)
  • some project stuff like Travis, Codecov, and license stuff -- happy to take that on
  • it wouldn't hurt to take a peek at https://github.com/rjeczalik/notify or any other FSEvents implementations in Go (e.g. convox/rack#1252) just to see if there is anything to learn from them
  • I'd really like to write a consumer of FSEvents to get a better feel for it in action. Anyone else using the fsevents package in a project?
  • review the APIs, documentation, and remove the "unstable" disclaimer from the README

Broken FSEvents for symlink paths on macOS 10.15

Which version of macOS are you using?

ProductName:	Mac OS X
ProductVersion:	10.15.1
BuildVersion:	19B88

Please describe the issue that occurred.

FSEvents for symlinks on macOS 10.15 aren't being sent or aren't being captured by fsnotify/fsevents.

Are you able to reproduce the issue? Please provide steps to reproduce and a code sample if possible.

Current tests fail on macOS 10.15 because ioutil.TempDir("", "fsexample") creates a file in /var/tmp/... which is a symlink to /private/var/tmp on macOS. I refactored fsevents_test.go in my fork (where I can run test on 10.15 via CircleCI) to show that the failing scenario is the one involving symlinks.

Workaround

The workaround is to use filepath.EvalSymlinks to get the "real" path then use that real path in the construction of your EventStream.

Go 1.10 support

Before reporting an issue, please ensure you are using the latest release of fsnotify.

Which version of macOS are you using?

-> % sw_vers
ProductName:	Mac OS X
ProductVersion:	10.13
BuildVersion:	17A365

Please describe the issue that occurred.

building on Go 1.10 introduces build errors

Are you able to reproduce the issue? Please provide steps to reproduce and a code sample if possible.

-> % GOROOT=/usr/local/go110 /usr/local/go110/bin/go test
# github.com/fsnotify/fsevents
./wrap.go:38: constant 36893488147419103231 overflows uint64
./wrap.go:80: cannot use f (type FSEventStreamRef) as type _Ctype_ConstFSEventStreamRef in argument to func literal
./wrap.go:85: cannot use f (type FSEventStreamRef) as type _Ctype_ConstFSEventStreamRef in argument to func literal
./wrap.go:91: cannot use f (type FSEventStreamRef) as type _Ctype_ConstFSEventStreamRef in argument to func literal
./wrap.go:97: cannot use f (type FSEventStreamRef) as type _Ctype_ConstFSEventStreamRef in argument to func literal
./wrap.go:112: cannot convert nil to type _Ctype_CFUUIDRef
./wrap.go:115: cannot use nil as type _Ctype_CFAllocatorRef in argument to func literal
./wrap.go:119: cannot convert nil to type _Ctype_CFStringRef
./wrap.go:122: cannot use nil as type _Ctype_CFAllocatorRef in argument to func literal
./wrap.go:175: cannot use nil as type _Ctype_CFAllocatorRef in argument to _Cfunc_CFStringCreateWithCString
./wrap.go:175: too many errors

Go 1.10 is still in beta1. Based on the release notes, I can't see why the implicit conversion to C to _Ctype is failing now.

Filewatching fails silently with >4096 paths

Before reporting an issue, please ensure you are using the latest release of fsevents.

Which version of macOS are you using?

ProductName:	Mac OS X
ProductVersion:	10.15.4
BuildVersion:	19E287

Please describe the issue that occurred.

Filewatching with fsevents.EventStream works just fine for up to 4096 paths; from 4097 paths upwards, I don't seem to be getting any events (nor error messages).

Are you able to reproduce the issue? Please provide steps to reproduce and a code sample if possible.

Repro case here (a small Go util that watches files specified as args, plus repro instructions).

FWIW, I think this may be an underlying problem with fsevents (because I could reproduce the problem with e.g. this fswatch util), but I'm not sure where to report; would be happy for guidance!

integrating into fsnotify

Any plans to wrap these low functions into a fsnotify interface?

The biggest things (to me) is the lack of an event channel and simplifying the exposed flags to those exposed by fsnotify.

TODO

  • Integration tests
  • Refine API
  • EventFlags.String()

Compile warning: compile on ventura warns that 'FSEventStreamScheduleWithRunLoop' is deprecated

Which version of macOS are you using?

ProductName: macOS
ProductVersion: 13.2.1
BuildVersion: 22D68

Please describe the issue that occurred.

Compilation throws a warning:

# github.com/fsnotify/fsevents
cgo-gcc-prolog:452:2: warning: 'FSEventStreamScheduleWithRunLoop' is deprecated: first deprecated in macOS 13.0 - Use FSEventStreamSetDispatchQueue instead. [-Wdeprecated-declarations]
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/CoreServices.framework/Frameworks/FSEvents.framework/Headers/FSEvents.h:1138:1: note: 'FSEventStreamScheduleWithRunLoop' has been explicitly marked deprecated here

There is no functionality problem right now, as far as I see. I hope it does not mean a future update to MacOS will break it.

Are you able to reproduce the issue? Please provide steps to reproduce and a code sample if possible.

Yes, on a machine that runs Ventura (13.0 and up) any compilation of a project that contains fsevents v0.1.1 will throw the warning.

rsync: only REMOVE Event received

Hi!

With a very simple test (on OS X):

package main

import (
    "fmt"
    "strings"
    "gopkg.in/fsnotify.v1"
)


func main() {
    watcher, err := fsnotify.NewWatcher()
    if err != nil {
        fmt.Printf("fsnotify.NewWatcher() creates error=<%v>", err)
    }
    fileorfolder := "/tmp/watchme.dir"
    fmt.Printf("watching %s\n", fileorfolder)
    err = watcher.Add(fileorfolder)
    if err != nil {
        fmt.Printf("watcher.Watch subfolder=<%v> error=<%v>", fileorfolder, err)
    }

    loop:
    for {
        select {
        case ev := <-watcher.Events:
            fmt.Printf("event %s\n", ev)
            if strings.Contains(ev.Name, "Bye") {
                break loop
            }
        case err := <-watcher.Errors:
            fmt.Printf("error: " + err.Error())
            break loop
        }
    }
}

and using rsync (rsync -avh /tmp/rsync.test/watchme.dir /tmp) to "copy" files ("foo.txt") to the watched location, I get

event "/tmp/watchme.dir/foo.txt": REMOVE

but nothing else. I would have expected a MOVE or CREATE for the file. On Linux I get

event "/tmp/watchme.dir/.foo.txt.HM02fS": CREATE
event "/tmp/watchme.dir/.foo.txt.HM02fS": CHMOD
event "/tmp/watchme.dir/.foo.txt.HM02fS": RENAME
event "/tmp/watchme.dir/foo.txt": CREATE

Get Started

Refer here: howeyc/fsnotify#54

I believe that we can begin with the code I originally contributed to fsnotify, and modify it to suit.

Failing test on Go 1.10+: cannot use nil as type...

Before reporting an issue, please ensure you are using the latest release of fsnotify.

Which version of macOS are you using?

ProductName: Mac OS X
ProductVersion: 10.13.6
BuildVersion: 17G2208

Please describe the issue that occurred.

go test fails in go1.11 and go1.10.4

❯ go test
# github.com/fsnotify/fsevents [github.com/fsnotify/fsevents.test]
./wrap.go:115:183: cannot use nil as type _Ctype_CFAllocatorRef in argument to func literal
./wrap.go:122:168: cannot use nil as type _Ctype_CFAllocatorRef in argument to func literal
./wrap.go:175:44: cannot use nil as type _Ctype_CFAllocatorRef in argument to _Cfunc_CFStringCreateWithCString
FAIL	github.com/fsnotify/fsevents [build failed]

tests pass with go1.9.7

Are you able to reproduce the issue? Please provide steps to reproduce and a code sample if possible.

yes, just run go test.

panic with Go 1.6

panic: runtime error: cgo argument has Go pointer to Go pointer

goroutine 1 [running]:
panic(0x4152820, 0xc82000a5b0)
/usr/local/go/src/runtime/panic.go:464 +0x3e6
github.com/fsnotify/fsevents.(_EventStream).Start(0xc82001c240)
/Users/dshills/Development/Go/src/github.com/fsnotify/fsevents/fsevents.go:213 +0x826
github.com/dshills/goauto.(_watchOSX).Start(0xc820016630, 0x11e1a300, 0xc820010520, 0x2, 0x2, 0x0, 0x0, 0x0)

After installing Go 1.6 all apps using fsevents panic on startup with this error.

drop support for go1.10 and before

There's a bunch of version-specific code in the repository which makes it awkward to read.
Go 1.10 is sufficiently old now that perhaps support for it can be dropped and thereby remove those build constraints.

Do not force absolute Paths

fsevents @ 1055680

Which version of macOS are you using?

ProductName:    Mac OS X
ProductVersion: 10.15.7
BuildVersion:   19H1217

Please describe the issue that occurred.

FSEvents api supports two different ways of starting an event stream:

  1. Uses a device id and function call FSEventStreamCreateRelativeToDevice which is expecting paths to be relative to the mount point of the device. This is called the per-disk event stream in the docs. The example/main.go uses this approach since it uses a non-zero device id. See the documentation here specifically about param pathsToWatchRelativeToDevice
pathsToWatchRelativeToDevice
A CFArray of CFStringRefs, each specifying a relative path to a directory on the device identified by the dev parameter. The paths should be relative to the root of the device. For example, if a volume "MyData" is mounted at "/Volumes/MyData" and you want to watch "/Volumes/MyData/Pictures/July", specify a path string of "Pictures/July". To watch the root of a volume pass a path of "" (the empty string).
  1. The alternative is with function FSEventStreamCreate which is expecting absolute paths. This is the "per-host" event stream constructor. fsevents uses this approach if the Device in the EventStream struct is zero. While the docs are ambiguous, on my system, the "per-host" approach only works with absolute paths.
pathsToWatch
A CFArray of CFStringRefs, each specifying a path to a directory, signifying the root of a filesystem hierarchy to be watched for modifications.

However, wrap.go/createPaths() transforms all paths in an EventStream struct to absolute paths at

fsevents/wrap.go

Lines 154 to 158 in f721bd2

func createPaths(paths []string) (C.CFArrayRef, error) {
cPaths := C.ArrayCreateMutable(C.int(len(paths)))
var errs []error
for _, path := range paths {
p, err := filepath.Abs(path)

This prevents using the "per-disk" mode for all volumes except the root volume at /.

Are you able to reproduce the issue? Please provide steps to reproduce and a code sample if possible.

Yes.

  1. Create a new Volume on OSX (I use APFS) mounted at /Volumes/fsevents-repo
  2. Use the example/main.go program with path /Volumes/fsevents-repo (the only change)
  3. go run ./example/main.go
  4. touch /Volumes/fsevents-repo/testfile and observe there is no event reports.
  5. Now, change wrap.go/createPaths to using
	for _, path := range paths {
		p := path
		cpath := C.CString(p)

and repeat the steps above.

PR coming shortly.

Cross-compiling code using fsevents

Which version of macOS are you using?

Github runner macos-latest (Apparently macOS 12 Monterey based on the documentation.

Please describe the issue that occurred.

In my build I'm cross compiling into various OSes and hardware platforms. I initially used the Ubuntu 12 runner to do the compilation, but that failed on macOS after some dependency introduced the use of fsevents.

I then added macos builders for the two build workflows for darwin/amd64 and darwin/arm64. This fails on darwin/arm64 as well.

Compiling it on my M1 machine works though.

Are you able to reproduce the issue? Please provide steps to reproduce and a code sample if possible.

Try to compile ccmanager to darwin/arm64 from any machine other than an arm mac.

Do you have any hints on how to succeed building the tool with Github actions? Thanks a lot!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.