devatoria / go-mesos-executor Goto Github PK

View Code? Open in Web Editor NEW

28.0 28.0 5.0 2.14 MB

Custom Apache Mesos container executor written in Go

License: MIT License

Makefile 0.85% Go 99.15%

containers docker golang mesos mesos-executor

go-mesos-executor's People

Contributors

Stargazers

Watchers

Forkers

jbbernat winternis boskiv seoyhaein andreaspeters

go-mesos-executor's Issues

Improve iptables hook by creating separate chains

iptables hook is used to manage iptables rules needed for container to be joined from the outside when using bridge networking mode. Those rules are injected in the default chains of iptables. We should inject them into separate ones in order to improve isolation.

Add command health checks missing features

Environments
Arguments
User

Hook to disable container privilege

Avoid a container to be run in privilege mode

Add statuses support for HTTP health checks

Build errors and some hints needed

I'm trying to build go-mesos-executor, but i'm facing some fatal errors in go build:

./main.go:43:14: cannot use "go.uber.org/zap".String("executorID", executorID) (type "go.uber.org/zap/zapcore".Field) as type "github.com/Devatoria/go-mesos-executor/vendor/go.uber.org/zap/zapcore".Field in argument to logger.GetInstance().Info
./main.go:44:14: cannot use "go.uber.org/zap".String("frameworkID", frameworkID) (type "go.uber.org/zap/zapcore".Field) as type "github.com/Devatoria/go-mesos-executor/vendor/go.uber.org/zap/zapcore".Field in argument to logger.GetInstance().Info
./main.go:51:14: cannot use "go.uber.org/zap".Error(err) (type "go.uber.org/zap/zapcore".Field) as type "github.com/Devatoria/go-mesos-executor/vendor/go.uber.org/zap/zapcore".Field in argument to logger.GetInstance().Fatal
./main.go:58:15: cannot use "go.uber.org/zap".Reflect("hooks", hooks) (type "go.uber.org/zap/zapcore".Field) as type "github.com/Devatoria/go-mesos-executor/vendor/go.uber.org/zap/zapcore".Field in argument to logger.GetInstance().Info
./main.go:76:14: cannot use "go.uber.org/zap".Error(err) (type "go.uber.org/zap/zapcore".Field) as type "github.com/Devatoria/go-mesos-executor/vendor/go.uber.org/zap/zapcore".Field in argument to logger.GetInstance().Fatal
./main.go:137:13: cannot use "go.uber.org/zap".Error(err) (type "go.uber.org/zap/zapcore".Field) as type "github.com/Devatoria/go-mesos-executor/vendor/go.uber.org/zap/zapcore".Field in argument to logger.GetInstance().Fatal
./main.go:145:13: cannot use "go.uber.org/zap".Error(err) (type "go.uber.org/zap/zapcore".Field) as type "github.com/Devatoria/go-mesos-executor/vendor/go.uber.org/zap/zapcore".Field in argument to logger.GetInstance().Fatal

I'm not a go expert but these are the steps that i did:
git clone https://github.com/Devatoria/go-mesos-executor
cd go-mesos-executor
go get
go build

Where i'm wrong ?
I'm using "go version go1.9.4 linux/amd64" on Centos 7.x

Another question: how i can use the hooks from my custom scheduler ? I didn't find any doc about it.
I suppose that i need to add some params to the json, or i'm wrong ?

Thanks in advance for you kind support

Review README.md and move to wiki

The README.md file contains a lot of information and is not easily readable. We must move some of those info to the wiki. They will be easy to find and README.md will be easier to read.

Add HTTPS healthchecks support

Review healthchecks tests

Health checks tests are too long (more than 7s to execute).eee

Re-register with the agent when not available

Allow executor to re-register with the agent, keeping containers running while agent is stopped

Need example

Thank you much for such interesting project.
Could you share simple guide how to attach your executor to Mesos agent?
I've use next config:

debug: true # Enable debug mode (more logs)
agent_endpoint: xx11-x1-38-15.xx.xxx.xx:5051
hooks: # Enabled hooks
#  - acl
  - removeContainer
#  - insertIptables
#  - removeIptables
proc_path: /proc # Mount path of host proc folder
iptables:
  container_bridge_interface: docker0 # Brigde interface for container network
acl:
  default_allowed_cidr: # Set of default IP (with CIDR) to allow
    - 172.17.0.1/32 # You should add host IP here for framework-based health checks

Debug output

./go-mesos-executor --debug --docker_socket /var/run/docker.sock
{"level":"info","ts":1506511341.5039887,"caller":"go-mesos-executor/main.go:36","msg":"Initializing the executor","executorID":"","frameworkID":""}
{"level":"info","ts":1506511341.5041852,"caller":"go-mesos-executor/main.go:51","msg":"Creating hook manager","hooks":["removeContainer"]}
{"level":"debug","ts":1506511341.5042639,"caller":"hook/hook_manager.go:91","msg":"Disabling acl post-run hook"}
{"level":"debug","ts":1506511341.5043116,"caller":"hook/hook_manager.go:91","msg":"Disabling insertIptables post-run hook"}
{"level":"debug","ts":1506511341.504338,"caller":"hook/hook_manager.go:91","msg":"Disabling removeIptables pre-stop hook"}
{"level":"fatal","ts":1506511341.5157998,"caller":"go-mesos-executor/main.go:67","msg":"An error occured while running the executor","error":"malformed request"}

Hook to handle network mode

In some condition, it can be useful to disable some network mode (host).

Enable TravisCI

In order to be able to check that a branch is: building, passing tests and passing linters.

Add tests on namespaces package

This package doesn't have test and is critical if it doesn't work (could lead to network settings leaks, such as iptables rules leaks).

Enable Viper WatchConfig

Viper WatchConfig system allows the configuration to be hot reloaded and to execute some things on change. It could be used to do some things on running containers, such as change the agent re-register delay or re-register threshold.

Hook to disable to disable volume

Hook to avoid container to mount volume from host

Improve hooks system

It would be cool to have a more complete hooks system with some features like:

a SetupHook function called on hook registration and used to initialize the hook
allow hook to have hook-specific fields in the structure (private) to store some values across the hook lifecycle

It could probably be done using interface and composition systems, but it also needs to review a lot of things (tests, hook manager, ...). To be discussed.

Refactor error messages

Review the whole application error messages logging to be able to easily identify where the problem comes from when it happens.