Giter VIP home page Giter VIP logo

damon's Introduction

NOTICE: SUPPORT FOR THIS PROJECT ENDED ON 18 November 2020

This projected was owned and maintained by Jet.com (Walmart). This project has reached its end of life and Walmart no longer supports this project.

We will no longer be monitoring the issues for this project or reviewing pull requests. You are free to continue using this project under the license terms or forks of this project at your own risk. This project is no longer subject to Jet.com/Walmart's bug bounty program or other security monitoring.

Actions you can take

We recommend you take the following action:

  • Review any configuration files used for build automation and make appropriate updates to remove or replace this project
  • Notify other members of your team and/or organization of this change
  • Notify your security team to help you evaluate alternative options

Forking and transition of ownership

For security reasons, Walmart does not transfer the ownership of our primary repos on Github or other platforms to other individuals/organizations. Further, we do not transfer ownership of packages for public package management systems.

If you would like to fork this package and continue development, you should choose a new name for the project and create your own packages, build automation, etc.

Please review the licensing terms of this project, which continue to be in effect even after decommission.

ORIGINAL README BELOW


Build status

Damon

Damon is a supervisor program to constrain windows executables that are run under the raw_exec driver in Nomad.

Usage

To use Damon, run it before your command.

damon.exe yourapp.exe [args]

Configuration

Damon uses environment variables to configure process monitoring and resource constraints.

Logging Options

  • DAMON_LOG_MAX_FILES: the number of old logs to keep after rotating.
  • DAMON_LOG_MAX_SIZE: the maximum size (in MB) of the active log file before it gets rotated.
  • DAMON_LOG_DIR: directory in which to place damon log files. When DAMON_LOG_DIR is unset, it will attempt to use the standard nomad log directory ${NOMAD_ALLOC_DIR}/logs. If NOMAD_ALLOC_DIR is unset, then it will default to the current working directory.
  • DAMON_NOMAD_LOG_SUFFIX: Is appended to the log name of the active log file. Rotated log files contain a datestamp. The default value is .damon.log
  • DAMON_LOG_NAME: Is the full name of the log file (without the directory) - Setting this overrides DAMON_NOMAD_LOG_SUFFIX. When this is unset, it will default to ${NOMAD_TASK_NAME}${DAMON_NOMAD_LOG_SUFFIX}

Constraint Options

  • DAMON_ENFORCE_CPU_LIMIT: When set to Y - it enforces CPU constraints on the wrapped process. Set to 'N' to disable CPU-rate limits. (Default: 'Y')
  • DAMON_ENFORCE_MEMORY_LIMIT: When set to Y - it enforces memory limits on the wrapped process. Set to 'N' to disable memory limits. (Default: 'Y')
  • DAMON_CPU_LIMIT: The CPU Limit in MHz. Defaults to NOMAD_CPU_LIMIT.
  • DAMON_MEMORY_LIMIT: The Memory Limit in MB. Defaults to NOMAD_MEMORY_LIMIT.
  • DAMON_RESTRICTED_TOKEN: When set to Y - it runs the wrapped process with a Restricted Token:
    • Drops all Privileges
    • Disables the BUILTIN\Administrator SID

Metrics Options

  • DAMON_ADDR: Listens on this address to serve prometheus metrics. Default: ${NOMAD_ADDR_damon} This option is designed to work with the NOMAD_ADDR_damon environment variable. This means you should change your job spec to:
    • request a port labeled "damon"
    • add a service to the task that advertises the "damon" port to Consul service discovery - so that your prometheus infrastructure can find it and scrape it.
  • DAMON_METRICS_ENDPOINT: The path to the prometheus metrics endpoint. Default: /metrics

Building & Testing Damon

Included with this repository is make.ps1 which can be used to build damon.exe and also run tests.

Build Binary

.\make.ps1 -Build

Lint Code

Runs golangci-lint against the codebase. It will Install golangci-lint if it doesn't exist in ${GOPATH}/bin.

.\make.ps1 -Lint

Test Code

Runs tests and generates code coverage files.

.\make.ps1 -Test

Give it a Try

Check out the examples directory for scripts and job definitions.

Be sure to alter to environment variables, artifact locations, etc... to match your environment.

damon's People

Contributors

ddreier avatar enricosada avatar erichgoldman avatar jorgef avatar justenwalker avatar thr27 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

damon's Issues

Expose CPU and Memory limits as prometheus metrics

Expose the limits that are being enforced on the task as Prometheus metrics. These should just be gauges for:

  • damon_cpu_limit_hz : CPU Rate limit converted to HZ
  • damon_cpu_limit_percent : CPU Rate limit converted to percent of total compute on the host: Limit HZ / Total Machine HZ` (0.0 - 1.0 not 0 - 100)
  • damon_memory_limit_bytes : Memory Limit converted to Bytes

Implement Nomad Task Driver support (0.9.x)

Damon was developed long before Nomad had the concept of a Task Driver. It was always the plan that some day we'd like to contribute this back to Nomad as a part of the engine. Now since Nomad 0.9.0 preview supports pluggable task drivers, we can use that as another way to run windows executables with Damon.

This alternate mode needs documentation as well as implementation.

Some Open Questions still:

  • How will Damon report metrics as a Task Driver? Right now it is done as another separate task service and advertised port.
  • Will the task driver have access to the full job spec? Possibly to read in more details about the job that are not present entirely in the environment variables.
  • How does the task driver log? Is the log aggregated in the nomad client logs? Originally we liked the log to be along-side the task since it was convenient for log shipping application logs with damon events to correlate problems. I'm sure we can continue to log to file ourselves with rotate logs; but maybe make it configurable.

Unsafe DLL Loading

You're using syscall, but you should be using x/sys/windows's lazy DLL struct instead:

kernel32DLL = syscall.NewLazyDLL("kernel32.dll")
advapi32DLL = syscall.NewLazyDLL("advapi32.dll")
userenvDLL = syscall.NewLazyDLL("userenv.dll")
psapiDLL = syscall.NewLazyDLL("psapi.dll")
iphlpapiDLL = syscall.NewLazyDLL("iphlpapi.dll")

iphlpapi.dll and userenv.dll are not part of \KnownDlls.

Your blog post indicates you know about the right way to do it but for some reason didn't.

Generate windows syscalls using `go generate`

Use go generate tool to generate windows syscalls instead of writing them out manually.
This will help with maintainability

//go:generate go run golang.org/x/sys/windows/mkwinsyscall -output zsyscall_windows.go syscall_windows.go

Suggestion from #18 (comment) by @zx2c4 with details and examples

By the way, there's an easier way of dealing with all this using //sys. Check it out:

In this file (or in any file, really), I define a bunch of calls like this one:

https://github.com/WireGuard/wireguard-windows/blob/3f40da2044fd468ed05f3902608e62382b81ec9c/tunnel/firewall/syscall_windows.go#L14-L15

And then in this file, I have a go generate directive:

https://github.com/WireGuard/wireguard-windows/blob/3f40da2044fd468ed05f3902608e62382b81ec9c/tunnel/firewall/mksyscall.go#L8

Which winds up creating a file containing the fuction, like:

https://github.com/WireGuard/wireguard-windows/blob/3f40da2044fd468ed05f3902608e62382b81ec9c/tunnel/firewall/zsyscall_windows.go#L39-L52

https://github.com/WireGuard/wireguard-windows/blob/3f40da2044fd468ed05f3902608e62382b81ec9c/tunnel/firewall/zsyscall_windows.go#L90-L100

Start monitoring TCP Connection counts. Enforce TCP Connection count limits by killing.

There are a limited number of ephemeral ports on the machine. In Windows 2016 this is around 16k. A process which opens up many connections at once, or has a connection leak is likely to use up all available ephemeral ports - causing other services to fail to bind to their assigned nomad ports, and other outbound connection failures.

There already exists functionality in the win32 package to query the TCP Tables for connections by PID. We should use this to:

  1. Expose a damon_tcp_connection_count metric so that monitoring and alerting can be done in Prometheus
  2. Add an optional DAMON_TCP_CONNECTION_LIMIT configuration to set an upper bound on the number of TCP connections a process can make. If this option is set, we should terminate the child process when it exceeds this count.

There isn't a lot we can do to prevent an application from opening another connection; so the safest option would be to terminate it to prevent it from growing unbounded.

This also mean we'd need to enumerate the entire process tree, since there could be more than one process under the child that is asking for connections. Terminating the parent process should be enough to kill the child processes as well given that, once Damon exits, the job object will terminate all processes still in the JobObject since the last handle will have closed.

Reorganize package structure

Damon was not really meant to be used as a library and imported by other modules.
In order to make this more explicit, most packages should be moved under /internal except for the CLI which should be put under /cmd

Some guidelines to follow in golang-standards/project-layout

ACCESS_DENIED on Resume running on Windows7

Using damon on windows7 fails on starting an application.
Damon terminates with ACCESS_DENIED error on trying to resume threads in child process.

{"level":"info","cmdline":["D:\dev\go.dev\src\github.com\jet\damon\damon.exe","C:\Windows\System32\notepad.exe"],"revision":"90081519ee8206e7af6671120ed382b0298ec0af","version":{"Revision":"90081519ee8206e7af6671120ed382b0298ec0af","Number":"0.1.0","PreRelease":"","BuildMetadata":""},"time":"2019-05-17T17:16:44+02:00","message":"damon starting"}
{"level":"error","error":"The handle is invalid.","time":"2019-05-17T17:16:44+02:00","message":"win32: failed to close process handle"}
{"level":"error","stacktrace":["github.com/jet/damon/container.(*Container).Start\n\tD:/dev/go.dev/src/github.com/jet/damon/container/container.go:181","main.main\n\tD:/dev/go.dev/src/github.com/jet/damon/main.go:82","runtime.main\n\td:/dev/golang/src/runtime/proc.go:200","runtime.goexit\n\td:/dev/golang/src/runtime/asm_amd64.s:1337"],"error":"container: Could not resume process main thread: Access is denied.","time":"2019-05-17T17:16:44+02:00","message":"damon startup error"}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.