adroll / batchiepatchie Goto Github PK

"documentation on deployment page" link in the Batchiepatchie - Quick start page is broken.

Currently, it is pointing to https://github.com/AdRoll/batchiepatchie/blob/master/docs/docs/deployment
It should be pointed to https://github.com/AdRoll/batchiepatchie/blob/master/docs/docs/deployment.md instead

I am trying to run locally using the instructions. When executing the command docker-compose up. I am getting the following error.

[batchiepatchie_api 5/11] RUN go get:
#14 336.3 # github.com/AdRoll/batchiepatchie
#14 336.3 ./logentries.go:9:47: not enough arguments in call to logentriesrus.NewLogentriesrusHook
#14 336.3 have (string)
#14 336.3 want (string, string)

failed to solve: rpc error: code = Unknown desc = executor failed running [/bin/sh -c go get]: exit code: 2

Not sure if i am doing something wrong

Sometimes the batch job type counts (count, group-by job-status) is very useful and it's not possible to get that in the UI. That is, the UI could be almost exactly like the batch dashboard but add the option to filter on batch-queue wildcard matches (queue WHERE LIKE).

I followed docs and it appeared it does not work:

frontend_1 | [HPM] Error occurred while trying to proxy request /api/v1/jobs/stats?start=1601046858&end=1601651658 from localhost:8080 to http://api:5454/ (ENOTFOUND) (https://nodejs.org/api/errors.html#errors_common_system_errors)

Any ideas?

I am trying to deploy batchiepatchie on ECS with Terraform. In order to have all the benefits of Terraform, I want to be able to save the Terraform code to our BitBucket but obviously that means that I don't want to save the config.toml file as it contains the RDS password.

As a result, I wish to use the ability you have to point to the config file in S3 to get the toml file. I have never used go before but I think that you are pointing all the AWS client to us-east-1, am I correct? Would you be able to make this variable in some way as we are using eu-west-1? There are a number of places where I think you invoke the AWS Client but I'm not sure of the correct place to edit this.

On a separate note, can you add to the documentation how you point the front end to a custom backend, I think you should use API_BASE_URL but can you confirm this and also explain what BASE_URL is. Also, it would be good to understnad the BATCHIEPATCHIE_IP in the backend, the command you use to get the local ip doesn't work.

Finally, I am getting an error in the backend "SELECT job_queue FROM activated_job_queues : failed with error: pq: relation "activated_job_queues" does not exist", can you help me to debug this. I have launched the postgres db which seems to be fine and also launched the migrations as a separate task which exits with the 0 code so I'm not sure what might be wrong here.

We have batchie patchie running in a production setup. Whenever new job spins up backend throws nil pointer reference and crashes.

{"level":"info","msg":"Logging ECS cluster statuses.","time":"2023-01-19T11:25:36Z"}
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xa54007]

goroutine 11 [running]:
github.com/AdRoll/batchiepatchie/jobs.MonitorECSClusters({0x7f192019aff8, 0xc000012c60}, {0x160a640, 0x0, 0x1?})
	/home/ubuntu/batchiepatchie/jobs/monitor_ecs_clusters.go:126 +0x16e7
github.com/AdRoll/batchiepatchie/syncer.RunPeriodicScaler.func1()
	/home/ubuntu/batchiepatchie/syncer/batchsync.go:296 +0x13e
created by github.com/AdRoll/batchiepatchie/syncer.RunPeriodicScaler
	/home/ubuntu/batchiepatchie/syncer/batchsync.go:287 +0x6a

This happens when the job status is STARTING in AWS console. When the status change to RUNNING I can restart batchiepatchie again.

FYI, my setup is running on ubuntu 22.04 in an EC2 and the capacity provider for batch is Fargate

The documentation says:

Our docker-compose.yml file passes environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY as environment variables to the Batchiepatchie container so if you have these variables set up on your host system, the credentials should be passed correctly.

Perhaps it was true at one point but it doesn't seem to be now. It works if I do this:

diff --git a/docker-compose.yml b/docker-compose.yml
index cbd312e..7ec9aaf 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -5,6 +5,8 @@ services:
     environment:
       - BUILD_ENV=${BUILD_ENV}
       - BATCHIEPATCHIE_CONFIG=batchiepatchie-dockercompose-config.toml
+      - AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
+      - AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
     volumes:
       - .:/go/src/github.com/AdRoll/batchiepatchie
     ports:
@@ -57,4 +59,3 @@ networks:
     ipam:
       config:
        - subnet: "172.29.0.0/16"
-

I'm trying to get a version of this running to test out for a deployment, however I can't get it to build. I'm able to use the docker setup without issue (thanks!), however the actual deployment instructions and build fails on the go get, it looks like the repo was move from the SemanticSugar org to AdRoll. For example https://github.com/SemanticSugar/batchiepatchie no longer exists but https://github.com/AdRoll/batchiepatchie does.

I am very new to go so not sure if I'm doing something wrong or if there is an actual issue here. Can you confirm either way? I was able to build it by basically copying this repo into the gopath directory, based on the docker config.

Thanks for building this!

Is it possible to release an updated version of running/deployment instructions as the current ones lead to a lot of errors?

I'm trying to evaluate using batchiepatchie for our AWS Batch jobs. I tried follow instructions and use docker-compose up to get a locally running batchiepatchie. Sadly, it seems to fail

api_1         | 0:58:41 main        | Build Failed: 
api_1         |  go build github.com/AdRoll/batchiepatchie: mkdir tmp/: permission denied
batchiepatchie_api_1 exited with code 1

I peeked into the config and saw that it does something else with BUILD_ENV=PRODUCTION. So I tried it:

api_1         | go build github.com/AdRoll/batchiepatchie: copying /tmp/go-build184121519/b001/exe/a.out: open batchiepatchie: permission denied
batchiepatchie_api_1 exited with code 1

This failed as well. The UI does come up but fails to talk to the backend, obviously.

I'm not sure if I'm doing something wrong here...

I have been using batchiepatchie for years. However I am trying to run it on a new company and the deployment seems to be broken.

I have also noticed plenty of PRs by dependabot are being left ignored.

Is the project being maintained, if not, I would not mind forking it and taking care of it. In my opinion it is still the best OSS way to manage aws batch at scale.

I was super excited to a see a project that can keep track of Batch Job history for more than 24 hours. We already make heavy use of AWS Batch but their interface in the console is a bit lacking.

But on trying to run BatchiePatchie locally with docker-compose, I've immediately run into problems. What I tried doing is this:

• Updated batchiepatchie-dockercompose-config.toml to set the region to us-east-1 instead of us-west-2, since that's what we use
• Ran this command:

  AWS_ACCESS_KEY_ID=<myaccesskey> AWS_SECRET_ACCESS_KEY=<mysecret> docker-compose up
  

When the containers eventually load and I visit localhost:8080, I get the interface but no data. On the "Job queues" tab it doesn't show anything under "All job queues." If I look at the container logs I see messages like this one:

{"bytes_in":"0","bytes_out":"3","error":"RequestError: send request failed\ncaused by: Post \"https://batch.us-east-1.amazonaws.com/v1/describejobqueues\": net/http: TLS handshake timeout","host":"localhost:8080","latency":"167230091","latency_human":"2m47.230091342s","level":"info","method":"GET","msg":"Handled request","path":"/api/v1/job_queues/all","referer":"http://localhost:8080/job_queues","remote_ip":"172.29.0.5","request_id":"","status":500,"time":"2020-10-13T18:10:47Z","time_rfc3339":"2020-10-13T18:10:47Z","uri":"/api/v1/job_queues/all","user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.75 Safari/537.36"}

What's worse is when I check the CloudTrail events for the user represented by the access key here, I don't see any events. It's as if BatchiePatchie isn't actually querying AWS at all. And if I try to hit http://127.0.0.1:5454/api/v1/job_queues/all directly, it takes a really long time before eventually just returning an empty object {}.

What am I missing? I'm hesitant to go through the work of deploying this when I can't even get it function properly with docker-compose.

Next to the REFRESH button, add a POLL button that will first request the backend-API to poll the selected queues for new job descriptions and then refresh the UI. With this option, the auto-poll config setting (sync_period) could be relaxed to many minutes.

Attaching to batchiepatchie_frontend_1
frontend_1 |
frontend_1 | > [email protected] dev /opt/frontend
frontend_1 | > webpack-dev-server --host 0.0.0.0
frontend_1 |
frontend_1 | internal/modules/cjs/loader.js:584
frontend_1 | throw err;
frontend_1 | ^
frontend_1 |
frontend_1 | Error: Cannot find module 'webpack-cli/bin/config-yargs'
frontend_1 | at Function.Module._resolveFilename (internal/modules/cjs/loader.js:582:15)
frontend_1 | at Function.Module._load (internal/modules/cjs/loader.js:508:25)
frontend_1 | at Module.require (internal/modules/cjs/loader.js:637:17)
frontend_1 | at require (internal/modules/cjs/helpers.js:22:18)
frontend_1 | at Object. (/opt/frontend/node_modules/webpack-dev-server/bin/webpack-dev-server.js:84:1)
frontend_1 | at Module._compile (internal/modules/cjs/loader.js:701:30)
frontend_1 | at Object.Module._extensions..js (internal/modules/cjs/loader.js:712:10)
frontend_1 | at Module.load (internal/modules/cjs/loader.js:600:32)
frontend_1 | at tryModuleLoad (internal/modules/cjs/loader.js:539:12)
frontend_1 | at Function.Module._load (internal/modules/cjs/loader.js:531:3)
frontend_1 | npm ERR! code ELIFECYCLE
frontend_1 | npm ERR! errno 1
frontend_1 | npm ERR! [email protected] dev: webpack-dev-server --host 0.0.0.0
frontend_1 | npm ERR! Exit status 1
frontend_1 | npm ERR!
frontend_1 | npm ERR! Failed at the [email protected] dev script.
frontend_1 | npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
frontend_1 |
frontend_1 | npm ERR! A complete log of this run can be found in:
frontend_1 | npm ERR! /root/.npm/_logs/2020-03-25T02_29_50_891Z-debug.log
batchiepatchie_frontend_1 exited with code 1

When scrolling through the pages, at times hitting the "Next >>" button at the bottom of the page adds "1" to the current string value of the page instead of the numerical value.

i.e., if on page 3, and you hit next, it will get in a mode that takes you to page "31". if you hit next again, it will take you to page "311", always adding a string "1" to the page.

The "Previous <<" button works well.

When I try to run docker-compose up on an M1 Mac, I get this error:

api_1         | + fresh -c fresh.conf
api_1         | Loading settings from fresh.conf
api_1         | 22:25:52 runner      | InitFolders
api_1         | 22:25:52 runner      | mkdir ./tmp
api_1         | 22:25:52 runner      | mkdir ./tmp: file exists
api_1         | inotify_init: function not implemented
batchiepatchie_api_1 exited with code 1

cc @jcshott

The sync_period configuration option does not apply globally to all polling operations, e.g.

• scaling with AWS Batch
• Logging compute environment changes
• Timed out killer

Add or document configuration settings for toggling these features on/off and, when on, setting the polling period (in seconds).

I've managed to get things working with the quickstart. However when I look at runnin job I see that the sections for instance id or private ip are not being populated. Is there a setting that controls this?

# batchiepatchie
./logentries.go:9:47: not enough arguments in call to logentriesrus.NewLogentriesrusHook
        have (string)
        want (string, string)

NewLogEntriesHook changed to require 2 arguments jcftang/logentriesrus@a32c4b1