Giter VIP home page Giter VIP logo

panamax-api's Introduction

Panamax - Docker Management for Humans

Panamax is a containerized app creator with an open-source app marketplace hosted in GitHub. Panamax provides a friendly interface for users of Docker, Fleet & CoreOS. With Panamax, you can easily create, share, and deploy any containerized app no matter how complex it might be. Learn more at Panamax.io or browse the Panamax Wiki.

Panamax API

Master_Build_Status Coverage Status

The API behind The Panamax UI.

The Panamax API interacts with a number of APIs running on the local system to launch and gather information about applications:

Additional, external APIs are also used to do things like query the Docker Hub and collect usage metrics:

NOTE

This repo is no longer being maintained. Users are welcome to fork it, but we make no warranty of its functionality.

Getting Started

If you're trying to install panamax as a complete package: Follow these instructions.

If you are a potential contributor or would like to install the API only, read on.

Pre-requisites

In its current form Panamax only works with CoreOS as the host system. The Panamax API application is meant to run inside a Docker container on a CoreOS host where it has visibility to the Docker, etcd, and systemd-journal-gatewayd APIs.

We recommend you use the Running CoreOS on Vagrant instructions to get CoreOS running on your local system in VirtualBox. Make sure you have VirtualBox, Vagrant and the CoreOS Vagrantfile installed before proceeding.

Check-out the Code

A typical development set-up for Panamax API will consist of your local machine (running whatever OS you choose), a VirtualBox VM running CoreOS and then a Docker container hosting the Panamax API application.

Ultimately, you need to get the panamax-api source code into the Docker container, and there are a number of ways to accomplish that:

  • git clone the source code directly into the Docker container
  • git clone the source code in the CoreOS host and then share it into the Docker container via volume mount
  • git clone the source code on the local system, share it into CoreOS via rsync or NFS, and then share it from CoreOS to the Docker container via volume mount.

While the third option is the most complicated it does give you the flexibility to use your usual editor/IDE on your local system while running the application itself in the Docker container. This is the option that we'll walk-through below.

Start by cloning the panamax-api repository:

git clone https://github.com/CenturyLinkLabs/panamax-api.git

Start CoreOS

  1. Update your Vagrantfile to share the panamax-api directory (the one cloned above) into the CoreOS VM. If you used the Vagrantfile provided in the CoreOS installation instructions you should find a sample synced_folder instruction that has been commented-out. Edit it to reflect the path to your panamax-api directory and un-comment it.

    config.vm.synced_folder "/Users/jdoe/panamax-api", "/home/core/panamax-api", id: "core", :nfs => true, :mount_options => ['nolock,vers=3,udp']

  2. Open a port so that you can reach the Panamax API from your local machine. Place the following line in your Vagrantfile (you can choose whatever port numbers you like).

    config.vm.network :forwarded_port, guest: 8888, host: 8888

  3. Start the CoreOS virtual machine. From the directory where your Vagrantfile is located do the following

    vagrant up && vagrant ssh

Start the Container

At this point you should find yourself at the CoreOS prompt.

  1. Start etcd, fleet and journal-gatewayd services. These three services are used by the Panamax API and need to be running in CoreOS.

    sudo systemctl start etcd sudo systemctl start fleet sudo systemctl start systemd-journal-gatewayd.socket

If you get tired of manually starting these services there are instructions in the CoreOS documentation for starting these services automatically via Cloud-Config.

  1. Pull the base docker images for running the panamax-api container.

    docker pull centurylink/panamax-ruby-base

  2. Start the container

    docker run -it
    -v /home/core/panamax-api:/var/app/panamax-api
    -v /run/docker.sock:/run/docker.sock
    -v /run/fleet.sock:/run/fleet.sock
    -p 8888:3000
    -e "JOURNAL_ENDPOINT=http://10.1.42.1:19531"
    centurylink/panamax-ruby-base /bin/bash

The first -v argument bind mounts the panamax-api source directory into the container. The container path you use for the volume mount can be anything you choose (it's /var/app/panamax-api in the example above). Obviously, the host path must match the the directory in CoreOS where the code was shared.

The second and third -v arguments bind mount the Docker and fleet daemon's respective API sockets into the container. This is how the Panamax API is able to interact with the Docker and fleet daemon from inside the container.

The -p argument creates a mapping between port 3000 in the container (the default port used by the Rails application) and port 8888 on CoreOS (note that host port here needs to match the guest port specified by the mapping in the Vagrantfile). This gives us a network path from the local machine, through CoreOS, to the application running inside the panamax-api container.

The -e argument is used to inject the endpoint for the Journal API. The IP address used in this environment variable should match that of the docker0 network interfact in CoreOS. You should be able to use the value shown above as-is, but we've seen a few occasions where the docker0 interface is assigned a different address. You can check the address assigned to the docker0 interface by running ifconfig at the CoreOS prompt.

Start the Application

You should now be at a prompt inside the running container.

  1. Change to the directory containing the panamax-api source code

    cd /var/app/panamax-api

  2. Install the application's gem dependencies

    bundle

  3. Set-up the SQLite database and load the sample templates

    rake db:create db:migrate db:seed rake panamax:templates:load

  4. Star the application

    rails server

If you used all the configuration values specified in these instructions, the Panamax API should now be accessible from your local machine on port 8888:

$ curl http://localhost:8888/local_images.json

[{"id":"8effcd007cad6bbdb936d547fac60f65c5f24f5a579e8d34e50e220d588dac70","virtual_size":436409804,"tags":["centurylink/panamax-ruby-base:latest"]}]

Run the automated tests

rspec spec

panamax-api's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

panamax-api's Issues

Faraday::Error::TimeoutError (Net::ReadTimeout):

Not sure why that happened, but it happened :)

I, [2014-08-23T10:17:19.383945 #24]  INFO -- : Started GET "/apps/5.json" for 172.17.0.4 at 2014-08-23 10:17:19 +0000
I, [2014-08-23T10:17:19.735639 #24]  INFO -- : Processing by AppsController#show as JSON
I, [2014-08-23T10:17:19.735848 #24]  INFO -- :   Parameters: {"id"=>"5"}
I, [2014-08-23T10:17:20.126284 #24]  INFO -- : Completed 200 OK in 388ms (Views: 251.6ms | ActiveRecord: 134.3ms)
I, [2014-08-23T10:17:20.632440 #24]  INFO -- : Started GET "/apps/5/services/9.json" for 172.17.0.4 at 2014-08-23 10:17:20 +0000
I, [2014-08-23T10:17:20.633432 #24]  INFO -- : Processing by ServicesController#show as JSON
I, [2014-08-23T10:17:20.633538 #24]  INFO -- :   Parameters: {"app_id"=>"5", "id"=>"9"}
I, [2014-08-23T10:17:20.675703 #24]  INFO -- : Completed 200 OK in 42ms (Views: 39.9ms | ActiveRecord: 0.3ms)
E, [2014-08-23T10:17:24.401088 #24] ERROR -- :
Faraday::Error::TimeoutError (Net::ReadTimeout):
  /usr/local/lib/ruby/2.1.0/net/protocol.rb:158:in `rescue in rbuf_fill'
  /usr/local/lib/ruby/2.1.0/net/protocol.rb:152:in `rbuf_fill'
  /usr/local/lib/ruby/2.1.0/net/protocol.rb:122:in `read_all'
  /usr/local/lib/ruby/2.1.0/net/http/response.rb:292:in `block in read_body_0'
  /usr/local/lib/ruby/2.1.0/net/http/response.rb:250:in `inflater'
  /usr/local/lib/ruby/2.1.0/net/http/response.rb:274:in `read_body_0'
  /usr/local/lib/ruby/2.1.0/net/http/response.rb:201:in `read_body'
  /usr/local/lib/ruby/2.1.0/net/http.rb:1129:in `block in get'
  /usr/local/lib/ruby/2.1.0/net/http.rb:1415:in `block (2 levels) in transport_request'
  /usr/local/lib/ruby/2.1.0/net/http/response.rb:162:in `reading_body'
  /usr/local/lib/ruby/2.1.0/net/http.rb:1414:in `block in transport_request'
  /usr/local/lib/ruby/2.1.0/net/http.rb:1405:in `catch'
  /usr/local/lib/ruby/2.1.0/net/http.rb:1405:in `transport_request'
  /usr/local/lib/ruby/2.1.0/net/http.rb:1378:in `request'
  /usr/local/lib/ruby/2.1.0/net/http.rb:1371:in `block in request'
  /usr/local/lib/ruby/2.1.0/net/http.rb:853:in `start'
  /usr/local/lib/ruby/2.1.0/net/http.rb:1369:in `request'
  /usr/local/lib/ruby/2.1.0/net/http.rb:1128:in `get'
  /usr/local/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/adapter/net_http.rb:73:in `perform_request'
  /usr/local/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/adapter/net_http.rb:38:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/response.rb:8:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/faraday_middleware-0.9.0/lib/faraday_middleware/response_middleware.rb:30:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/faraday_middleware-0.9.0/lib/faraday_middleware/request/encode_json.rb:23:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/connection.rb:253:in `run_request'
  /usr/local/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/connection.rb:106:in `get'
  /usr/src/app/lib/panamax_agent/request.rb:15:in `request'
  /usr/src/app/lib/panamax_agent/request.rb:6:in `block (2 levels) in <module:Request>'
  /usr/src/app/lib/panamax_agent/journal/client/entries.rb:21:in `get_entries_by_fields'
  /usr/src/app/lib/panamax_agent/journal/client.rb:33:in `list_journal_entries'
  /usr/src/app/app/models/concerns/app_journal.rb:6:in `journal'
  /usr/src/app/app/controllers/apps_controller.rb:43:in `journal'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/abstract_controller/base.rb:189:in `process_action'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal/rendering.rb:10:in `process_action'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/abstract_controller/callbacks.rb:20:in `block in process_action'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:113:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:113:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:229:in `block in halting'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:166:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:166:in `block in halting'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:86:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:86:in `run_callbacks'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/abstract_controller/callbacks.rb:19:in `process_action'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal/rescue.rb:29:in `process_action'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal/instrumentation.rb:31:in `block in process_action'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/notifications.rb:159:in `block in instrument'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/notifications.rb:159:in `instrument'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal/instrumentation.rb:30:in `process_action'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal/params_wrapper.rb:250:in `process_action'
  /usr/local/lib/ruby/gems/2.1.0/gems/activerecord-4.1.4/lib/active_record/railties/controller_runtime.rb:18:in `process_action'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/abstract_controller/base.rb:136:in `process'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionview-4.1.4/lib/action_view/rendering.rb:30:in `process'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal.rb:196:in `dispatch'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal/rack_delegation.rb:13:in `dispatch'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal.rb:232:in `block in action'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/routing/route_set.rb:82:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/routing/route_set.rb:82:in `dispatch'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/routing/route_set.rb:50:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/journey/router.rb:71:in `block in call'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/journey/router.rb:59:in `each'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/journey/router.rb:59:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/routing/route_set.rb:678:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/etag.rb:23:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/conditionalget.rb:25:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/head.rb:11:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/params_parser.rb:27:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/flash.rb:254:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/session/abstract/id.rb:225:in `context'
  /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/session/abstract/id.rb:220:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/cookies.rb:560:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/activerecord-4.1.4/lib/active_record/query_cache.rb:36:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/activerecord-4.1.4/lib/active_record/connection_adapters/abstract/connection_pool.rb:621:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:82:in `run_callbacks'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/remote_ip.rb:76:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/railties-4.1.4/lib/rails/rack/logger.rb:38:in `call_app'
  /usr/local/lib/ruby/gems/2.1.0/gems/railties-4.1.4/lib/rails/rack/logger.rb:20:in `block in call'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/tagged_logging.rb:68:in `block in tagged'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/tagged_logging.rb:26:in `tagged'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/tagged_logging.rb:68:in `tagged'
  /usr/local/lib/ruby/gems/2.1.0/gems/railties-4.1.4/lib/rails/rack/logger.rb:20:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/request_id.rb:21:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/methodoverride.rb:21:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/runtime.rb:17:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/cache/strategy/local_cache_middleware.rb:26:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/sendfile.rb:112:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/railties-4.1.4/lib/rails/engine.rb:514:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/railties-4.1.4/lib/rails/application.rb:144:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/content_length.rb:14:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/puma-2.8.2/lib/puma/server.rb:490:in `handle_request'
  /usr/local/lib/ruby/gems/2.1.0/gems/puma-2.8.2/lib/puma/server.rb:361:in `process_client'
  /usr/local/lib/ruby/gems/2.1.0/gems/puma-2.8.2/lib/puma/server.rb:254:in `block in run'
  /usr/local/lib/ruby/gems/2.1.0/gems/puma-2.8.2/lib/puma/thread_pool.rb:92:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/puma-2.8.2/lib/puma/thread_pool.rb:92:in `block in spawn_thread'


I, [2014-08-23T10:17:24.402883 #24]  INFO -- : Completed 500 Internal Server Error in 13151ms (Views: 0.3ms | ActiveRecord: 0.3ms)
E, [2014-08-23T10:17:29.856085 #24] ERROR -- :
Faraday::Error::TimeoutError (Net::ReadTimeout):
  /usr/local/lib/ruby/2.1.0/net/protocol.rb:158:in `rescue in rbuf_fill'
  /usr/local/lib/ruby/2.1.0/net/protocol.rb:152:in `rbuf_fill'
  /usr/local/lib/ruby/2.1.0/net/protocol.rb:134:in `readuntil'
  /usr/local/lib/ruby/2.1.0/net/protocol.rb:144:in `readline'
  /usr/local/lib/ruby/2.1.0/net/http/response.rb:39:in `read_status_line'
  /usr/local/lib/ruby/2.1.0/net/http/response.rb:28:in `read_new'
  /usr/local/lib/ruby/2.1.0/net/http.rb:1408:in `block in transport_request'
  /usr/local/lib/ruby/2.1.0/net/http.rb:1405:in `catch'
  /usr/local/lib/ruby/2.1.0/net/http.rb:1405:in `transport_request'
  /usr/local/lib/ruby/2.1.0/net/http.rb:1378:in `request'
  /usr/local/lib/ruby/2.1.0/net/http.rb:1371:in `block in request'
  /usr/local/lib/ruby/2.1.0/net/http.rb:853:in `start'
  /usr/local/lib/ruby/2.1.0/net/http.rb:1369:in `request'
  /usr/local/lib/ruby/2.1.0/net/http.rb:1128:in `get'
  /usr/local/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/adapter/net_http.rb:73:in `perform_request'
  /usr/local/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/adapter/net_http.rb:38:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/response.rb:8:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/faraday_middleware-0.9.0/lib/faraday_middleware/response_middleware.rb:30:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/faraday_middleware-0.9.0/lib/faraday_middleware/request/encode_json.rb:23:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/connection.rb:253:in `run_request'
  /usr/local/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/connection.rb:106:in `get'
  /usr/src/app/lib/panamax_agent/request.rb:15:in `request'
  /usr/src/app/lib/panamax_agent/request.rb:6:in `block (2 levels) in <module:Request>'
  /usr/src/app/lib/panamax_agent/journal/client/entries.rb:21:in `get_entries_by_fields'
  /usr/src/app/lib/panamax_agent/journal/client.rb:33:in `list_journal_entries'
  /usr/src/app/app/models/concerns/service_journal.rb:5:in `journal'
  /usr/src/app/app/controllers/services_controller.rb:32:in `journal'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/abstract_controller/base.rb:189:in `process_action'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal/rendering.rb:10:in `process_action'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/abstract_controller/callbacks.rb:20:in `block in process_action'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:113:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:113:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:229:in `block in halting'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:166:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:166:in `block in halting'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:86:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:86:in `run_callbacks'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/abstract_controller/callbacks.rb:19:in `process_action'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal/rescue.rb:29:in `process_action'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal/instrumentation.rb:31:in `block in process_action'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/notifications.rb:159:in `block in instrument'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/notifications.rb:159:in `instrument'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal/instrumentation.rb:30:in `process_action'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal/params_wrapper.rb:250:in `process_action'
  /usr/local/lib/ruby/gems/2.1.0/gems/activerecord-4.1.4/lib/active_record/railties/controller_runtime.rb:18:in `process_action'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/abstract_controller/base.rb:136:in `process'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionview-4.1.4/lib/action_view/rendering.rb:30:in `process'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal.rb:196:in `dispatch'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal/rack_delegation.rb:13:in `dispatch'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal.rb:232:in `block in action'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/routing/route_set.rb:82:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/routing/route_set.rb:82:in `dispatch'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/routing/route_set.rb:50:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/journey/router.rb:71:in `block in call'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/journey/router.rb:59:in `each'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/journey/router.rb:59:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/routing/route_set.rb:678:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/etag.rb:23:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/conditionalget.rb:25:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/head.rb:11:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/params_parser.rb:27:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/flash.rb:254:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/session/abstract/id.rb:225:in `context'
  /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/session/abstract/id.rb:220:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/cookies.rb:560:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/activerecord-4.1.4/lib/active_record/query_cache.rb:36:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/activerecord-4.1.4/lib/active_record/connection_adapters/abstract/connection_pool.rb:621:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:82:in `run_callbacks'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/remote_ip.rb:76:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/railties-4.1.4/lib/rails/rack/logger.rb:38:in `call_app'
  /usr/local/lib/ruby/gems/2.1.0/gems/railties-4.1.4/lib/rails/rack/logger.rb:20:in `block in call'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/tagged_logging.rb:68:in `block in tagged'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/tagged_logging.rb:26:in `tagged'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/tagged_logging.rb:68:in `tagged'
  /usr/local/lib/ruby/gems/2.1.0/gems/railties-4.1.4/lib/rails/rack/logger.rb:20:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/request_id.rb:21:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/methodoverride.rb:21:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/runtime.rb:17:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/cache/strategy/local_cache_middleware.rb:26:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/sendfile.rb:112:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/railties-4.1.4/lib/rails/engine.rb:514:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/railties-4.1.4/lib/rails/application.rb:144:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/content_length.rb:14:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/puma-2.8.2/lib/puma/server.rb:490:in `handle_request'
  /usr/local/lib/ruby/gems/2.1.0/gems/puma-2.8.2/lib/puma/server.rb:361:in `process_client'
  /usr/local/lib/ruby/gems/2.1.0/gems/puma-2.8.2/lib/puma/server.rb:254:in `block in run'
  /usr/local/lib/ruby/gems/2.1.0/gems/puma-2.8.2/lib/puma/thread_pool.rb:92:in `call'
  /usr/local/lib/ruby/gems/2.1.0/gems/puma-2.8.2/lib/puma/thread_pool.rb:92:in `block in spawn_thread'


Allow for Privileged Containers

In order to use specific features, such as the ability to create a TUN device for VPNs, the Docker container needs to be run with the "--privileged" flag. Obviously this wouldn't be ideal as a default, but it would open all sorts of possibilities with cross-host networked containers or ambassadors.

Repeated unexpected terminations of PMX_API service

Over the past two days as I'm testing Panamax I've run into multiple instances of a complete shutdown of the service, including all running Docker containers - both the PMX ones and my own. I have another issue open regarding service failure and initially it seemed that removing potentially troublesome character sequences from image names helped, but only to the extent that the service failures were spaced further apart. I'm pasting some logs in the hope there's something there that warrants getting more details on.

Results are varied but eventually all services disappear from fleet, and their status in systemctl is usually "not found" when a 'systemctl restart ' is attempted. Others in systemctl are seemingly left running but must be restarted to bring them back up. Clicking 'rebuild' in the App view seems to be the quickest way to recover after the API shutdown.

Apologies for the length of some of these logs, journalctl can get a bit verbose it seems.

Output from 'journalctl -u panamax-api' of most recent spontaneous service shutdown and restart

Aug 14 19:06:50 panamax-vm docker[15666]: I, [2014-08-14T19:06:50.383951 #23]  INFO -- : Started GET "/apps/8.json" for 172.17.0.109 at 2014-08-14 19:06:50 +0000
Aug 14 19:06:50 panamax-vm docker[15666]: I, [2014-08-14T19:06:50.388183 #23]  INFO -- : Processing by AppsController#show as JSON
Aug 14 19:06:50 panamax-vm docker[15666]: I, [2014-08-14T19:06:50.389979 #23]  INFO -- :   Parameters: {"id"=>"8"}
Aug 14 19:06:50 panamax-vm docker[15666]: I, [2014-08-14T19:06:50.417166 #23]  INFO -- : Completed 200 OK in 25ms (Views: 3.0ms | ActiveRecord: 19.8ms)
Aug 14 19:06:50 panamax-vm docker[15666]: I, [2014-08-14T19:06:50.423189 #23]  INFO -- : Started GET "/apps/8/services.json" for 172.17.0.109 at 2014-08-14 19:06:50 +0000
Aug 14 19:06:50 panamax-vm docker[15666]: I, [2014-08-14T19:06:50.426638 #23]  INFO -- : Processing by ServicesController#index as JSON
Aug 14 19:06:50 panamax-vm docker[15666]: I, [2014-08-14T19:06:50.427971 #23]  INFO -- :   Parameters: {"app_id"=>"8"}
Aug 14 19:06:50 panamax-vm docker[15666]: I, [2014-08-14T19:06:50.697251 #23]  INFO -- : Completed 200 OK in 268ms (Views: 263.7ms | ActiveRecord: 2.4ms)
Aug 14 19:10:01 panamax-vm systemd[1]: panamax-api.service: main process exited, code=exited, status=1/FAILURE
Aug 14 19:10:03 panamax-vm systemd[1]: Unit panamax-api.service entered failed state.
Aug 14 19:10:04 panamax-vm docker[10069]: PMX_API
Aug 14 19:10:04 panamax-vm systemd[1]: panamax-api.service holdoff time over, scheduling restart.
Aug 14 19:10:04 panamax-vm systemd[1]: panamax-api.service failed to schedule restart job: Transaction is destructive.
Aug 14 19:10:04 panamax-vm systemd[1]: Unit panamax-api.service entered failed state.
Aug 14 19:11:34 panamax-vm systemd[1]: Starting Panamax API...
Aug 14 19:11:34 panamax-vm docker[10228]: PMX_API
Aug 14 19:11:34 panamax-vm systemd[1]: Started Panamax API.
Aug 14 19:11:36 panamax-vm docker[10244]: db/mnt/production.sqlite3 already exists
Aug 14 19:11:45 panamax-vm docker[10244]: => Booting Puma
Aug 14 19:11:45 panamax-vm docker[10244]: => Rails 4.1.4 application starting in production on http://0.0.0.0:3000
Aug 14 19:11:45 panamax-vm docker[10244]: => Run `rails server -h` for more startup options
Aug 14 19:11:45 panamax-vm docker[10244]: => Notice: server is listening on all interfaces (0.0.0.0). Consider using 127.0.0.1 (--binding option)
Aug 14 19:11:45 panamax-vm docker[10244]: => Ctrl-C to shutdown server
Aug 14 19:11:46 panamax-vm docker[10244]: Puma 2.8.2 starting...
Aug 14 19:11:46 panamax-vm docker[10244]: * Min threads: 0, max threads: 16
Aug 14 19:11:46 panamax-vm docker[10244]: * Environment: production
Aug 14 19:11:46 panamax-vm docker[10244]: * Listening on tcp://0.0.0.0:3000

Output of 'docker logs PMX_API' (complete)

core@panamax-vm ~ $ docker logs -t PMX_API
[Aug 14 19:11:36.812] db/mnt/production.sqlite3 already exists
[Aug 14 19:11:45.578] => Booting Puma
[Aug 14 19:11:45.578] => Rails 4.1.4 application starting in production on http://0.0.0.0:3000
[Aug 14 19:11:45.578] => Run `rails server -h` for more startup options
[Aug 14 19:11:45.578] => Notice: server is listening on all interfaces (0.0.0.0). Consider using 127.0.0.1 (--binding option)
[Aug 14 19:11:45.578] => Ctrl-C to shutdown server
[Aug 14 19:11:46.110] Puma 2.8.2 starting...
[Aug 14 19:11:46.110] * Min threads: 0, max threads: 16
[Aug 14 19:11:46.110] * Environment: production
[Aug 14 19:11:46.110] * Listening on tcp://0.0.0.0:3000

Output of 'docker logs PMX_UI' (truncated)

core@panamax-vm ~ $ docker logs -t PMX_UI 
[Aug 14 19:11:37.667] => Booting Puma
[Aug 14 19:11:37.667] => Rails 4.1.4 application starting in production on http://0.0.0.0:3000
[Aug 14 19:11:37.667] => Run `rails server -h` for more startup options
[Aug 14 19:11:37.667] => Notice: server is listening on all interfaces (0.0.0.0). Consider using 127.0.0.1 (--binding option)
[Aug 14 19:11:37.667] => Ctrl-C to shutdown server
[Aug 14 19:11:38.272] Puma 2.8.2 starting...
[Aug 14 19:11:38.273] * Min threads: 0, max threads: 16
[Aug 14 19:11:38.273] * Environment: production
[Aug 14 19:11:38.273] * Listening on tcp://0.0.0.0:3000
[Aug 14 19:11:43.732] I, [2014-08-14T19:11:43.732013 #8]  INFO -- : Started GET "/host_health" for 141.213.145.52 at 2014-08-14 19:11:43 +0000
[Aug 14 19:11:43.743] I, [2014-08-14T19:11:43.743161 #8]  INFO -- : Processing by HostHealthController#index as JSON
[Aug 14 19:11:43.774] I, [2014-08-14T19:11:43.773776 #8]  INFO -- : Completed 200 OK in 30ms (Views: 0.3ms)
<CONTINUES>

Tally of all 'main process exited' occurrences (some intended restarts)

Aug 13 23:41:19 panamax-vm systemd[1]: panamax-api.service: main process exited, code=exited, status=129/n/a
Aug 14 00:08:06 panamax-vm systemd[1]: panamax-api.service: main process exited, code=exited, status=1/FAILURE
Aug 14 00:17:26 panamax-vm systemd[1]: panamax-api.service: main process exited, code=exited, status=255/n/a
Aug 14 00:18:24 panamax-vm systemd[1]: panamax-api.service: main process exited, code=exited, status=255/n/a
Aug 14 00:26:44 panamax-vm systemd[1]: panamax-api.service: main process exited, code=exited, status=1/FAILURE
Aug 14 01:06:43 panamax-vm systemd[1]: panamax-api.service: main process exited, code=exited, status=129/n/a
Aug 14 01:24:58 panamax-vm systemd[1]: panamax-api.service: main process exited, code=exited, status=1/FAILURE
Aug 14 01:55:38 panamax-vm systemd[1]: panamax-api.service: main process exited, code=exited, status=1/FAILURE
Aug 14 05:03:57 panamax-vm systemd[1]: panamax-api.service: main process exited, code=exited, status=129/n/a
Aug 14 05:40:59 panamax-vm systemd[1]: panamax-api.service: main process exited, code=exited, status=1/FAILURE
Aug 14 05:44:07 panamax-vm systemd[1]: panamax-api.service: main process exited, code=exited, status=255/n/a
Aug 14 05:45:03 panamax-vm systemd[1]: panamax-api.service: main process exited, code=exited, status=255/n/a
Aug 14 06:18:49 panamax-vm systemd[1]: panamax-api.service: main process exited, code=exited, status=129/n/a
Aug 14 13:28:12 panamax-vm systemd[1]: panamax-api.service: main process exited, code=exited, status=255/n/a
Aug 14 13:42:30 panamax-vm systemd[1]: panamax-api.service: main process exited, code=exited, status=255/n/a
Aug 14 13:43:44 panamax-vm systemd[1]: panamax-api.service: main process exited, code=exited, status=2/INVALIDARGUMENT
Aug 14 14:13:33 panamax-vm systemd[1]: panamax-api.service: main process exited, code=exited, status=1/FAILURE
Aug 14 15:15:42 panamax-vm systemd[1]: panamax-api.service: main process exited, code=exited, status=2/INVALIDARGUMENT
Aug 14 16:02:34 panamax-vm systemd[1]: panamax-api.service: main process exited, code=exited, status=129/n/a
Aug 14 16:14:45 panamax-vm systemd[1]: panamax-api.service: main process exited, code=exited, status=255/n/a
Aug 14 18:37:28 panamax-vm systemd[1]: panamax-api.service: main process exited, code=exited, status=2/INVALIDARGUMENT
Aug 14 19:10:01 panamax-vm systemd[1]: panamax-api.service: main process exited, code=exited, status=1/FAILURE

Output of journalctl -u fleet as of last time API restarted

Aug 14 18:39:56 panamax-vm fleet[17747]: I0814 18:39:56.763806 17747 event.go:85] EventJobDestroyed(postgres.service): Job(postgres.service) unscheduled, deciding what to do
Aug 14 18:39:56 panamax-vm fleet[17747]: I0814 18:39:56.765539 17747 agent.go:628] Unloading Job(postgres.service)
Aug 14 18:39:56 panamax-vm fleet[17747]: I0814 18:39:56.785062 17747 agent.go:631] Checking outstanding JobOffers
Aug 14 18:39:56 panamax-vm fleet[17747]: I0814 18:39:56.789629 17747 event.go:98] EventUnitStateUpdated(postgres.service): reporting state to Registry
Aug 14 18:39:57 panamax-vm fleet[17747]: I0814 18:39:57.806832 17747 event.go:85] EventJobDestroyed(munkiwebadmin.service): Job(munkiwebadmin.service) unscheduled, deciding what to do
Aug 14 18:39:57 panamax-vm fleet[17747]: I0814 18:39:57.806879 17747 agent.go:628] Unloading Job(munkiwebadmin.service)
Aug 14 18:39:57 panamax-vm fleet[17747]: I0814 18:39:57.811471 17747 manager.go:114] Stopped systemd unit munkiwebadmin.service(done)
Aug 14 18:39:57 panamax-vm fleet[17747]: I0814 18:39:57.812876 17747 manager.go:174] Removing systemd unit munkiwebadmin.service
Aug 14 18:39:57 panamax-vm fleet[17747]: I0814 18:39:57.814546 17747 agent.go:631] Checking outstanding JobOffers
Aug 14 18:39:57 panamax-vm fleet[17747]: I0814 18:39:57.819776 17747 event.go:85] EventJobDestroyed(sal.service): Job(sal.service) unscheduled, deciding what to do
Aug 14 18:39:57 panamax-vm fleet[17747]: I0814 18:39:57.819801 17747 agent.go:628] Unloading Job(sal.service)
Aug 14 18:39:57 panamax-vm fleet[17747]: I0814 18:39:57.823712 17747 agent.go:631] Checking outstanding JobOffers
Aug 14 18:39:57 panamax-vm fleet[17747]: I0814 18:39:57.824995 17747 manager.go:137] Instructing systemd to reload units
Aug 14 18:39:57 panamax-vm fleet[17747]: I0814 18:39:57.825087 17747 manager.go:114] Stopped systemd unit sal.service(done)
Aug 14 18:39:57 panamax-vm fleet[17747]: I0814 18:39:57.825100 17747 manager.go:174] Removing systemd unit sal.service
Aug 14 18:39:57 panamax-vm fleet[17747]: I0814 18:39:57.838105 17747 event.go:85] EventJobDestroyed(munkireport-php.service): Job(munkireport-php.service) unscheduled, deciding what to do
Aug 14 18:39:57 panamax-vm fleet[17747]: I0814 18:39:57.838131 17747 agent.go:628] Unloading Job(munkireport-php.service)
Aug 14 18:39:57 panamax-vm fleet[17747]: I0814 18:39:57.844088 17747 agent.go:631] Checking outstanding JobOffers
Aug 14 18:39:57 panamax-vm fleet[17747]: I0814 18:39:57.849648 17747 event.go:85] EventJobDestroyed(bsdpy.service): Job(bsdpy.service) unscheduled, deciding what to do
Aug 14 18:39:57 panamax-vm fleet[17747]: I0814 18:39:57.849676 17747 agent.go:628] Unloading Job(bsdpy.service)
Aug 14 18:39:57 panamax-vm fleet[17747]: I0814 18:39:57.853697 17747 agent.go:631] Checking outstanding JobOffers
Aug 14 18:39:57 panamax-vm fleet[17747]: I0814 18:39:57.967159 17747 manager.go:137] Instructing systemd to reload units
Aug 14 18:39:58 panamax-vm fleet[17747]: I0814 18:39:58.260224 17747 manager.go:114] Stopped systemd unit munkireport-php.service(done)
Aug 14 18:39:58 panamax-vm fleet[17747]: I0814 18:39:58.260459 17747 manager.go:174] Removing systemd unit munkireport-php.service
Aug 14 18:39:58 panamax-vm fleet[17747]: I0814 18:39:58.261914 17747 manager.go:137] Instructing systemd to reload units
Aug 14 18:39:58 panamax-vm fleet[17747]: I0814 18:39:58.414413 17747 manager.go:114] Stopped systemd unit bsdpy.service(done)
Aug 14 18:39:58 panamax-vm fleet[17747]: I0814 18:39:58.415224 17747 manager.go:174] Removing systemd unit bsdpy.service
Aug 14 18:39:58 panamax-vm fleet[17747]: I0814 18:39:58.420342 17747 manager.go:137] Instructing systemd to reload units
Aug 14 18:39:58 panamax-vm fleet[17747]: I0814 18:39:58.902523 17747 event.go:29] CommandLoadJob(postgres.service): publishing JobOffer
Aug 14 18:39:58 panamax-vm fleet[17747]: I0814 18:39:58.914738 17747 engine.go:88] Published JobOffer(postgres.service)
Aug 14 18:39:58 panamax-vm fleet[17747]: I0814 18:39:58.920849 17747 event.go:27] EventJobOffered(postgres.service): deciding whether to bid or not
Aug 14 18:39:58 panamax-vm fleet[17747]: I0814 18:39:58.920929 17747 agent.go:481] Job(postgres.service) has requirements: map[]
Aug 14 18:39:58 panamax-vm fleet[17747]: I0814 18:39:58.920952 17747 agent.go:389] EventJobOffered(postgres.service): passed all criteria, submitting JobBid
Aug 14 18:39:58 panamax-vm fleet[17747]: I0814 18:39:58.920961 17747 agent.go:413] Submitting JobBid for Job(postgres.service)
Aug 14 18:39:58 panamax-vm fleet[17747]: I0814 18:39:58.926788 17747 engine.go:118] Scheduled Job(postgres.service) to Machine(350cb64408b94f16b7e236150643b71e)
Aug 14 18:39:58 panamax-vm fleet[17747]: I0814 18:39:58.927639 17747 event.go:82] EventJobBidSubmitted(postgres.service): successfully scheduled Job to Machine(350cb64408b94f16b7e236150643b71e)
Aug 14 18:39:58 panamax-vm fleet[17747]: I0814 18:39:58.930583 17747 event.go:39] EventJobScheduled(postgres.service): Job scheduled here, informing Agent
Aug 14 18:39:58 panamax-vm fleet[17747]: I0814 18:39:58.930646 17747 agent.go:585] Dropping offer and bid for Job(postgres.service) from cache
Aug 14 18:39:58 panamax-vm fleet[17747]: I0814 18:39:58.933838 17747 agent.go:481] Job(postgres.service) has requirements: map[]
Aug 14 18:39:58 panamax-vm fleet[17747]: I0814 18:39:58.933891 17747 agent.go:264] Loading Job(postgres.service)
Aug 14 18:39:58 panamax-vm fleet[17747]: I0814 18:39:58.933912 17747 manager.go:161] Writing systemd unit postgres.service
Aug 14 18:39:58 panamax-vm fleet[17747]: I0814 18:39:58.934689 17747 manager.go:137] Instructing systemd to reload units
Aug 14 18:39:58 panamax-vm fleet[17747]: I0814 18:39:58.994659 17747 agent.go:603] Bidding for all possible peers of Job(postgres.service)
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.001720 17747 event.go:29] CommandLoadJob(munkiwebadmin.service): publishing JobOffer
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.003870 17747 engine.go:88] Published JobOffer(munkiwebadmin.service)
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.017753 17747 event.go:27] EventJobOffered(munkiwebadmin.service): deciding whether to bid or not
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.017824 17747 agent.go:481] Job(munkiwebadmin.service) has requirements: map[]
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.017842 17747 agent.go:389] EventJobOffered(munkiwebadmin.service): passed all criteria, submitting JobBid
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.017850 17747 agent.go:413] Submitting JobBid for Job(munkiwebadmin.service)
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.025421 17747 engine.go:118] Scheduled Job(munkiwebadmin.service) to Machine(350cb64408b94f16b7e236150643b71e)
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.027143 17747 event.go:82] EventJobBidSubmitted(munkiwebadmin.service): successfully scheduled Job to Machine(350cb64408b94f16b7e236150643b71e)
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.030089 17747 event.go:39] EventJobScheduled(munkiwebadmin.service): Job scheduled here, informing Agent
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.030139 17747 agent.go:585] Dropping offer and bid for Job(munkiwebadmin.service) from cache
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.033386 17747 agent.go:481] Job(munkiwebadmin.service) has requirements: map[]
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.033445 17747 agent.go:264] Loading Job(munkiwebadmin.service)
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.033458 17747 manager.go:161] Writing systemd unit munkiwebadmin.service
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.035683 17747 manager.go:137] Instructing systemd to reload units
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.168600 17747 agent.go:603] Bidding for all possible peers of Job(munkiwebadmin.service)
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.219921 17747 event.go:29] CommandLoadJob(sal.service): publishing JobOffer
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.222638 17747 engine.go:88] Published JobOffer(sal.service)
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.232425 17747 event.go:27] EventJobOffered(sal.service): deciding whether to bid or not
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.232479 17747 agent.go:481] Job(sal.service) has requirements: map[]
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.232497 17747 agent.go:389] EventJobOffered(sal.service): passed all criteria, submitting JobBid
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.232505 17747 agent.go:413] Submitting JobBid for Job(sal.service)
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.237922 17747 engine.go:118] Scheduled Job(sal.service) to Machine(350cb64408b94f16b7e236150643b71e)
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.238466 17747 event.go:82] EventJobBidSubmitted(sal.service): successfully scheduled Job to Machine(350cb64408b94f16b7e236150643b71e)
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.239797 17747 event.go:39] EventJobScheduled(sal.service): Job scheduled here, informing Agent
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.239817 17747 agent.go:585] Dropping offer and bid for Job(sal.service) from cache
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.242926 17747 agent.go:481] Job(sal.service) has requirements: map[]
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.242982 17747 agent.go:264] Loading Job(sal.service)
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.242996 17747 manager.go:161] Writing systemd unit sal.service
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.244892 17747 manager.go:137] Instructing systemd to reload units
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.301524 17747 agent.go:603] Bidding for all possible peers of Job(sal.service)
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.370749 17747 event.go:29] CommandLoadJob(munkireport-php.service): publishing JobOffer
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.372542 17747 engine.go:88] Published JobOffer(munkireport-php.service)
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.375516 17747 event.go:27] EventJobOffered(munkireport-php.service): deciding whether to bid or not
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.375569 17747 agent.go:481] Job(munkireport-php.service) has requirements: map[]
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.375587 17747 agent.go:389] EventJobOffered(munkireport-php.service): passed all criteria, submitting JobBid
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.375595 17747 agent.go:413] Submitting JobBid for Job(munkireport-php.service)
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.380691 17747 engine.go:118] Scheduled Job(munkireport-php.service) to Machine(350cb64408b94f16b7e236150643b71e)
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.381252 17747 event.go:82] EventJobBidSubmitted(munkireport-php.service): successfully scheduled Job to Machine(350cb64408b94f16b7e236150643b71e)
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.382778 17747 event.go:39] EventJobScheduled(munkireport-php.service): Job scheduled here, informing Agent
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.382802 17747 agent.go:585] Dropping offer and bid for Job(munkireport-php.service) from cache
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.387011 17747 agent.go:481] Job(munkireport-php.service) has requirements: map[]
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.387018 17747 agent.go:264] Loading Job(munkireport-php.service)
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.387044 17747 manager.go:161] Writing systemd unit munkireport-php.service
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.388464 17747 manager.go:137] Instructing systemd to reload units
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.444841 17747 agent.go:603] Bidding for all possible peers of Job(munkireport-php.service)
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.495085 17747 event.go:29] CommandLoadJob(bsdpy.service): publishing JobOffer
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.498332 17747 engine.go:88] Published JobOffer(bsdpy.service)
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.502676 17747 event.go:27] EventJobOffered(bsdpy.service): deciding whether to bid or not
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.502751 17747 agent.go:481] Job(bsdpy.service) has requirements: map[]
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.502768 17747 agent.go:389] EventJobOffered(bsdpy.service): passed all criteria, submitting JobBid
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.502776 17747 agent.go:413] Submitting JobBid for Job(bsdpy.service)
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.507061 17747 engine.go:118] Scheduled Job(bsdpy.service) to Machine(350cb64408b94f16b7e236150643b71e)
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.507624 17747 event.go:82] EventJobBidSubmitted(bsdpy.service): successfully scheduled Job to Machine(350cb64408b94f16b7e236150643b71e)
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.509359 17747 event.go:39] EventJobScheduled(bsdpy.service): Job scheduled here, informing Agent
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.509386 17747 agent.go:585] Dropping offer and bid for Job(bsdpy.service) from cache
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.512538 17747 agent.go:481] Job(bsdpy.service) has requirements: map[]
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.512568 17747 agent.go:264] Loading Job(bsdpy.service)
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.512573 17747 manager.go:161] Writing systemd unit bsdpy.service
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.513697 17747 manager.go:137] Instructing systemd to reload units
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.567930 17747 agent.go:603] Bidding for all possible peers of Job(bsdpy.service)
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.614543 17747 event.go:53] CommandStartJob(postgres.service): instructing Agent to start Job
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.618482 17747 manager.go:114] Stopped systemd unit postgres.service(canceled)
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.618512 17747 manager.go:174] Removing systemd unit postgres.service
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.620474 17747 manager.go:137] Instructing systemd to reload units
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.624824 17747 event.go:53] CommandStartJob(munkiwebadmin.service): instructing Agent to start Job
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.633117 17747 event.go:53] CommandStartJob(sal.service): instructing Agent to start Job
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.640200 17747 event.go:53] CommandStartJob(munkireport-php.service): instructing Agent to start Job
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.650872 17747 event.go:53] CommandStartJob(bsdpy.service): instructing Agent to start Job
Aug 14 18:39:59 panamax-vm fleet[17747]: E0814 18:39:59.718864 17747 manager.go:104] Failed to start systemd unit munkiwebadmin.service: Unit postgres.service failed to load: No such file or directory.
Aug 14 18:39:59 panamax-vm fleet[17747]: E0814 18:39:59.721130 17747 manager.go:104] Failed to start systemd unit sal.service: Unit postgres.service failed to load: No such file or directory.
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.783467 17747 manager.go:106] Started systemd unit postgres.service(failed)
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.790570 17747 event.go:98] EventUnitStateUpdated(bsdpy.service): reporting state to Registry
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.794911 17747 event.go:98] EventUnitStateUpdated(munkireport-php.service): reporting state to Registry
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.801616 17747 event.go:98] EventUnitStateUpdated(sal.service): reporting state to Registry
Aug 14 18:39:59 panamax-vm fleet[17747]: I0814 18:39:59.805991 17747 event.go:98] EventUnitStateUpdated(munkiwebadmin.service): reporting state to Registry
Aug 14 18:40:02 panamax-vm fleet[17747]: I0814 18:40:02.415756 17747 manager.go:106] Started systemd unit bsdpy.service(done)
Aug 14 18:40:02 panamax-vm fleet[17747]: I0814 18:40:02.429460 17747 manager.go:106] Started systemd unit munkireport-php.service(done)
Aug 14 18:40:02 panamax-vm fleet[17747]: I0814 18:40:02.795406 17747 event.go:98] EventUnitStateUpdated(bsdpy.service): reporting state to Registry
Aug 14 18:40:02 panamax-vm fleet[17747]: I0814 18:40:02.797770 17747 event.go:98] EventUnitStateUpdated(munkireport-php.service): reporting state to Registry
Aug 14 18:41:34 panamax-vm fleet[17747]: I0814 18:41:34.649950 17747 event.go:85] EventJobDestroyed(postgres.service): Job(postgres.service) unscheduled, deciding what to do
Aug 14 18:41:34 panamax-vm fleet[17747]: I0814 18:41:34.651585 17747 agent.go:628] Unloading Job(postgres.service)
Aug 14 18:41:34 panamax-vm fleet[17747]: I0814 18:41:34.672568 17747 manager.go:114] Stopped systemd unit postgres.service(done)
Aug 14 18:41:34 panamax-vm fleet[17747]: I0814 18:41:34.672610 17747 manager.go:174] Removing systemd unit postgres.service
Aug 14 18:41:34 panamax-vm fleet[17747]: I0814 18:41:34.682459 17747 manager.go:137] Instructing systemd to reload units
Aug 14 18:41:34 panamax-vm fleet[17747]: I0814 18:41:34.683102 17747 agent.go:631] Checking outstanding JobOffers
Aug 14 18:41:34 panamax-vm fleet[17747]: I0814 18:41:34.748495 17747 event.go:85] EventJobDestroyed(munkiwebadmin.service): Job(munkiwebadmin.service) unscheduled, deciding what to do
Aug 14 18:41:34 panamax-vm fleet[17747]: I0814 18:41:34.748523 17747 agent.go:628] Unloading Job(munkiwebadmin.service)
Aug 14 18:41:34 panamax-vm fleet[17747]: I0814 18:41:34.751775 17747 agent.go:631] Checking outstanding JobOffers
Aug 14 18:41:34 panamax-vm fleet[17747]: I0814 18:41:34.767334 17747 event.go:85] EventJobDestroyed(sal.service): Job(sal.service) unscheduled, deciding what to do
Aug 14 18:41:34 panamax-vm fleet[17747]: I0814 18:41:34.767367 17747 agent.go:628] Unloading Job(sal.service)
Aug 14 18:41:34 panamax-vm fleet[17747]: I0814 18:41:34.770489 17747 agent.go:631] Checking outstanding JobOffers
Aug 14 18:41:34 panamax-vm fleet[17747]: I0814 18:41:34.779436 17747 event.go:85] EventJobDestroyed(munkireport-php.service): Job(munkireport-php.service) unscheduled, deciding what to do
Aug 14 18:41:34 panamax-vm fleet[17747]: I0814 18:41:34.779464 17747 agent.go:628] Unloading Job(munkireport-php.service)
Aug 14 18:41:34 panamax-vm fleet[17747]: I0814 18:41:34.783118 17747 agent.go:631] Checking outstanding JobOffers
Aug 14 18:41:34 panamax-vm fleet[17747]: I0814 18:41:34.794966 17747 event.go:85] EventJobDestroyed(bsdpy.service): Job(bsdpy.service) unscheduled, deciding what to do
Aug 14 18:41:34 panamax-vm fleet[17747]: I0814 18:41:34.794992 17747 agent.go:628] Unloading Job(bsdpy.service)
Aug 14 18:41:34 panamax-vm fleet[17747]: I0814 18:41:34.799632 17747 agent.go:631] Checking outstanding JobOffers
Aug 14 18:41:34 panamax-vm fleet[17747]: I0814 18:41:34.945923 17747 manager.go:114] Stopped systemd unit munkiwebadmin.service(done)
Aug 14 18:41:34 panamax-vm fleet[17747]: I0814 18:41:34.945949 17747 manager.go:174] Removing systemd unit munkiwebadmin.service
Aug 14 18:41:34 panamax-vm fleet[17747]: I0814 18:41:34.951543 17747 manager.go:114] Stopped systemd unit sal.service(done)
Aug 14 18:41:34 panamax-vm fleet[17747]: I0814 18:41:34.951579 17747 manager.go:174] Removing systemd unit sal.service
Aug 14 18:41:34 panamax-vm fleet[17747]: I0814 18:41:34.954418 17747 manager.go:137] Instructing systemd to reload units
Aug 14 18:41:34 panamax-vm fleet[17747]: I0814 18:41:34.966998 17747 manager.go:137] Instructing systemd to reload units
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.211560 17747 event.go:98] EventUnitStateUpdated(bsdpy.service): reporting state to Registry
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.214573 17747 event.go:98] EventUnitStateUpdated(munkireport-php.service): reporting state to Registry
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.362168 17747 manager.go:114] Stopped systemd unit bsdpy.service(done)
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.362196 17747 manager.go:174] Removing systemd unit bsdpy.service
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.363738 17747 manager.go:137] Instructing systemd to reload units
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.364486 17747 manager.go:114] Stopped systemd unit munkireport-php.service(done)
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.364514 17747 manager.go:174] Removing systemd unit munkireport-php.service
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.431092 17747 manager.go:137] Instructing systemd to reload units
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.825540 17747 event.go:29] CommandLoadJob(postgres.service): publishing JobOffer
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.828336 17747 engine.go:88] Published JobOffer(postgres.service)
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.843101 17747 event.go:27] EventJobOffered(postgres.service): deciding whether to bid or not
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.843187 17747 agent.go:481] Job(postgres.service) has requirements: map[]
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.843212 17747 agent.go:389] EventJobOffered(postgres.service): passed all criteria, submitting JobBid
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.843224 17747 agent.go:413] Submitting JobBid for Job(postgres.service)
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.852250 17747 engine.go:118] Scheduled Job(postgres.service) to Machine(350cb64408b94f16b7e236150643b71e)
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.852917 17747 event.go:82] EventJobBidSubmitted(postgres.service): successfully scheduled Job to Machine(350cb64408b94f16b7e236150643b71e)
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.855572 17747 event.go:39] EventJobScheduled(postgres.service): Job scheduled here, informing Agent
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.855634 17747 agent.go:585] Dropping offer and bid for Job(postgres.service) from cache
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.859675 17747 agent.go:481] Job(postgres.service) has requirements: map[]
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.859738 17747 agent.go:264] Loading Job(postgres.service)
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.859753 17747 manager.go:161] Writing systemd unit postgres.service
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.861771 17747 manager.go:137] Instructing systemd to reload units
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.920420 17747 agent.go:603] Bidding for all possible peers of Job(postgres.service)
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.949019 17747 event.go:29] CommandLoadJob(munkiwebadmin.service): publishing JobOffer
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.952246 17747 engine.go:88] Published JobOffer(munkiwebadmin.service)
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.956168 17747 event.go:27] EventJobOffered(munkiwebadmin.service): deciding whether to bid or not
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.956409 17747 agent.go:481] Job(munkiwebadmin.service) has requirements: map[]
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.956439 17747 agent.go:389] EventJobOffered(munkiwebadmin.service): passed all criteria, submitting JobBid
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.956454 17747 agent.go:413] Submitting JobBid for Job(munkiwebadmin.service)
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.960748 17747 engine.go:118] Scheduled Job(munkiwebadmin.service) to Machine(350cb64408b94f16b7e236150643b71e)
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.961333 17747 event.go:82] EventJobBidSubmitted(munkiwebadmin.service): successfully scheduled Job to Machine(350cb64408b94f16b7e236150643b71e)
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.962531 17747 event.go:39] EventJobScheduled(munkiwebadmin.service): Job scheduled here, informing Agent
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.962554 17747 agent.go:585] Dropping offer and bid for Job(munkiwebadmin.service) from cache
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.966154 17747 agent.go:481] Job(munkiwebadmin.service) has requirements: map[]
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.966184 17747 agent.go:264] Loading Job(munkiwebadmin.service)
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.966194 17747 manager.go:161] Writing systemd unit munkiwebadmin.service
Aug 14 18:41:35 panamax-vm fleet[17747]: I0814 18:41:35.967180 17747 manager.go:137] Instructing systemd to reload units
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.037340 17747 event.go:98] EventUnitStateUpdated(postgres.service): reporting state to Registry
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.038524 17747 event.go:98] EventUnitStateUpdated(munkiwebadmin.service): reporting state to Registry
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.038632 17747 agent.go:603] Bidding for all possible peers of Job(munkiwebadmin.service)
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.089186 17747 event.go:29] CommandLoadJob(sal.service): publishing JobOffer
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.092353 17747 engine.go:88] Published JobOffer(sal.service)
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.097091 17747 event.go:27] EventJobOffered(sal.service): deciding whether to bid or not
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.097865 17747 agent.go:481] Job(sal.service) has requirements: map[]
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.097890 17747 agent.go:389] EventJobOffered(sal.service): passed all criteria, submitting JobBid
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.097899 17747 agent.go:413] Submitting JobBid for Job(sal.service)
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.114941 17747 engine.go:118] Scheduled Job(sal.service) to Machine(350cb64408b94f16b7e236150643b71e)
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.116692 17747 event.go:82] EventJobBidSubmitted(sal.service): successfully scheduled Job to Machine(350cb64408b94f16b7e236150643b71e)
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.124032 17747 event.go:39] EventJobScheduled(sal.service): Job scheduled here, informing Agent
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.124102 17747 agent.go:585] Dropping offer and bid for Job(sal.service) from cache
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.129149 17747 agent.go:481] Job(sal.service) has requirements: map[]
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.129370 17747 agent.go:264] Loading Job(sal.service)
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.129387 17747 manager.go:161] Writing systemd unit sal.service
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.131580 17747 manager.go:137] Instructing systemd to reload units
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.217327 17747 agent.go:603] Bidding for all possible peers of Job(sal.service)
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.346009 17747 event.go:29] CommandLoadJob(munkireport-php.service): publishing JobOffer
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.349373 17747 engine.go:88] Published JobOffer(munkireport-php.service)
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.354639 17747 event.go:27] EventJobOffered(munkireport-php.service): deciding whether to bid or not
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.355742 17747 agent.go:481] Job(munkireport-php.service) has requirements: map[]
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.357188 17747 agent.go:389] EventJobOffered(munkireport-php.service): passed all criteria, submitting JobBid
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.358250 17747 agent.go:413] Submitting JobBid for Job(munkireport-php.service)
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.366353 17747 engine.go:118] Scheduled Job(munkireport-php.service) to Machine(350cb64408b94f16b7e236150643b71e)
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.368430 17747 event.go:82] EventJobBidSubmitted(munkireport-php.service): successfully scheduled Job to Machine(350cb64408b94f16b7e236150643b71e)
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.373266 17747 event.go:39] EventJobScheduled(munkireport-php.service): Job scheduled here, informing Agent
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.374544 17747 agent.go:585] Dropping offer and bid for Job(munkireport-php.service) from cache
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.379804 17747 agent.go:481] Job(munkireport-php.service) has requirements: map[]
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.379869 17747 agent.go:264] Loading Job(munkireport-php.service)
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.379889 17747 manager.go:161] Writing systemd unit munkireport-php.service
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.381832 17747 manager.go:137] Instructing systemd to reload units
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.445072 17747 agent.go:603] Bidding for all possible peers of Job(munkireport-php.service)
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.445624 17747 agent.go:611] Job(munkireport-php.service) loaded, now starting it
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.449828 17747 event.go:29] CommandLoadJob(bsdpy.service): publishing JobOffer
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.450781 17747 engine.go:88] Published JobOffer(bsdpy.service)
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.457478 17747 event.go:53] CommandStartJob(postgres.service): instructing Agent to start Job
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.467798 17747 event.go:53] CommandStartJob(munkiwebadmin.service): instructing Agent to start Job
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.473524 17747 event.go:53] CommandStartJob(sal.service): instructing Agent to start Job
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.490781 17747 event.go:29] CommandLoadJob(bsdpy.service): publishing JobOffer
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.492339 17747 engine.go:88] Published JobOffer(bsdpy.service)
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.497652 17747 event.go:27] EventJobOffered(bsdpy.service): deciding whether to bid or not
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.498383 17747 agent.go:481] Job(bsdpy.service) has requirements: map[]
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.499248 17747 agent.go:389] EventJobOffered(bsdpy.service): passed all criteria, submitting JobBid
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.499744 17747 agent.go:413] Submitting JobBid for Job(bsdpy.service)
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.519673 17747 event.go:27] EventJobOffered(bsdpy.service): deciding whether to bid or not
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.519769 17747 agent.go:481] Job(bsdpy.service) has requirements: map[]
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.519783 17747 agent.go:389] EventJobOffered(bsdpy.service): passed all criteria, submitting JobBid
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.519791 17747 agent.go:413] Submitting JobBid for Job(bsdpy.service)
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.524401 17747 engine.go:118] Scheduled Job(bsdpy.service) to Machine(350cb64408b94f16b7e236150643b71e)
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.525062 17747 event.go:82] EventJobBidSubmitted(bsdpy.service): successfully scheduled Job to Machine(350cb64408b94f16b7e236150643b71e)
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.526874 17747 event.go:84] EventJobBidSubmitted(bsdpy.service): failed to schedule Job to Machine(350cb64408b94f16b7e236150643b71e)
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.529131 17747 event.go:39] EventJobScheduled(bsdpy.service): Job scheduled here, informing Agent
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.529148 17747 agent.go:585] Dropping offer and bid for Job(bsdpy.service) from cache
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.533369 17747 agent.go:481] Job(bsdpy.service) has requirements: map[]
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.533478 17747 agent.go:264] Loading Job(bsdpy.service)
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.533492 17747 manager.go:161] Writing systemd unit bsdpy.service
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.561399 17747 manager.go:137] Instructing systemd to reload units
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.665716 17747 agent.go:603] Bidding for all possible peers of Job(bsdpy.service)
Aug 14 18:41:36 panamax-vm fleet[17747]: I0814 18:41:36.666538 17747 agent.go:611] Job(bsdpy.service) loaded, now starting it
Aug 14 18:41:37 panamax-vm fleet[17747]: I0814 18:41:37.000387 17747 event.go:98] EventUnitStateUpdated(bsdpy.service): reporting state to Registry
Aug 14 18:41:37 panamax-vm fleet[17747]: I0814 18:41:37.003058 17747 event.go:98] EventUnitStateUpdated(munkireport-php.service): reporting state to Registry
Aug 14 18:41:37 panamax-vm fleet[17747]: I0814 18:41:37.006412 17747 event.go:98] EventUnitStateUpdated(postgres.service): reporting state to Registry
Aug 14 18:41:37 panamax-vm fleet[17747]: I0814 18:41:37.008384 17747 event.go:98] EventUnitStateUpdated(sal.service): reporting state to Registry
Aug 14 18:41:37 panamax-vm fleet[17747]: I0814 18:41:37.009808 17747 event.go:98] EventUnitStateUpdated(munkiwebadmin.service): reporting state to Registry
Aug 14 18:41:38 panamax-vm fleet[17747]: I0814 18:41:38.520506 17747 manager.go:106] Started systemd unit bsdpy.service(done)
Aug 14 18:41:39 panamax-vm fleet[17747]: I0814 18:41:39.005098 17747 event.go:98] EventUnitStateUpdated(bsdpy.service): reporting state to Registry
Aug 14 18:41:39 panamax-vm fleet[17747]: I0814 18:41:39.222175 17747 manager.go:106] Started systemd unit munkireport-php.service(done)
Aug 14 18:41:39 panamax-vm fleet[17747]: I0814 18:41:39.508429 17747 manager.go:106] Started systemd unit postgres.service(failed)
Aug 14 18:41:39 panamax-vm fleet[17747]: I0814 18:41:39.514820 17747 manager.go:106] Started systemd unit sal.service(dependency)
Aug 14 18:41:39 panamax-vm fleet[17747]: I0814 18:41:39.519172 17747 manager.go:106] Started systemd unit munkiwebadmin.service(dependency)
Aug 14 18:41:40 panamax-vm fleet[17747]: I0814 18:41:40.006132 17747 event.go:98] EventUnitStateUpdated(munkireport-php.service): reporting state to Registry
Aug 14 18:41:40 panamax-vm fleet[17747]: I0814 18:41:40.007790 17747 event.go:98] EventUnitStateUpdated(postgres.service): reporting state to Registry
Aug 14 18:41:40 panamax-vm fleet[17747]: I0814 18:41:40.008878 17747 event.go:98] EventUnitStateUpdated(sal.service): reporting state to Registry
Aug 14 18:41:40 panamax-vm fleet[17747]: I0814 18:41:40.010332 17747 event.go:98] EventUnitStateUpdated(munkiwebadmin.service): reporting state to Registry
Aug 14 18:41:50 panamax-vm fleet[17747]: I0814 18:41:50.009884 17747 event.go:98] EventUnitStateUpdated(postgres.service): reporting state to Registry
Aug 14 18:41:50 panamax-vm fleet[17747]: I0814 18:41:50.013480 17747 event.go:98] EventUnitStateUpdated(sal.service): reporting state to Registry
Aug 14 18:41:50 panamax-vm fleet[17747]: I0814 18:41:50.018798 17747 event.go:98] EventUnitStateUpdated(munkiwebadmin.service): reporting state to Registry
Aug 14 18:41:52 panamax-vm fleet[17747]: I0814 18:41:52.041059 17747 event.go:98] EventUnitStateUpdated(postgres.service): reporting state to Registry
Aug 14 18:41:52 panamax-vm fleet[17747]: I0814 18:41:52.044482 17747 event.go:98] EventUnitStateUpdated(sal.service): reporting state to Registry
Aug 14 18:41:52 panamax-vm fleet[17747]: I0814 18:41:52.047108 17747 event.go:98] EventUnitStateUpdated(munkiwebadmin.service): reporting state to Registry
Aug 14 18:41:53 panamax-vm fleet[17747]: I0814 18:41:53.028797 17747 event.go:98] EventUnitStateUpdated(sal.service): reporting state to Registry
Aug 14 18:41:54 panamax-vm fleet[17747]: I0814 18:41:54.049446 17747 event.go:98] EventUnitStateUpdated(sal.service): reporting state to Registry
Aug 14 18:41:54 panamax-vm fleet[17747]: I0814 18:41:54.061354 17747 event.go:98] EventUnitStateUpdated(munkiwebadmin.service): reporting state to Registry

Running containers stop, fleetctl is empty, getting 'Job state 'loaded' could not be achieved' in UI

I've been trying to setup a basic app with a few (3-4) services and keep running into a complete shutdown of all Docker containers, including the three PMX-related ones. After all containers shut down the PMX ones eventually come back up but fleetctl list-units no longer has any services listed and the UI responds to any service modifications with "Job state 'loaded' could not be achieved".

I assume the PMX containers come up because they're separate systemd entries whereas PMX services are entered into fleet. Restarting the fleet service after this meltdown doesn't restore the services in my app.

I am not quite certain what logging there may be to determine what causes Docker to shut all containers down without warning, but I'll poke around more to see if CoreOS logs anything pertinent.

Let me know if I can pull anything more specific to narrow down.

This trace is from the PMX_API container logs related to the job state load failure:

I, [2014-08-13T14:47:48.243606 #23]  INFO -- : Started PUT "/apps/2/services/17.json" for 172.17.0.5 at 2014-08-13 14:47:48 +0000
I, [2014-08-13T14:47:48.245228 #23]  INFO -- : Processing by ServicesController#update as JSON
I, [2014-08-13T14:47:48.245491 #23]  INFO -- :   Parameters: <PARAMATERS>
W, [2014-08-13T14:47:48.246032 #23]  WARN -- : Can't verify CSRF token authenticity
E, [2014-08-13T14:47:50.412362 #23] ERROR -- : 
PanamaxAgent::Error (Job state 'loaded' could not be achieved):
    /usr/src/app/lib/panamax_agent/fleet/client.rb:92:in `wait_for_load_state'
    /usr/src/app/lib/panamax_agent/fleet/client.rb:39:in `load'
    /usr/src/app/app/services/service_manager.rb:16:in `load'
    /usr/src/app/app/models/service.rb:49:in `submit'
    /usr/local/lib/ruby/gems/2.1.0/gems/activerecord-4.1.4/lib/active_record/relation/delegation.rb:46:in `each'
    /usr/local/lib/ruby/gems/2.1.0/gems/activerecord-4.1.4/lib/active_record/relation/delegation.rb:46:in `each'
    /usr/src/app/app/models/app.rb:17:in `restart'
    /usr/src/app/app/controllers/services_controller.rb:21:in `update'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/abstract_controller/base.rb:189:in `process_action'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal/rendering.rb:10:in `process_action'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/abstract_controller/callbacks.rb:20:in `block in process_action'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:113:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:113:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:229:in `block in halting'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:166:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:166:in `block in halting'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:86:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:86:in `run_callbacks'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/abstract_controller/callbacks.rb:19:in `process_action'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal/rescue.rb:29:in `process_action'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal/instrumentation.rb:31:in `block in process_action'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/notifications.rb:159:in `block in instrument'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/notifications.rb:159:in `instrument'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal/instrumentation.rb:30:in `process_action'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal/params_wrapper.rb:250:in `process_action'
    /usr/local/lib/ruby/gems/2.1.0/gems/activerecord-4.1.4/lib/active_record/railties/controller_runtime.rb:18:in `process_action'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/abstract_controller/base.rb:136:in `process'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionview-4.1.4/lib/action_view/rendering.rb:30:in `process'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal.rb:196:in `dispatch'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal/rack_delegation.rb:13:in `dispatch'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal.rb:232:in `block in action'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/routing/route_set.rb:82:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/routing/route_set.rb:82:in `dispatch'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/routing/route_set.rb:50:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/journey/router.rb:71:in `block in call'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/journey/router.rb:59:in `each'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/journey/router.rb:59:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/routing/route_set.rb:678:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/etag.rb:23:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/conditionalget.rb:35:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/head.rb:11:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/params_parser.rb:27:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/flash.rb:254:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/session/abstract/id.rb:225:in `context'
    /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/session/abstract/id.rb:220:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/cookies.rb:560:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/activerecord-4.1.4/lib/active_record/query_cache.rb:36:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/activerecord-4.1.4/lib/active_record/connection_adapters/abstract/connection_pool.rb:621:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:82:in `run_callbacks'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/remote_ip.rb:76:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/railties-4.1.4/lib/rails/rack/logger.rb:38:in `call_app'
    /usr/local/lib/ruby/gems/2.1.0/gems/railties-4.1.4/lib/rails/rack/logger.rb:20:in `block in call'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/tagged_logging.rb:68:in `block in tagged'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/tagged_logging.rb:26:in `tagged'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/tagged_logging.rb:68:in `tagged'
    /usr/local/lib/ruby/gems/2.1.0/gems/railties-4.1.4/lib/rails/rack/logger.rb:20:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/request_id.rb:21:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/methodoverride.rb:21:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/runtime.rb:17:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/cache/strategy/local_cache_middleware.rb:26:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/sendfile.rb:112:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/railties-4.1.4/lib/rails/engine.rb:514:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/railties-4.1.4/lib/rails/application.rb:144:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/content_length.rb:14:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/puma-2.8.2/lib/puma/server.rb:490:in `handle_request'
    /usr/local/lib/ruby/gems/2.1.0/gems/puma-2.8.2/lib/puma/server.rb:361:in `process_client'
    /usr/local/lib/ruby/gems/2.1.0/gems/puma-2.8.2/lib/puma/server.rb:254:in `block in run'
    /usr/local/lib/ruby/gems/2.1.0/gems/puma-2.8.2/lib/puma/thread_pool.rb:92:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/puma-2.8.2/lib/puma/thread_pool.rb:92:in `block in spawn_thread'

There's also an instance of this error in the PMX_API log:

Faraday::Error::TimeoutError (Net::ReadTimeout):
    /usr/local/lib/ruby/2.1.0/net/protocol.rb:158:in `rescue in rbuf_fill'
    /usr/local/lib/ruby/2.1.0/net/protocol.rb:152:in `rbuf_fill'
    /usr/local/lib/ruby/2.1.0/net/protocol.rb:122:in `read_all'
    /usr/local/lib/ruby/2.1.0/net/http/response.rb:292:in `block in read_body_0'
    /usr/local/lib/ruby/2.1.0/net/http/response.rb:250:in `inflater'
    /usr/local/lib/ruby/2.1.0/net/http/response.rb:274:in `read_body_0'
    /usr/local/lib/ruby/2.1.0/net/http/response.rb:201:in `read_body'
    /usr/local/lib/ruby/2.1.0/net/http.rb:1129:in `block in get'
    /usr/local/lib/ruby/2.1.0/net/http.rb:1415:in `block (2 levels) in transport_request'
    /usr/local/lib/ruby/2.1.0/net/http/response.rb:162:in `reading_body'
    /usr/local/lib/ruby/2.1.0/net/http.rb:1414:in `block in transport_request'
    /usr/local/lib/ruby/2.1.0/net/http.rb:1405:in `catch'
    /usr/local/lib/ruby/2.1.0/net/http.rb:1405:in `transport_request'
    /usr/local/lib/ruby/2.1.0/net/http.rb:1378:in `request'
    /usr/local/lib/ruby/2.1.0/net/http.rb:1371:in `block in request'
    /usr/local/lib/ruby/2.1.0/net/http.rb:853:in `start'
    /usr/local/lib/ruby/2.1.0/net/http.rb:1369:in `request'
    /usr/local/lib/ruby/2.1.0/net/http.rb:1128:in `get'
    /usr/local/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/adapter/net_http.rb:73:in `perform_request'
    /usr/local/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/adapter/net_http.rb:38:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/response.rb:8:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/faraday_middleware-0.9.0/lib/faraday_middleware/response_middleware.rb:30:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/faraday_middleware-0.9.0/lib/faraday_middleware/request/encode_json.rb:23:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/connection.rb:253:in `run_request'
    /usr/local/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/connection.rb:106:in `get'
    /usr/src/app/lib/panamax_agent/request.rb:15:in `request'
    /usr/src/app/lib/panamax_agent/request.rb:6:in `block (2 levels) in <module:Request>'
    /usr/src/app/lib/panamax_agent/journal/client/entries.rb:21:in `get_entries_by_fields'
    /usr/src/app/lib/panamax_agent/journal/client.rb:33:in `list_journal_entries'
    /usr/src/app/app/models/concerns/app_journal.rb:6:in `journal'
    /usr/src/app/app/controllers/apps_controller.rb:43:in `journal'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/abstract_controller/base.rb:189:in `process_action'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal/rendering.rb:10:in `process_action'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/abstract_controller/callbacks.rb:20:in `block in process_action'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:113:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:113:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:229:in `block in halting'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:166:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:166:in `block in halting'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:86:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:86:in `run_callbacks'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/abstract_controller/callbacks.rb:19:in `process_action'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal/rescue.rb:29:in `process_action'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal/instrumentation.rb:31:in `block in process_action'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/notifications.rb:159:in `block in instrument'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/notifications.rb:159:in `instrument'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal/instrumentation.rb:30:in `process_action'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal/params_wrapper.rb:250:in `process_action'
    /usr/local/lib/ruby/gems/2.1.0/gems/activerecord-4.1.4/lib/active_record/railties/controller_runtime.rb:18:in `process_action'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/abstract_controller/base.rb:136:in `process'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionview-4.1.4/lib/action_view/rendering.rb:30:in `process'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal.rb:196:in `dispatch'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal/rack_delegation.rb:13:in `dispatch'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_controller/metal.rb:232:in `block in action'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/routing/route_set.rb:82:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/routing/route_set.rb:82:in `dispatch'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/routing/route_set.rb:50:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/journey/router.rb:71:in `block in call'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/journey/router.rb:59:in `each'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/journey/router.rb:59:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/routing/route_set.rb:678:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/etag.rb:23:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/conditionalget.rb:25:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/head.rb:11:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/params_parser.rb:27:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/flash.rb:254:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/session/abstract/id.rb:225:in `context'
    /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/session/abstract/id.rb:220:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/cookies.rb:560:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/activerecord-4.1.4/lib/active_record/query_cache.rb:36:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/activerecord-4.1.4/lib/active_record/connection_adapters/abstract/connection_pool.rb:621:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/callbacks.rb:82:in `run_callbacks'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/remote_ip.rb:76:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/railties-4.1.4/lib/rails/rack/logger.rb:38:in `call_app'
    /usr/local/lib/ruby/gems/2.1.0/gems/railties-4.1.4/lib/rails/rack/logger.rb:20:in `block in call'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/tagged_logging.rb:68:in `block in tagged'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/tagged_logging.rb:26:in `tagged'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/tagged_logging.rb:68:in `tagged'
    /usr/local/lib/ruby/gems/2.1.0/gems/railties-4.1.4/lib/rails/rack/logger.rb:20:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/actionpack-4.1.4/lib/action_dispatch/middleware/request_id.rb:21:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/methodoverride.rb:21:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/runtime.rb:17:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.1.4/lib/active_support/cache/strategy/local_cache_middleware.rb:26:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/sendfile.rb:112:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/railties-4.1.4/lib/rails/engine.rb:514:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/railties-4.1.4/lib/rails/application.rb:144:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/rack-1.5.2/lib/rack/content_length.rb:14:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/puma-2.8.2/lib/puma/server.rb:490:in `handle_request'
    /usr/local/lib/ruby/gems/2.1.0/gems/puma-2.8.2/lib/puma/server.rb:361:in `process_client'
    /usr/local/lib/ruby/gems/2.1.0/gems/puma-2.8.2/lib/puma/server.rb:254:in `block in run'
    /usr/local/lib/ruby/gems/2.1.0/gems/puma-2.8.2/lib/puma/thread_pool.rb:92:in `call'
    /usr/local/lib/ruby/gems/2.1.0/gems/puma-2.8.2/lib/puma/thread_pool.rb:92:in `block in spawn_thread'

Is there a REST API access ?

I've looked around the documentation but didn't see anywhere if it's possible to control the panamax via some kind of REST or SOAP API ? and if so is there a list of endpoints / commands for it ?

Thanks

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.