kata-containers / ci Goto Github PK

View Code? Open in Web Editor NEW

13.0 13.0 34.0 1.12 MB

Kata Containers CI

Home Page: http://jenkins.katacontainers.io/view/CI%20Status/

License: Apache License 2.0

Shell 100.00%

ci's Introduction

Kata Containers

Welcome to Kata Containers!

This repository is the home of the Kata Containers code for the 2.0 and newer releases.

If you want to learn about Kata Containers, visit the main Kata Containers website.

Introduction

Kata Containers is an open source project and community working to build a standard implementation of lightweight Virtual Machines (VMs) that feel and perform like containers, but provide the workload isolation and security advantages of VMs.

License

The code is licensed under the Apache 2.0 license. See the license file for further details.

Platform support

Kata Containers currently runs on 64-bit systems supporting the following technologies:

Architecture	Virtualization technology
`x86_64`, `amd64`	Intel VT-x, AMD SVM
`aarch64` ("`arm64`")	ARM Hyp
`ppc64le`	IBM Power
`s390x`	IBM Z & LinuxONE SIE

Hardware requirements

The Kata Containers runtime provides a command to determine if your host system is capable of running and creating a Kata Container:

$ kata-runtime check

Notes:

This command runs a number of checks including connecting to the network to determine if a newer release of Kata Containers is available on GitHub. If you do not wish this to check to run, add the --no-network-checks option.

By default, only a brief success / failure message is printed. If more details are needed, the --verbose flag can be used to display the list of all the checks performed.

If the command is run as the root user additional checks are run (including checking if another incompatible hypervisor is running). When running as root, network checks are automatically disabled.

Getting started

See the installation documentation.

Documentation

See the official documentation including:

Configuration

Kata Containers uses a single configuration file which contains a number of sections for various parts of the Kata Containers system including the runtime, the agent and the hypervisor.

Hypervisors

See the hypervisors document and the Hypervisor specific configuration details.

Community

To learn more about the project, its community and governance, see the community repository. This is the first place to go if you wish to contribute to the project.

Getting help

See the community section for ways to contact us.

Raising issues

Please raise an issue in this repository.

Note: If you are reporting a security issue, please follow the vulnerability reporting process

Developers

See the developer guide.

Components

Main components

The table below lists the core parts of the project:

Component	Type	Description
runtime	core	Main component run by a container manager and providing a containerd shimv2 runtime implementation.
runtime-rs	core	The Rust version runtime.
agent	core	Management process running inside the virtual machine / POD that sets up the container environment.
`dragonball`	core	An optional built-in VMM brings out-of-the-box Kata Containers experience with optimizations on container workloads
documentation	documentation	Documentation common to all components (such as design and install documentation).
tests	tests	Excludes unit tests which live with the main code.

Additional components

The table below lists the remaining parts of the project:

Component	Type	Description
packaging	infrastructure	Scripts and metadata for producing packaged binaries (components, hypervisors, kernel and rootfs).
kernel	kernel	Linux kernel used by the hypervisor to boot the guest image. Patches are stored here.
osbuilder	infrastructure	Tool to create "mini O/S" rootfs and initrd images and kernel for the hypervisor.
kata-debug	infrastructure	Utility tool to gather Kata Containers debug information from Kubernetes clusters.
`agent-ctl`	utility	Tool that provides low-level access for testing the agent.
`kata-ctl`	utility	Tool that provides advanced commands and debug facilities.
`trace-forwarder`	utility	Agent tracing helper.
`runk`	utility	Standard OCI container runtime based on the agent.
`ci`	CI	Continuous Integration configuration files and scripts.
`ocp-ci`	CI	Continuous Integration configuration for the OpenShift pipelines.
`katacontainers.io`	Source for the `katacontainers.io` site.
`Webhook`	utility	Example of a simple admission controller webhook to annotate pods with the Kata runtime class

Packaging and releases

Kata Containers is now available natively for most distributions.

General tests

See the tests documentation.

Metrics tests

See the metrics documentation.

Glossary of Terms

See the glossary of terms related to Kata Containers.

ci's People

Contributors

Stargazers

Watchers

ci's Issues

Add issue URL to "bare" TODO/FIXME comments

The following files container TODO/FIXME's that need to reference a github issue:

VMs/metrics/3_complete_baseimage.sh
VMs/metrics/global_env.sh

Add ksm-throttler job

We need to add a Jenkins job for https://github.com/kata-containers/ksm-throttler which currently only has TravisCI configured.

Add CI job to test new released packages.

To ensure the Packages work as expected we should craete a CI job and run it every time a new package version is released.

VMs: occasionally jenkins user is not created, but ubuntu user is?

Sometimes a VM is generated that fails the jenkins user SSH with a credentials failure. Looking at the logs, oddly it seems the jenkins user key has ended up being assigned to the ubuntu user inside the VM:

ci-info: ++++++++++Authorized keys from /home/ubuntu/.ssh/authorized_keys for user ubuntu+++++++++++^M
ci-info: +---------+-------------------------------------------------+---------+-------------------+^M
ci-info: | Keytype |                Fingerprint (md5)                | Options |      Comment      |^M
ci-info: +---------+-------------------------------------------------+---------+-------------------+^M
ci-info: | ssh-rsa | bc:79:8e:c4:95:12:a3:02:de:9e:30:92:85:bd:ff:c2 |    -    | For virsh jenkins |^M
ci-info: +---------+-------------------------------------------------+---------+-------------------+^M

And indeed, doing an ssh as ubuntu@ instead of jenkins@ succeeds!

This feels 'random' if the jenkins user is created or not - and I've not figured out what the trigger is.
I would suspect that some non-YAML character in the SSH key in the user-env file is upsetting cloud-init, but I'd expect that to fail more spectacularly.

Now we know we can ssh in as ubuntu@, at least we can go look at the cloud-init logs in the VM.

[RFC] CI setups for metrics measurements

Hi.
For Clear Containers we have a Jenkins CI system running (inside Intel) that executes a set of metrics to try and detect and report performance regressions. We'd like to discuss options for setting up such a system for Kata.

There are a few requirements and features for such a system - I'll list them out here to help discussion:

The tests really need to run on 'bare metal'. Running them in a virtualised or cloud environment would introduce too much 'noise' into the results, making detection of regressions not viable. (well, for most of the tests - all time and compute based ones)
The current system is comprised of a Jenkins master (which can be a VM/cloud instance), and bare metal Jenkins slave build bots.
The build bots don't have to be particularly powerful machines (at present) - the metrics do not presently particularly stress 'at scale' setups.
Having different build bots running different Linux distributions would be good to get more coverage, but starting with just one is better than nothing
We may wish to discuss isolating the bare metal build machines from any other infrastructure. These machines will automatically download, build and execute any PRs submitted to the projects, with no guarantee what that code will do (yeah, that may sound a touch paranoid, but stuff does happen )
Slightly longer term, we should also look at tying the results into a storage and visualisation system for both historical data reference and trend tracking. My current thoughts here are to store the results into an elasticsearch and use kibana for the visualisation/analysis.

/cc @emonty @gnawux for any input from your side, wrt any setups you presently have or infra available etc.

metrics: Enable density metrics CI on cloud services

Some of our metrics tests (notably the memory footprint tests) should be pretty immune to which machine they run on and noisy neighbor effects - and thus we can in theory run up a metrics CI to track at least those metrics without the need for dedicated bare metal machines.
This will let us test/show the setup running and also let us bring up all the other infrastructure, like the backend historical database and GUI view/analysis front end.
We will need to adapt some of our current metrics run scripts to know when they are on a non-time-guaranteed machine and thus skip running and checking a bunch of metrics.
Given the transient nature of cloud build slaves, we may also have to find a way to adapt to handing an 'expected results' file to checkmetrics.

Add openSuSE / SLES testing

For parity with #3, it would be great if we could also get a few Jenkins nodes running SLES, particularly since SuSE is listed as a Supporting Company at the bottom of https://katacontainers.io/.

/cc @egernst.

ARM: get the ARM CIs up

Provide details and help to get the ARM CI's up for kata.

The basic information for setting up Jenkins should be available in the https://github.com/kata-containers/ci/tree/master/jenkins folder, but there will no doubt be differences (such as the use of Azure vs packet.net vs bare metal servers and agents), and the information is somewhat coded into the Jenkins xml configs, so unless you are a Jenkins expert, that may not be the most helpful ;-)

We may wish/need/want to extract some of the core information from those xml files and write a short .md file describing the basic requirements and setup.

/cc @chavafg for knowledge of existing x86 Azure Jenkins setup
/cc @gnawux @Pennyzct @Weichen81 for input on the ARM side.

Add Ubuntu 17.10 CI jobs

We should test with the latest LTS release (16.04) and the latest release (17.10).

metrics: resource and deploy metrics data store and GUI view

Back in April/May I proposed how we set up a metrics CI for Kata.
I have (finally!) gotten to boot (job here) a slave on our cloud system to start providing footprint/density data, and am about to deploy bare metal over on packet.net for further time/noise sensitive tests.

That allows us to take the next step, using the JSON DB hook to store historical data, allowing further long term analysis and tracking, and attaching a UI.

I've mocked this up locally before, and used elasticsearch for the DB and Kibana for the viewer (here is an example:

)

What I'd like to do now is source and agree some space to run up elastic and kibana so we can start gathering the historical data and bring up the UI.

My initial thought is that we can utilise the same cloud resource we use for the Jenkins master machine. We don't need time critical stuff here, but will need:

24/7 instances (so the results can be stored when the PRs run, and the GUI invoked at any time)
probably not lots of RAM
over time we may need disk space for the DB

@mnaser @cboylan would you be able to resource us for that?
And, if there is anybody out there with elastic or kibana experience, especially around any aspects of security and how we might set up account/login verification (ideally against github like we do with Jenkins), that would be great :-)

VMs: Disable unattended upgrades for Ubuntu VMs

The VMs based on the Ubuntu cloud image appear to have the Ubuntu default 'unattended upgrades' enabled. This races with the CI script use of apt, and occasionally the scripts fail to get the lock, and thus fail the CI run.

We should disable the unattended upgrades, either similarly to how we do for the Azure scripts, or through some apt config, or I believe one can remove some packages to get the same effect (but I cannot find a reference).

(RFC) Can we cache CI artifacts to speed up CI cycle times...

Each of our CI build/runs is pretty much a full isolated from source build, of almost all components. There are a couple of components it pulls as binary from an external source, but half the time the system builds those from source anyway due to version differences.
Given almost all components are identical across all builds, having the CI build everything from source is costing us a lot of CI resource time. We should discuss if there is any feasible way we can set up an 'artifact repository' that can somehow look at the version/commit/SHA of the source and the distro we are building on, and seeing if it can provide a pre-built binary...

There may be complications or wrinkles here, so here are some thought/notes:

Some larger items we currently extract from our OBS binary builds if we can (kernel, qemu etc.),
- but, if our version.yaml file does not match the pre-built binary (which happens often as we upgrade our HEAD, and the binaries are only from the last tagged release), then we build from source.
Ensuring we match pre-built binaries not only to their correct 'versions' (which should be fairly easy using either a version number for some components such as kernel or qemu, or a git SHA for some other components) but - also match the correct distro and its update status, could be tricky.
maybe just having a large ccache accessible by the build slaves would help greatly, and be a 'quick fix' for the qemu and kernel builds (they are in C, I don't think there is a ccache for go? - so it may not help with any k8s build/install for instance.)

For a start, we should probably have a first pass breakdown of where the time goes on a CI build at present. Before optimising, we should probably measure, measure, measure...

/cc @chavafg @jcvenegas @kata-containers/architecture-committee

Ability to run stand-alone scripts periodically

It would be really useful if we could run a bunch of scripts on the CI daily for things like tracking external dependencies:

checking if newer versions of docker/k8s are available.
checking if newer versions of qemu are available.
checking if new distro versions are available or old distro versions are EOL.
checking for new LTS kernel versions.

FTR, we track these sorts of versions in our versions database:

https://github.com/kata-containers/runtime/blob/master/versions.yaml

Ideally, if these scripts found a change, they'd ping a message to Slack/irc and maybe even raise a github issue for tracking purposes.

(RFC) Move CI builds from 'always' to 'on demand'

Currently we build all PRs automatically on submission or change. This has some benefits:

PRs are processed 'as soon as possible' without any human intervention, ideally helping the flow of PRs, as the submitter may see and fix failures before a reviewer even looks at it

but also some downsides:

our CI systems are pretty busy, which means sometimes a PR we'd really like to push through quickly has to wait >2h for the CIs to kick in
theoretically anybody can push a coin miner etc. PR to the repo and use our CI machines to run it etc.

The proposal for discussion is to move our CIs to be more reviewer driven. That is, the CIs will not automatically build a PR upon submission, but will wait for a key phrase or label from a maintainer/reviewer to be set before it will proceed to build. This will:

allow us to prioritise more which PRs get built when
reduce the CI machine usage overall (which should then benefit average CI submit-to-build times overall)
stop anybody PR'ing a bomb into the CI systems.

I believe our choices for CI trigger are using either:

a label
a key phrase in the comments

Opening this issue for heads up and comments. Expect the main discussion to happen in a Kata arch call.

/cc @egernst @annabellebertooch @chavafg @kata-containers/architecture-committee

jenkins: add procenv files to the artifacts that we need to archive.

We need to add:

procenv-root.log.gz
procenv-${USER}.log.gz

IBM Z: Get Jenkins up for IBM Z architecture

Use this ticket to gather info and discuss/diagnose getting Jenkins QA CI set up and running for IBM Z series.
/cc @ydjainopensource , from the IBM side.

From discussion so far, there are two machines available. We cannot allow open SSH into those machines from outside that org, which means we will need to set up a Jenkins Master as well as slaves on those machines.

To be clear then - we cannot allow the main Kata Jenkins master hosted on vexxhost SSH access into those machines - as it (well, the slave node user accounts) require passwordless sudo to run the tests, and thus in theory anybody with access to the master node with admin rights can install a terminal plugin and hop into the slaves as root etc. etc.

That will require us to hook that new master into the kata repos, and generate it a github kata bot token to allow it access the repos unrestricted and post results.

Suggestion is to get this hooked up and working on the proxy repo first.

Recommended pre-reading are these two items:

/cc @chavafg for Jenkins and CI Kata guru-ness

Create jenkins setup for documentation repo

We need Jenkins config for the https://github.com/kata-containers/documentation repo since once that's available, I can add a new doc repo CI test to test the install docs (for which we need VT-x, so Travis is out):

kata-containers/documentation#132

metrics: VMs: update documentation and scripts

There are some things that can be improved on the https://github.com/kata-containers/ci/tree/master/VMs/metrics scripts and documentation:

list the pre-reqs
document some of the user setup requirements
provide the jenkins start/stop script helpers

Zuul CI tracking issue

One requirement we have for some of the repo tests is to provide a terminal to the jobs (shim tests for example). Jenkins does not do this whereas Travis does.

We've got a temporary hack in place to work around the issue:

https://github.com/kata-containers/shim/blob/master/.ci/faketty.sh#L14

Related: kata-containers/proxy#74

/cc @cboylan.

Add "kubernetes-containerd-integration" Jenkins job.

Add a new job that should took an small amount of time.

Start Ubuntu VM
Use as starting point https://github.com/kata-containers/tests/blob/master/.ci/jenkins_job_build.sh
export in the some variables to allow indentify will tests k8s + contaienrd only.
export CI_JOB=CRI_CONTAINERD_K8S

Jenkins scripts should honour versions database for golang version

The config.xml files currently hard-code go1.9.2 (and amd64) in the <initScript/> script sections. But we should be testing with what is defined in the versions database:

https://github.com/kata-containers/runtime/blob/master/versions.yaml#L131..L136

Ideally, the script sections would be a single call to a single script kept in this repository.

That script would clone the tests repo then call https://github.com/kata-containers/tests/blob/master/.ci/install_go.sh (which will read https://github.com/kata-containers/runtime/blob/master/versions.yaml and dtrt).

/cc @jcvenegas.

Show environment before build starts

To help us debug oddities in Jenkins setup / behaviour, it would be really useful if 2 commands could be run before the main build triggers:

kata-collect-data.sh.
https://github.com/jamesodhunt/procenv (it's in the Ubuntu and Fedora repos).

Note: procenv produces a lot of output (all useful! ;) but to keep @grahamwhaley happy, we might want to find a way to redirect that output to a separate output file to avoid spamming the main job console?

Originally raised as: clearcontainers/jenkins#5.

Add RHEL testing

In Clear Containers we ended up needing to obtain docker-EE and RHEL license to facilitate testing on RHEL.

Need to investigate if we can use the developer program: https://developers.redhat.com/products/rhel/download/

Make it easier to debug CI failures

Currently, the CI runs a bunch of tests after enabling full debug.

We need the debug to determine the cause of failures but debugging a CI failure is still hard since:

There is a lot of output.

We attempt to mitigate this by chopping up the log files into chunks. That's helpful in some ways, but actually tends to make it more difficult to identify where the problem occurred using the Jenkins web interface at least (since the expectation is you will download all the compressed file chunks locally, and join them together to recreate the logs).
Some of the tests (rightly) provoke errors on purpose.

The problem here being that the captured log files contain lots of level=error entries. What is not clear to the casual reader is that most/all of those error log entries are expected. But if the CI fails, that makes it difficult to find the correct level=error log entry that indicates the real (unexpected) error.

We need to make honing in on real errors detected by the CI a trivial operation. Ideas:

Use the Jenkins pipeline plugin (#15).
Make the logger in all system components (slightly) smarter

With a small bit of rework, we could make the logger for agent/shim/proxy/runtime/throttler check for a particular environment variable. Iff that is set, we could add an extra log field. Then, callers (that is to say particular tests) could set this variable resulting in log entries such as:
```
level=error error="..." kata-tests=expected-error ...
```
It would then be easy to filter out such expected errors leaving a (hopefully) much smaller set of remaining errors that would indicate the reall (unexpected) failure.

Notes:
- Since the components would only be checking for a particular environment variable, there is no DoS issue (they won't log the value).
Issues:
- Although the envvar approach will work fine for unit tests, it will not be reliable when testing under docker atleast since docker doesn't not pass environment variables to the runtime. An alternative approach might be to use a magic runtime config file option and CLI options to pass to the shim/proxy to get the expected behaviour.
- If the variable is set and multiple errors occur, the extra field might hide a real issue (manual developer log inspection required).
- The envvar approach might involve breaking some of the existing tests into smaller units to ensure that only a single scenario runs with the "expected error" envvar set.

Add job for kubernetes-incubator/cri-o repo

metrics VM generation scripts are not multi-architecture aware

The VM generation scripts (in VMs/metrics - from #31) are not multi-architecture aware, and are hard wired (in the user-env files) to use the x86 apt repos for instance.
I've added a check so they now report and quit for any non x86_64 invocations, but if/when we get to use these on a non-x86 arch, we should fix the scripts (most likely by making arch specific user-env files if we need to modify the package or commands, or by doing a sed substitute on the arch, like we do for the USER presently).

Add initrd testing to the CI

Verify at least these setups:

- AGENT_INIT=yes TEST_INITRD=yes OSBUILDER_DISTRO=alpine
- AGENT_INIT=no TEST_INITRD=no OSBUILDER_DISTRO=clearlinux

Related: kata-containers/tests#173

Add initrd jobs

We should create Jenkins nodes running initrd in ubuntu 16.04 in all our different repositories.

metrics: Document the metrics CI system

We have a framework and some tooling in place that enables a metrics CI system to be built. Document those parts, and how they are intended to be configured and used.

For reference, parts so far include:

Add Jenkins config to repo

Add the Jenkins config files to this repo as we did for https://github.com/clearcontainers/jenkins.

Update config for kernel and throttler artifacts

We need to update the Jenkins config for new artifacts:

kata-containers/tests#748 adds kata-ksm_throttler* and kata-vc_throttler* artifacts.
kata-containers/tests#753 will add a kernel* artifact.

I wonder if there is a way to avoid having to hard-code the artifacts in the Jenkins XML as well as the teardown script? Could we create all the artifacts in a $PWD/artifacts/ directory and just configure Jenkins to add all files in that directory as artifacts?

Add Fedora 28 to CI

Related: #21.

Metrics: VMs: now require perf to be installed

With the addition of the network cpu test here, we now need to add perf into the metrics VMs.

Add Ubuntu 18.04 LTS to CI

Hot on the heels of #11, we need to add Ubuntu Bionic to the CI once it's released on 26 April.

Since 18.04 will then be both the newest Ubuntu release and the newest LTS Ubuntu release, I think we should consider retaining 17.10 and 16.04 (since it's likely most 16.04 users will wait until atleast the first point release of 18.04 before upgrading).

@chavafg, @grahamwhaley, @sboeuf - wdyt?

Zuul Job failures

Recently, the zuul jobs have been failing. I see different failures:

Job Time out.
For some reason, some of the jobs have a timeout and unfortunately, it seems that there is no way to check why (or in which step) the job has timed out.
For example: http://logs.openstack.org/06/706/225e10cfc4bb99722b6f5734a1e840138bcea8a0/third-party-check/kata-runsh/e318be6/ara-report/
shows that the run.yaml could not be executed and no logs of that task are available.
On the other hand I see that the post.yaml was executed, which collect the kata logs, meaning that the machine didn't hang, so I was wondering if there could be a way to know the reason of this timeout.
Unable to apply a git patch
Seems that sometimes, we are unable to apply a patch from git:

Applying patch: /home/zuul/src/github.com/kata-containers/packaging/obs-packaging/qemu-lite/patches/0001-memfd-fix-configure-test.patch

*** Please tell me who you are.

Run

  git config --global user.email "[email protected]"
  git config --global user.name "Your Name"

to set your account's default identity.
fatal: empty ident name (for <zuul@ubuntu-xenial-vexxhost-vexxhost-sjc1-0001993195.(none)>) not allowed

For this one, I think we have 2 options: 1. add a git config to the zuul jobs before running the setup or 2. change git am for patch

For some reason, sometimes the vexxhost machine where we run the CI do not have nested virtualization enabled:

time="2018-09-13T17:01:05Z" level=info msg="CPU property found" arch=amd64 description="Intel Architecture CPU" name=GenuineIntel pid=29482 source=runtime type=attribute
time="2018-09-13T17:01:05Z" level=error msg="CPU property not found" arch=amd64 description="Virtualization support" name=vmx pid=29482 source=runtime type=flag
time="2018-09-13T17:01:05Z" level=info msg="CPU property found" arch=amd64 description="64Bit CPU" name=lm pid=29482 source=runtime type=flag
time="2018-09-13T17:01:05Z" level=info msg="CPU property found" arch=amd64 description=SSE4.1 name=sse4_1 pid=29482 source=runtime type=flag
time="2018-09-13T17:01:05Z" level=info msg="kernel property found" arch=amd64 description="Kernel-based Virtual Machine" name=kvm pid=29482 source=runtime type=module
time="2018-09-13T17:01:05Z" level=info msg="kernel property found" arch=amd64 description="Host kernel accelerator for virtio" name=vhost pid=29482 source=runtime type=module
time="2018-09-13T17:01:05Z" level=info msg="kernel property found" arch=amd64 description="Host kernel accelerator for virtio network" name=vhost_net pid=29482 source=runtime type=module
time="2018-09-13T17:01:05Z" level=info msg="kernel property found" arch=amd64 description="Intel KVM" name=kvm_intel pid=29482 source=runtime type=module
time="2018-09-13T17:01:05Z" level=error msg="open /sys/module/kvm_intel/parameters/nested: no such file or directory" arch=amd64 name=kata-runtime pid=29482 source=runtime
open /sys/module/kvm_intel/parameters/nested: no such file or directory

CI for package testing

We need to create a basic CI test to run for packaged components.

Coverage

We need to run the CI for:

All distro versions.
All architectures.

Basic tests

Packages

All files are owned by root:root.
All file perms as as strict as possible:
- No files are group- or world-writable.
- No suid binaries, etc.
No files are empty.
No directories are empty.

Operational

Some basic tests to run:

Package upgrade.
Package install (delete all packages after upgrade and install "afresh").

Ensure a basic container can be created:

$ docker run --runtime kata-runtime -ti busybox true`

Ensure the ksm-throttler service can be started/stopped.
Ensure runtime shows two different config paths and that one is below "/etc/" (see clearcontainers/packaging#291):
```
$ kata-runtime --kata-show-default-config-paths
```
Ensure a config file in /etc/ takes priority over the default in /usr/share/default/kata-containers/.
Enable full debug, create a container and ensure all logs are parseable with the log-parser tool.
Run the data collection script and ensure it runs without error and produces output.

Ensure that the following all result in a full stack trace to the system journal:

for component in kata-runtime kata-shim kata-proxy
do
    for signal in SIGABRT SIGBUS SIGILL SIGQUIT SIGSEGV SIGSTKFLT SIGSYS SIGTRAP
    do
        sudo kill -${signal} ${component}
    done
done

It'll be tricky to do this for the runtime as it has a transitory lifetime so we'll have to get creative ;)

Ensure that if any system components are killed with SIGTERM (including qemu) that a message is logged to the system journal with level=error with details of what happened.
Verify debug console is available.

This assumes kata-containers/agent#169 becomes a reality.
Ensure vsock works (see kata-containers/runtime#1203).

Add CentOS 7 CI jobs

We should create Jenkins nodes running CentOS as it's a very popular distro choice.

Jenkins: some logs are getting large

I noticed today that some logs on failing builds were pretty big. I don't normally notice, and I see others are not anywhere near as big, so this may be an anomaly or mis-configure we need to work out.
In this instance, over at: http://jenkins.katacontainers.io/job/kata-containers-runtime-fedora-27-PR/281/console
the full log is 16030 KB. I'll try to extract a small snippet of something that looks like it is consuming space, as the link/log might eventually get purged?

Here is a big chunk of the tail for instance:

time="2018-07-26T14:02:09.456348948Z" level=info msg="time=\"2018-07-26T14:02:09.414953429Z\" level=info msg=\"agent grpc server starts\" name=kata-agent pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:09.470320482Z" level=info msg="time=\"2018-07-26T14:02:09.434437385Z\" level=debug msg=\"New serial channel event\" events=4 name=kata-agent pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:09.527165102Z" level=info msg="time=\"2018-07-26T14:02:09.491296877Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req= request=/grpc.Health/Check sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:09.530736182Z" level=info msg="time=\"2018-07-26T14:02:09.49453036Z\" level=debug msg=\"request end\" duration=\"106.991µs\" name=kata-agent pid=97 request=/grpc.Health/Check resp=\"status:SERVING \" sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:09.569764062Z" level=info msg="time=\"2018-07-26T14:02:09.532839091Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"interface:<device:\\"eth0\\" name:\\"eth0\\" IPAddresses:<address:\\"172.17.0.2\\" mask:\\"16\\" > mtu:1500 hwAddr:\\"02:42:ac:11:00:02\\" > \" request=/grpc.AgentService/UpdateInterface sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:09.57168743Z" level=info msg="time=\"2018-07-26T14:02:09.535800217Z\" level=info msg=\"Getting interface from MAC address\" interface-name=eth0 mac-address=\"02:42:ac:11:00:02\" name=kata-agent pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:09.57476444Z" level=info msg="time=\"2018-07-26T14:02:09.538770375Z\" level=info msg=\"Link found\" interface-name=eth0 link=\"&{LinkAttrs:{Index:2 MTU:1500 TxQLen:1000 Name:eth0 HardwareAddr:02:42:ac:11:00:02 Flags:broadcast|multicast RawFlags:4098 ParentIndex:0 MasterIndex:0 Namespace:<nil> Alias: Statistics:0xc4201a35d4 Promisc:0 Xdp:0xc420017460 EncapType:ether Protinfo:<nil> OperState:down NetNsID:0 NumTxQueues:0 NumRxQueues:0}}\" mac-address=\"02:42:ac:11:00:02\" name=kata-agent pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:09.577313231Z" level=info msg="time=\"2018-07-26T14:02:09.54153049Z\" level=debug msg=\"request end\" duration=7.057466ms name=kata-agent pid=97 request=/grpc.AgentService/UpdateInterface resp=\"device:\\"eth0\\" name:\\"eth0\\" IPAddresses:<address:\\"172.17.0.2\\" mask:\\"16\\" > mtu:1500 hwAddr:\\"02:42:ac:11:00:02\\" \" sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:09.594419937Z" level=info msg="time=\"2018-07-26T14:02:09.558589712Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"routes:<Routes:<gateway:\\"172.17.0.1\\" device:\\"eth0\\" > Routes:<dest:\\"172.17.0.0/16\\" device:\\"eth0\\" source:\\"172.17.0.2\\" scope:253 > > \" request=/grpc.AgentService/UpdateRoutes sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:09.600268682Z" level=info msg="time=\"2018-07-26T14:02:09.56422014Z\" level=debug msg=\"request end\" duration=4.93636ms name=kata-agent pid=97 request=/grpc.AgentService/UpdateRoutes resp=\"Routes:<gateway:\\"172.17.0.1\\" device:\\"eth0\\" > Routes:<dest:\\"172.17.0.0/16\\" device:\\"eth0\\" source:\\"172.17.0.2\\" scope:253 > \" sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:09.606287799Z" level=info msg="time=\"2018-07-26T14:02:09.570466531Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"hostname:\\"e7701863a690\\" storages:<driver:\\"9p\\" source:\\"kataShared\\" fstype:\\"9p\\" options:\\"trans=virtio,version=9p2000.L\\" options:\\"nodev\\" options:\\"msize=8192\\" mount_point:\\"/run/kata-containers/shared/containers/\\" > storages:<driver:\\"ephemeral\\" source:\\"shm\\" fstype:\\"tmpfs\\" options:\\"noexec\\" options:\\"nosuid\\" options:\\"nodev\\" options:\\"mode=1777\\" options:\\"size=67108864\\" mount_point:\\"/run/kata-containers/sandbox/shm\\" > \" request=/grpc.AgentService/CreateSandbox sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:09.612983269Z" level=info msg="time=\"2018-07-26T14:02:09.577035034Z\" level=debug mount-destination=/run/kata-containers/shared/containers/ mount-flags=4 mount-fstype=9p mount-options=\"trans=virtio,version=9p2000.L,msize=8192\" mount-source=kataShared name=kata-agent pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:09.619195459Z" level=info msg="time=\"2018-07-26T14:02:09.583210961Z\" level=debug mount-destination=/run/kata-containers/sandbox/shm mount-flags=14 mount-fstype=tmpfs mount-options=\"mode=1777,size=67108864\" mount-source=shm name=kata-agent pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:09.621768464Z" level=info msg="time=\"2018-07-26T14:02:09.585351482Z\" level=debug msg=\"request end\" duration=14.663494ms name=kata-agent pid=97 request=/grpc.AgentService/CreateSandbox resp=\"&Empty{}\" sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:09.648953891Z" level=info msg="time=\"2018-07-26T14:02:09.612522124Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" exec_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" OCI:<Version:\\"1.0.1\\" Process:<Terminal:true User:<AdditionalGids:10 > Args:\\"sh\\" Env:\\"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\\" Env:\\"HOSTNAME=e7701863a690\\" Env:\\"TERM=xterm\\" Cwd:\\"/\\" Capabilities:<Bounding:\\"CAP_CHOWN\\" Bounding:\\"CAP_DAC_OVERRIDE\\" Bounding:\\"CAP_FSETID\\" Bounding:\\"CAP_FOWNER\\" Bounding:\\"CAP_MKNOD\\" Bounding:\\"CAP_NET_RAW\\" Bounding:\\"CAP_SETGID\\" Bounding:\\"CAP_SETUID\\" Bounding:\\"CAP_SETFCAP\\" Bounding:\\"CAP_SETPCAP\\" Bounding:\\"CAP_NET_BIND_SERVICE\\" Bounding:\\"CAP_SYS_CHROOT\\" Bounding:\\"CAP_KILL\\" Bounding:\\"CAP_AUDIT_WRITE\\" Effective:\\"CAP_CHOWN\\" Effective:\\"CAP_DAC_OVERRIDE\\" Effective:\\"CAP_FSETID\\" Effective:\\"CAP_FOWNER\\" Effective:\\"CAP_MKNOD\\" Effective:\\"CAP_NET_RAW\\" Effective:\\"CAP_SETGID\\" Effective:\\"CAP_SETUID\\" Effective:\\"CAP_SETFCAP\\" Effective:\\"CAP_SETPCAP\\" Effective:\\"CAP_NET_BIND_SERVICE\\" Effective:\\"CAP_SYS_CHROOT\\" Effective:\\"CAP_KILL\\" Effective:\\"CAP_AUDIT_WRITE\\" Inheritable:\\"CAP_CHOWN\\" Inheritable:\\"CAP_DAC_OVERRIDE\\" Inheritable:\\"CAP_FSETID\\" Inheritable:\\"CAP_FOWNER\\" Inheritable:\\"CAP_MKNOD\\" Inheritable:\\"CAP_NET_RAW\\" Inheritable:\\"CAP_SETGID\\" Inheritable:\\"CAP_SETUID\\" Inheritable:\\"CAP_SETFCAP\\" Inheritable:\\"CAP_SETPCAP\\" Inheritable:\\"CAP_NET_BIND_SERVICE\\" Inheritable:\\"CAP_SYS_CHROOT\\" Inheritable:\\"CAP_KILL\\" Inheritable:\\"CAP_AUDIT_WRITE\\" Permitted:\\"CAP_CHOWN\\" Permitted:\\"CAP_DAC_OVERRIDE\\" Permitted:\\"CAP_FSETID\\" Permitted:\\"CAP_FOWNER\\" Permitted:\\"CAP_MKNOD\\" Permitted:\\"CAP_NET_RAW\\" Permitted:\\"CAP_SETGID\\" Permitted:\\"CAP_SETUID\\" Permitted:\\"CAP_SETFCAP\\" Permitted:\\"CAP_SETPCAP\\" Permitted:\\"CAP_NET_BIND_SERVICE\\" Permitted:\\"CAP_SYS_CHROOT\\" Permitted:\\"CAP_KILL\\" Permitted:\\"CAP_AUDIT_WRITE\\" > > Root:<Path:\\"/run/kata-containers/shared/containers/e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9/rootfs\\" > Hostname:\\"e7701863a690\\" Mounts:<destination:\\"/proc\\" source:\\"proc\\" type:\\"proc\\" options:\\"nosuid\\" options:\\"noexec\\" options:\\"nodev\\" > Mounts:<destination:\\"/dev\\" source:\\"tmpfs\\" type:\\"tmpfs\\" options:\\"nosuid\\" options:\\"strictatime\\" options:\\"mode=755\\" options:\\"size=65536k\\" > Mounts:<destination:\\"/dev/pts\\" source:\\"devpts\\" type:\\"devpts\\" options:\\"nosuid\\" options:\\"noexec\\" options:\\"newinstance\\" options:\\"ptmxmode=0666\\" options:\\"mode=0620\\" options:\\"gid=5\\" > Mounts:<destination:\\"/sys\\" source:\\"sysfs\\" type:\\"sysfs\\" options:\\"nosuid\\" options:\\"noexec\\" options:\\"nodev\\" options:\\"ro\\" > Mounts:<destination:\\"/sys/fs/cgroup\\" source:\\"cgroup\\" type:\\"cgroup\\" options:\\"ro\\" options:\\"nosuid\\" options:\\"noexec\\" options:\\"nodev\\" > Mounts:<destination:\\"/dev/mqueue\\" source:\\"mqueue\\" type:\\"mqueue\\" options:\\"nosuid\\" options:\\"noexec\\" options:\\"nodev\\" > Mounts:<destination:\\"/etc/resolv.conf\\" source:\\"/run/kata-containers/shared/containers/e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9-299a1ab3e44baf20-resolv.conf\\" type:\\"bind\\" options:\\"rbind\\" options:\\"rprivate\\" > Mounts:<destination:\\"/etc/hostname\\" source:\\"/run/kata-containers/shared/containers/e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9-c1e28925ed070e3e-hostname\\" type:\\"bind\\" options:\\"rbind\\" options:\\"rprivate\\" > Mounts:<destination:\\"/etc/hosts\\" source:\\"/run/kata-containers/shared/containers/e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9-f42f7de8b01a43cc-hosts\\" type:\\"bind\\" options:\\"rbind\\" options:\\"rprivate\\" > Mounts:<destination:\\"/dev/shm\\" source:\\"/run/kata-containers/sandbox/shm\\" type:\\"bind\\" options:\\"rbind\\" > Linux:<Resources:<CPU:<> > CgroupsPath:\\"/docker/e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" Namespaces:<Type:\\"mount\\" > Namespaces:<Type:\\"uts\\" > Namespaces:<Type:\\"ipc\\" > MaskedPaths:\\"/proc/kcore\\" MaskedPaths:\\"/proc/keys\\" MaskedPaths:\\"/proc/latency_stats\\" MaskedPaths:\\"/proc/timer_list\\" MaskedPaths:\\"/proc/timer_stats\\" MaskedPaths:\\"/proc/sched_debug\\" MaskedPaths:\\"/proc/scsi\\" MaskedPaths:\\"/sys/firmware\\" ReadonlyPaths:\\"/proc/asound\\" ReadonlyPaths:\\"/proc/bus\\" ReadonlyPaths:\\"/proc/fs\\" ReadonlyPaths:\\"/proc/irq\\" ReadonlyPaths:\\"/proc/sys\\" ReadonlyPaths:\\"/proc/sysrq-trigger\\" > > \" request=/grpc.AgentService/CreateContainer sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:09.661809943Z" level=info msg="[    1.551878] pci 0000:00:02.0: PCI bridge to [bus 01]
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:09.662895843Z" level=info msg="[    1.552887] pci 0000:00:02.0:   bridge window [io  0xc000-0xcfff]
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:09.665747602Z" level=info msg="[    1.555927] pci 0000:00:02.0:   bridge window [mem 0xfe600000-0xfe7fffff]
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:09.667677452Z" level=info msg="[    1.557678] pci 0000:00:02.0:   bridge window [mem 0xfea00000-0xfebfffff 64bit pref]
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:09.780969285Z" level=info msg="time=\"2018-07-26T14:02:09.744567943Z\" level=debug msg=\"request end\" duration=131.093379ms name=kata-agent pid=97 request=/grpc.AgentService/CreateContainer resp=\"&Empty{}\" sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:09.783245235Z" level=info msg="time=\"2018-07-26T14:02:09.74652703Z\" level=info msg=\"ignoring unexpected signal\" name=kata-agent pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 signal=\"child exited\" source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:09.795022441Z" level=info msg="time=\"2018-07-26T14:02:09.759186316Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" exec_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" \" request=/grpc.AgentService/TtyWinResize sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:09.795601458Z" level=info msg="time=\"2018-07-26T14:02:09.759515514Z\" level=debug msg=\"request end\" duration=\"126.62µs\" name=kata-agent pid=97 request=/grpc.AgentService/TtyWinResize resp=\"&Empty{}\" sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:09.801902741Z" level=info msg="time=\"2018-07-26T14:02:09.76555796Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" exec_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" len:32768 \" request=/grpc.AgentService/ReadStdout sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:09.839380508Z" level=info msg="time=\"2018-07-26T14:02:09.803517336Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" \" request=/grpc.AgentService/StartContainer sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:09.848128269Z" level=info msg="time=\"2018-07-26T14:02:09.812322196Z\" level=debug msg=\"request end\" duration=8.572254ms name=kata-agent pid=97 request=/grpc.AgentService/StartContainer resp=\"&Empty{}\" sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:09.869961939Z" level=info msg="time=\"2018-07-26T14:02:09.833689655Z\" level=debug msg=\"request end\" duration=66.923233ms name=kata-agent pid=97 request=/grpc.AgentService/ReadStdout resp=\"data:\\"/ # \\" \" sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:09.881630612Z" level=info msg="time=\"2018-07-26T14:02:09.845127619Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" exec_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" len:32768 \" request=/grpc.AgentService/ReadStdout sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.601184113Z" level=info msg="[    3.491501] CPU1 has been hot-added
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.605055779Z" level=info msg="time=\"2018-07-26T14:02:11.569260965Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"nb_cpus:1 \" request=/grpc.AgentService/OnlineCPUMem sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.605434549Z" level=info msg="time=\"2018-07-26T14:02:11.569469373Z\" level=debug msg=\"request end\" duration=\"10.288µs\" name=kata-agent pid=97 request=/grpc.AgentService/OnlineCPUMem resp=\"&Empty{}\" sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.607457173Z" level=info msg="time=\"2018-07-26T14:02:11.571554182Z\" level=debug msg=\"connecting vCPUs\" name=kata-agent pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent vcpus-to-connect=1
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.616162064Z" level=info msg="[    3.506314] x86: Booting SMP configuration:
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.61696027Z" level=info msg="[    3.506834] smpboot: Booting Node 0 Processor 1 APIC 0x3
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.626523627Z" level=info msg="time=\"2018-07-26T14:02:11.589105579Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" resources:<Memory:<> CPU:<Quota:100000 Period:100000 Cpus:\\"0-1\\" > Pids:<> BlockIO:<> > \" request=/grpc.AgentService/UpdateContainer sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.630064509Z" level=info msg="[    1.582289] kvm-clock: cpu 1, msr 0:7ffdb041, secondary cpu clock
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.630999785Z" level=info msg="[    3.517722] smpboot: CPU 1 Converting physical 3 to logical package 1
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.631633989Z" level=info msg="[    3.519910] KVM setup async PF for cpu 1
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.632548996Z" level=info msg="[    3.520749] kvm-stealtime: cpu 1, msr 7fc94e80
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.636003125Z" level=info msg="[    3.525993] Will online and init hotplugged CPU: 1
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.662003492Z" level=info msg="time=\"2018-07-26T14:02:11.62584051Z\" level=debug msg=\"connecting vCPUs\" name=kata-agent pid=97 range-of-vcpus=0-1 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.662847262Z" level=info msg="time=\"2018-07-26T14:02:11.626426619Z\" level=debug msg=\"updating cpuset cgroup\" container=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 name=kata-agent pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.663276506Z" level=info msg="time=\"2018-07-26T14:02:11.62748964Z\" level=debug msg=\"updating cpuset cgroup\" name=kata-agent path=/sys/fs/cgroup/cpuset/docker pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.66381497Z" level=info msg="time=\"2018-07-26T14:02:11.627772475Z\" level=debug msg=\"updating cpuset cgroup\" name=kata-agent path=/sys/fs/cgroup/cpuset/docker/e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.664138331Z" level=info msg="time=\"2018-07-26T14:02:11.628309853Z\" level=debug msg=\"updating cpuset cgroup\" name=kata-agent path=/sys/fs/cgroup/cpuset/docker pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.664451143Z" level=info msg="time=\"2018-07-26T14:02:11.628618438Z\" level=debug msg=\"updating cpuset cgroup\" name=kata-agent path=/sys/fs/cgroup/cpuset/docker/e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.667772023Z" level=info msg="time=\"2018-07-26T14:02:11.631800302Z\" level=debug msg=\"request end\" duration=40.658773ms name=kata-agent pid=97 request=/grpc.AgentService/UpdateContainer resp=\"&Empty{}\" sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.760448601Z" level=info msg="time=\"2018-07-26T14:02:11.724241857Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" exec_id:\\"291e50fb-f94b-4b26-a2a0-285d26c3b7b3\\" process:<User:<> Args:\\"nproc\\" Env:\\"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\\" Env:\\"HOSTNAME=e7701863a690\\" Cwd:\\"/\\" > \" request=/grpc.AgentService/ExecProcess sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.797429064Z" level=info msg="time=\"2018-07-26T14:02:11.761627327Z\" level=debug msg=\"request end\" duration=36.315427ms name=kata-agent pid=97 request=/grpc.AgentService/ExecProcess resp=\"&Empty{}\" sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.798077284Z" level=info msg="time=\"2018-07-26T14:02:11.762222078Z\" level=info msg=\"ignoring unexpected signal\" name=kata-agent pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 signal=\"child exited\" source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.798606531Z" level=info msg="time=\"2018-07-26T14:02:11.762668089Z\" level=info msg=\"ignoring unexpected signal\" name=kata-agent pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 signal=\"child exited\" source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.806183951Z" level=info msg="time=\"2018-07-26T14:02:11.770137281Z\" level=debug msg=\"process exited\" name=kata-agent pid=141 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent status=0
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.806267504Z" level=info msg="time=\"2018-07-26T14:02:11.770334717Z\" level=info msg=\"ignoring unexpected signal\" name=kata-agent pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 signal=\"child exited\" source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.813383093Z" level=info msg="time=\"2018-07-26T14:02:11.776088919Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" exec_id:\\"291e50fb-f94b-4b26-a2a0-285d26c3b7b3\\" len:32768 \" request=/grpc.AgentService/ReadStdout sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.813489237Z" level=info msg="time=\"2018-07-26T14:02:11.776300416Z\" level=debug msg=\"request end\" duration=\"66.696µs\" name=kata-agent pid=97 request=/grpc.AgentService/ReadStdout resp=\"data:\\"2\\n\\" \" sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.813550627Z" level=info msg="time=\"2018-07-26T14:02:11.776476027Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" exec_id:\\"291e50fb-f94b-4b26-a2a0-285d26c3b7b3\\" len:32768 \" request=/grpc.AgentService/ReadStderr sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.813603099Z" level=info msg="time=\"2018-07-26T14:02:11.776635485Z\" level=debug msg=\"request end\" duration=\"39.818µs\" error=EOF name=kata-agent pid=97 request=/grpc.AgentService/ReadStderr resp= sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.813649141Z" level=info msg="time=\"2018-07-26T14:02:11.777168967Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" exec_id:\\"291e50fb-f94b-4b26-a2a0-285d26c3b7b3\\" \" request=/grpc.AgentService/CloseStdin sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.813702185Z" level=info msg="time=\"2018-07-26T14:02:11.777329533Z\" level=debug msg=\"request end\" duration=\"24.352µs\" name=kata-agent pid=97 request=/grpc.AgentService/CloseStdin resp=\"&Empty{}\" sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.819551636Z" level=info msg="time=\"2018-07-26T14:02:11.77945017Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" exec_id:\\"291e50fb-f94b-4b26-a2a0-285d26c3b7b3\\" len:32768 \" request=/grpc.AgentService/ReadStdout sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.819623656Z" level=info msg="time=\"2018-07-26T14:02:11.780159981Z\" level=debug msg=\"request end\" duration=\"26.098µs\" error=EOF name=kata-agent pid=97 request=/grpc.AgentService/ReadStdout resp= sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.822856618Z" level=info msg="time=\"2018-07-26T14:02:11.785653838Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" exec_id:\\"291e50fb-f94b-4b26-a2a0-285d26c3b7b3\\" \" request=/grpc.AgentService/WaitProcess sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.822953972Z" level=info msg="time=\"2018-07-26T14:02:11.786356714Z\" level=debug msg=\"request end\" duration=\"442.837µs\" name=kata-agent pid=97 request=/grpc.AgentService/WaitProcess resp= sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.826075483Z" level=debug msg="Copy stream error: %vread unix /run/vc/sbs/e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9/proxy.sock->@: read: connection reset by peer" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=proxy
time="2018-07-26T14:02:11.951971647Z" level=info msg="[    3.840478] CPU2 has been hot-added
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.952265196Z" level=info msg="[    3.842425] CPU3 has been hot-added
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.957009698Z" level=info msg="time=\"2018-07-26T14:02:11.92118698Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"nb_cpus:2 \" request=/grpc.AgentService/OnlineCPUMem sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.957354429Z" level=info msg="time=\"2018-07-26T14:02:11.921593201Z\" level=debug msg=\"request end\" duration=\"12.525µs\" name=kata-agent pid=97 request=/grpc.AgentService/OnlineCPUMem resp=\"&Empty{}\" sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.95769044Z" level=info msg="time=\"2018-07-26T14:02:11.92177889Z\" level=debug msg=\"connecting vCPUs\" name=kata-agent pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent vcpus-to-connect=2
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.959811778Z" level=info msg="[    3.849669] smpboot: Booting Node 0 Processor 2 APIC 0x2
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.968148681Z" level=info msg="[    3.658846] kvm-clock: cpu 2, msr 0:7ffdb081, secondary cpu clock
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.968866358Z" level=info msg="[    3.858043] KVM setup async PF for cpu 2
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.969885782Z" level=info msg="[    3.858844] kvm-stealtime: cpu 2, msr 7fd14e8time=\"2018-07-26T14:02:11.933783314Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" resources:<Memory:<> CPU:<Quota:300000 Period:100000 Cpus:\\"1,2\\" > Pids:<> BlockIO:<> > \" request=/grpc.AgentService/UpdateContainer sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent0
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.970284126Z" level=info msg="
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.971839986Z" level=info msg="[    3.861803] Will online and init hotplugged CPU: 2
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.973616726Z" level=info msg="[    3.863796] smpboot: Booting Node 0 Processor 3 APIC 0x1
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.98269272Z" level=info msg="[    3.658847] kvm-clock: cpu 3, msr 0:7ffdb0c1, secondary cpu clock
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.983241732Z" level=info msg="[    3.869910] smpboot: CPU 3 Converting physical 1 to logical package 3
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.983585924Z" level=info msg="[    3.872441] KVM setup async PF for cpu 3
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.984202528Z" level=info msg="[    3.873300] kvm-stealtime: cpu 3, msr 7fd94e80
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.986018205Z" level=info msg="[    3.875889] Will online and init hotplugged CPU: 3
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.988873732Z" level=info msg="time=\"2018-07-26T14:02:11.953075053Z\" level=debug msg=\"connecting vCPUs\" name=kata-agent pid=97 range-of-vcpus=0-3 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.989126894Z" level=info msg="time=\"2018-07-26T14:02:11.953275834Z\" level=debug msg=\"updating cpuset cgroup\" container=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 name=kata-agent pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.989362662Z" level=info msg="time=\"2018-07-26T14:02:11.953446237Z\" level=debug msg=\"updating container cpuset cgroup parents\" cpuset=0-1 name=kata-agent pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.989623374Z" level=info msg="time=\"2018-07-26T14:02:11.953610348Z\" level=debug msg=\"updating cpuset cgroup\" name=kata-agent path=/sys/fs/cgroup/cpuset/docker pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.98998523Z" level=info msg="time=\"2018-07-26T14:02:11.954209668Z\" level=debug msg=\"updating cpuset cgroup\" name=kata-agent path=/sys/fs/cgroup/cpuset/docker pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.990209492Z" level=info msg="time=\"2018-07-26T14:02:11.954404518Z\" level=warning msg=\"Could not update container cpuset cgroup\" error=\"Could not update cpuset cgroup '1,2': write /sys/fs/cgroup/cpuset/docker/cpuset.cpus: device or resource busy\" name=kata-agent pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:11.992190686Z" level=info msg="time=\"2018-07-26T14:02:11.956391346Z\" level=debug msg=\"request end\" duration=21.738246ms name=kata-agent pid=97 request=/grpc.AgentService/UpdateContainer resp=\"&Empty{}\" sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.079324163Z" level=info msg="time=\"2018-07-26T14:02:12.043446099Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" exec_id:\\"fbc97a29-4e82-419a-8b39-25e95bc8be83\\" process:<User:<> Args:\\"nproc\\" Env:\\"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\\" Env:\\"HOSTNAME=e7701863a690\\" Cwd:\\"/\\" > \" request=/grpc.AgentService/ExecProcess sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.12640183Z" level=info msg="time=\"2018-07-26T14:02:12.090375936Z\" level=debug msg=\"request end\" duration=46.572647ms name=kata-agent pid=97 request=/grpc.AgentService/ExecProcess resp=\"&Empty{}\" sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.128112512Z" level=info msg="time=\"2018-07-26T14:02:12.09218875Z\" level=info msg=\"ignoring unexpected signal\" name=kata-agent pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 signal=\"child exited\" source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.131210885Z" level=info msg="time=\"2018-07-26T14:02:12.093106849Z\" level=info msg=\"ignoring unexpected signal\" name=kata-agent pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 signal=\"child exited\" source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.143568941Z" level=info msg="time=\"2018-07-26T14:02:12.107502952Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" exec_id:\\"fbc97a29-4e82-419a-8b39-25e95bc8be83\\" len:32768 \" request=/grpc.AgentService/ReadStdout sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.145851511Z" level=info msg="time=\"2018-07-26T14:02:12.110087696Z\" level=debug msg=\"request end\" duration=\"38.048µs\" name=kata-agent pid=97 request=/grpc.AgentService/ReadStdout resp=\"data:\\"2\\n\\" \" sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.146493549Z" level=info msg="time=\"2018-07-26T14:02:12.109241685Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" exec_id:\\"fbc97a29-4e82-419a-8b39-25e95bc8be83\\" len:32768 \" request=/grpc.AgentService/ReadStderr sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.147135136Z" level=info msg="time=\"2018-07-26T14:02:12.111306383Z\" level=debug msg=\"request end\" duration=\"395.287µs\" error=EOF name=kata-agent pid=97 request=/grpc.AgentService/ReadStderr resp= sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.147889159Z" level=info msg="time=\"2018-07-26T14:02:12.10932008Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" exec_id:\\"fbc97a29-4e82-419a-8b39-25e95bc8be83\\" \" request=/grpc.AgentService/CloseStdin sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.14839857Z" level=info msg="time=\"2018-07-26T14:02:12.112288765Z\" level=debug msg=\"request end\" duration=\"16.904µs\" name=kata-agent pid=97 request=/grpc.AgentService/CloseStdin resp=\"&Empty{}\" sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.149670655Z" level=info msg="time=\"2018-07-26T14:02:12.110012903Z\" level=debug msg=\"process exited\" name=kata-agent pid=166 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent status=0
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.150820103Z" level=info msg="time=\"2018-07-26T14:02:12.113088935Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" exec_id:\\"fbc97a29-4e82-419a-8b39-25e95bc8be83\\" len:32768 \" request=/grpc.AgentService/ReadStdout sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.151441268Z" level=info msg="time=\"2018-07-26T14:02:12.115391978Z\" level=debug msg=\"request end\" duration=\"87.53µs\" error=EOF name=kata-agent pid=97 request=/grpc.AgentService/ReadStdout resp= sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.152126091Z" level=info msg="time=\"2018-07-26T14:02:12.116122654Z\" level=info msg=\"ignoring unexpected signal\" name=kata-agent pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 signal=\"child exited\" source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.153373071Z" level=info msg="time=\"2018-07-26T14:02:12.117523332Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" exec_id:\\"fbc97a29-4e82-419a-8b39-25e95bc8be83\\" \" request=/grpc.AgentService/WaitProcess sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.154131005Z" level=info msg="time=\"2018-07-26T14:02:12.117774038Z\" level=debug msg=\"request end\" duration=\"40.604µs\" name=kata-agent pid=97 request=/grpc.AgentService/WaitProcess resp= sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.287820877Z" level=info msg="[    4.177139] Unregister pv shared memory for cpu 3
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.291380801Z" level=info msg="[    4.181583] smpboot: CPU 3 is now offline
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.300157035Z" level=info msg="time=\"2018-07-26T14:02:12.264294452Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" resources:<Memory:<> CPU:<Quota:200000 Period:100000 Cpus:\\"1\\" > Pids:<> BlockIO:<> > \" request=/grpc.AgentService/UpdateContainer sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.300571578Z" level=info msg="time=\"2018-07-26T14:02:12.264616783Z\" level=debug msg=\"updating cpuset cgroup\" name=kata-agent path=/sys/fs/cgroup/cpuset/docker pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.300902648Z" level=info msg="time=\"2018-07-26T14:02:12.265050243Z\" level=warning msg=\"Could not update container cpuset cgroup\" error=\"Could not update cpuset cgroup '1': write /sys/fs/cgroup/cpuset/docker/cpuset.cpus: device or resource busy\" name=kata-agent pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.302934397Z" level=info msg="time=\"2018-07-26T14:02:12.267111024Z\" level=debug msg=\"request end\" duration=2.510075ms name=kata-agent pid=97 request=/grpc.AgentService/UpdateContainer resp=\"&Empty{}\" sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.304615623Z" level=debug msg="Copy stream error: %vread unix /run/vc/sbs/e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9/proxy.sock->@: read: connection reset by peer" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=proxy
time="2018-07-26T14:02:12.416537409Z" level=info msg="time=\"2018-07-26T14:02:12.380706804Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" exec_id:\\"53bdc4bb-d2dd-415f-970d-563c05d16fa5\\" process:<User:<> Args:\\"nproc\\" Env:\\"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\\" Env:\\"HOSTNAME=e7701863a690\\" Cwd:\\"/\\" > \" request=/grpc.AgentService/ExecProcess sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.531590032Z" level=info msg="time=\"2018-07-26T14:02:12.49562375Z\" level=debug msg=\"request end\" duration=114.604389ms name=kata-agent pid=97 request=/grpc.AgentService/ExecProcess resp=\"&Empty{}\" sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.534037137Z" level=info msg="time=\"2018-07-26T14:02:12.498014412Z\" level=info msg=\"ignoring unexpected signal\" name=kata-agent pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 signal=\"child exited\" source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.534257302Z" level=info msg="time=\"2018-07-26T14:02:12.498382725Z\" level=info msg=\"ignoring unexpected signal\" name=kata-agent pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 signal=\"child exited\" source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.541947072Z" level=info msg="time=\"2018-07-26T14:02:12.506169439Z\" level=debug msg=\"process exited\" name=kata-agent pid=182 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent status=0
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.542095282Z" level=info msg="time=\"2018-07-26T14:02:12.506321924Z\" level=info msg=\"ignoring unexpected signal\" name=kata-agent pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 signal=\"child exited\" source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.544811592Z" level=info msg="time=\"2018-07-26T14:02:12.509014637Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" exec_id:\\"53bdc4bb-d2dd-415f-970d-563c05d16fa5\\" len:32768 \" request=/grpc.AgentService/ReadStdout sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.544974044Z" level=info msg="time=\"2018-07-26T14:02:12.509225007Z\" level=debug msg=\"request end\" duration=\"27.103µs\" name=kata-agent pid=97 request=/grpc.AgentService/ReadStdout resp=\"data:\\"1\\n\\" \" sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.545148337Z" level=info msg="time=\"2018-07-26T14:02:12.509401387Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" exec_id:\\"53bdc4bb-d2dd-415f-970d-563c05d16fa5\\" len:32768 \" request=/grpc.AgentService/ReadStderr sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.54532403Z" level=info msg="time=\"2018-07-26T14:02:12.509544358Z\" level=debug msg=\"request end\" duration=\"12.012µs\" error=EOF name=kata-agent pid=97 request=/grpc.AgentService/ReadStderr resp= sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.545864866Z" level=info msg="time=\"2018-07-26T14:02:12.50990007Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" exec_id:\\"53bdc4bb-d2dd-415f-970d-563c05d16fa5\\" \" request=/grpc.AgentService/CloseStdin sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.546009559Z" level=info msg="time=\"2018-07-26T14:02:12.510264831Z\" level=debug msg=\"request end\" duration=\"23.579µs\" name=kata-agent pid=97 request=/grpc.AgentService/CloseStdin resp=\"&Empty{}\" sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.547903822Z" level=info msg="time=\"2018-07-26T14:02:12.512125522Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" exec_id:\\"53bdc4bb-d2dd-415f-970d-563c05d16fa5\\" len:32768 \" request=/grpc.AgentService/ReadStdout sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.548069008Z" level=info msg="time=\"2018-07-26T14:02:12.512308804Z\" level=debug msg=\"request end\" duration=\"30.884µs\" error=EOF name=kata-agent pid=97 request=/grpc.AgentService/ReadStdout resp= sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.549874133Z" level=info msg="time=\"2018-07-26T14:02:12.513840452Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" exec_id:\\"53bdc4bb-d2dd-415f-970d-563c05d16fa5\\" \" request=/grpc.AgentService/WaitProcess sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.550039292Z" level=info msg="time=\"2018-07-26T14:02:12.51428426Z\" level=debug msg=\"request end\" duration=\"28.229µs\" name=kata-agent pid=97 request=/grpc.AgentService/WaitProcess resp= sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.656282245Z" level=info msg="time=\"2018-07-26T14:02:12.619451444Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" exec_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" signal:9 \" request=/grpc.AgentService/SignalProcess sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.657419975Z" level=info msg="time=\"2018-07-26T14:02:12.621177796Z\" level=debug msg=\"request end\" duration=\"455.971µs\" name=kata-agent pid=97 request=/grpc.AgentService/SignalProcess resp=\"&Empty{}\" sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.662158788Z" level=info msg="time=\"2018-07-26T14:02:12.62618891Z\" level=debug msg=\"request end\" duration=2.779847358s error=\"read /dev/ptmx: input/output error\" name=kata-agent pid=97 request=/grpc.AgentService/ReadStdout resp= sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.667860826Z" level=info msg="time=\"2018-07-26T14:02:12.628231173Z\" level=debug msg=\"process exited\" name=kata-agent pid=122 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent status=137
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.667910987Z" level=info msg="time=\"2018-07-26T14:02:12.628658072Z\" level=info msg=\"ignoring unexpected signal\" name=kata-agent pid=97 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 signal=\"child exited\" source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.667936704Z" level=info msg="time=\"2018-07-26T14:02:12.629633075Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" exec_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" \" request=/grpc.AgentService/WaitProcess sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.66797612Z" level=info msg="time=\"2018-07-26T14:02:12.629993183Z\" level=debug msg=\"request end\" duration=\"92.197µs\" name=kata-agent pid=97 request=/grpc.AgentService/WaitProcess resp=\"status:137 \" sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.764688424Z" level=info msg="time=\"2018-07-26T14:02:12.727809375Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" signal:9 \" request=/grpc.AgentService/SignalProcess sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.765934473Z" level=info msg="time=\"2018-07-26T14:02:12.729574091Z\" level=info msg=\"discarding signal as container stopped\" containerID=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 name=kata-agent pid=97 sandbox=e7701863a690 signal=killed source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.766443282Z" level=info msg="time=\"2018-07-26T14:02:12.730499745Z\" level=debug msg=\"request end\" duration=1.055028ms name=kata-agent pid=97 request=/grpc.AgentService/SignalProcess resp=\"&Empty{}\" sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.769860448Z" level=info msg="time=\"2018-07-26T14:02:12.733762145Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req=\"container_id:\\"e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9\\" \" request=/grpc.AgentService/RemoveContainer sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.775247948Z" level=info msg="time=\"2018-07-26T14:02:12.735687412Z\" level=debug msg=\"request end\" duration=1.434505ms name=kata-agent pid=97 request=/grpc.AgentService/RemoveContainer resp=\"&Empty{}\" sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.818435614Z" level=info msg="[    4.708277] Unregister pv shared memory for cpu 2
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.820055362Z" level=info msg="[    4.710287] NOHZ: local_softirq_pending 202
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.822955743Z" level=info msg="[    4.712992] smpboot: CPU 2 is now offline
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.835642882Z" level=info msg="[    4.725699] Unregister pv shared memory for cpu 1
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.838960324Z" level=info msg="[    4.729097] smpboot: CPU 1 is now offline
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.845170932Z" level=info msg="time=\"2018-07-26T14:02:12.809280874Z\" level=debug msg=\"new request\" name=kata-agent pid=97 req= request=/grpc.AgentService/DestroySandbox sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.86822113Z" level=info msg="time=\"2018-07-26T14:02:12.831794029Z\" level=debug msg=\"request end\" duration=22.052817ms name=kata-agent pid=97 request=/grpc.AgentService/DestroySandbox resp=\"&Empty{}\" sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=agent
time="2018-07-26T14:02:12.870254503Z" level=fatal msg="channel error" error="accept unix /run/vc/sbs/e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9/proxy.sock: use of closed network connection" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=proxy
time="2018-07-26T14:02:12.870311335Z" level=debug msg="Copy stream error: %vread unix /run/vc/sbs/e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9/proxy.sock->@: use of closed network connection" name=kata-proxy pid=25392 sandbox=e7701863a690c620b639402d274c02b777ef000eaadfef37216fc00db4b052f9 source=proxy
POST BUILD TASK : FAILURE
END OF POST BUILD TASK : 0
Archiving artifacts
Setting status of 9a497fedf56d4ea482a1ae6997e028f2beae4a91 to FAILURE with url http://jenkins.katacontainers.io/job/kata-containers-runtime-fedora-27-PR/281/ and message: 'Build finished. '
Using context: jenkins-ci-fedora-27
Finished: FAILURE

This is from about halfway through - looks like it is debug dumps that are huge?

" name=kata-proxy pid=18007 sandbox=f48da4b1a9f4cdd12c8f212df91a4117a9ef70e5b164ea9aa165d61b93dc4718 source=agent
time="2018-07-26T13:58:33.679399579Z" level=info msg="[    0.525436] initcall dscp_mt_init+0x0/0x11 returned 0 after 0 usecs
" name=kata-proxy pid=18007 sandbox=f48da4b1a9f4cdd12c8f212df91a4117a9ef70e5b164ea9aa165d61b93dc4718 source=agent
time="2018-07-26T13:58:33.679760374Z" level=info msg="[    0.525800] calling  ecn_mt_init+0x0/0x11 @ 1
" name=kata-proxy pid=18007 sandbox=f48da4b1a9f4cdd12c8f212df91a4117a9ef70e5b164ea9aa165d61b93dc4718 source=agent
time="2018-07-26T13:58:33.680131341Z" level=info msg="[    0.526293] initcall ecn_mt_init+0x0/0x11 returned 0 after 0 usecs
" name=kata-proxy pid=18007 sandbox=f48da4b1a9f4cdd12c8f212df91a4117a9ef70e5b164ea9aa165d61b93dc4718 source=agent
time="2018-07-26T13:58:33.68048675Z" level=info msg="[    0.526533] calling  esp_mt_init+0x0/0x11 @ 1
" name=kata-proxy pid=18007 sandbox=f48da4b1a9f4cdd12c8f212df91a4117a9ef70e5b164ea9aa165d61b93dc4718 source=agent
time="2018-07-26T13:58:33.680867159Z" level=info msg="[    0.526951] initcall esp_mt_init+0x0/0x11 returned 0 after 0 usecs
" name=kata-proxy pid=18007 sandbox=f48da4b1a9f4cdd12c8f212df91a4117a9ef70e5b164ea9aa165d61b93dc4718 source=agent
time="2018-07-26T13:58:33.681197578Z" level=info msg="[    0.527337] calling  hashlimit_mt_init+0x0/0x82 @ 1
" name=kata-proxy pid=18007 sandbox=f48da4b1a9f4cdd12c8f212df91a4117a9ef70e5b164ea9aa165d61b93dc4718 source=agent
time="2018-07-26T13:58:33.68164312Z" level=info msg="[    0.527611] initcall hashlimit_mt_init+0x0/0x82 returned 0 after 11 usecs
" name=kata-proxy pid=18007 sandbox=f48da4b1a9f4cdd12c8f212df91a4117a9ef70e5b164ea9aa165d61b93dc4718 source=agent
time="2018-07-26T13:58:33.681984624Z" level=info msg="[    0.528199] calling  helper_mt_init+0x0/0xc @ 1
" name=kata-proxy pid=18007 sandbox=f48da4b1a9f4cdd12c8f212df91a4117a9ef70e5b164ea9aa165d61b93dc4718 source=agent
time="2018-07-26T13:58:33.682392496Z" level=info msg="[    0.528387] initcall helper_mt_init+0x0/0xc returned 0 after 0 usecs
" name=kata-proxy pid=18007 sandbox=f48da4b1a9f4cdd12c8f212df91a4117a9ef70e5b164ea9aa165d61b93dc4718 source=agent
time="2018-07-26T13:58:33.682799778Z" level=info msg="[    0.528785] calling  hl_mt_init+0x0/0x11 @ 1
" name=kata-proxy pid=18007 sandbox=f48da4b1a9f4cdd12c8f212df91a4117a9ef70e5b164ea9aa165d61b93dc4718 source=agent
time="2018-07-26T13:58:33.683367615Z" level=info msg="[    0.529477] initcall hl_mt_init+0x0/0x11 returned 0 after 0 usecs
" name=kata-proxy pid=18007 sandbox=f48da4b1a9f4cdd12c8f212df91a4117a9ef70e5b164ea9aa165d61b93dc4718 source=agent
time="2018-07-26T13:58:33.683784634Z" level=info msg="[    0.529801] calling  comp_mt_init+0x0/0x11 @ 1
" name=kata-proxy pid=18007 sandbox=f48da4b1a9f4cdd12c8f212df91a4117a9ef70e5b164ea9aa165d61b93dc4718 source=agent
time="2018-07-26T13:58:33.684153849Z" level=info msg="[    0.530319] initcall comp_mt_init+0x0/0x11 returned 0 after 0 usecs
" name=kata-proxy pid=18007 sandbox=f48da4b1a9f4cdd12c8f212df91a4117a9ef70e5b164ea9aa165d61b93dc4718 source=agent
time="2018-07-26T13:58:33.684512766Z" level=info msg="[    0.530555] calling  iprange_mt_init+0x0/0x11 @ 1
" name=kata-proxy pid=18007 sandbox=f48da4b1a9f4cdd12c8f212df91a4117a9ef70e5b164ea9aa165d61b93dc4718 source=agent
time="2018-07-26T13:58:33.684869386Z" level=info msg="[    0.530946] initcall iprange_mt_init+0x0/0x11 returned 0 after 0 us

[RFC]: CI scripts only work in a 'clean environment'

I thought I'd open this issue mostly for informational purposes, now we have a number of new CI scenarios coming online.
Marking this as an RFC, as we may wish to discuss if we want or need to fix any of these. They may not all be fixable in a trivial manner. It may not matter if we can get all CIs to use a fresh instance for every build/run.

AFAIK, our CI scripts only work reliably when run in a clean environment. That is, you cannot run a build/test in an environment (bare-metal server or VM for instance) if a build/test/run has previously been run there. There are a number of things that may make this unreliable:

Part of the git checkout/merge/branch cycle cannot always rewind/ffwd, and requests a manual fixup/intervention:

07:20:22 Setup env for kata repository: github.com/kata-containers/runtime
07:20:22 Cloning into '/srv/jenkins/workspace/kata-gitvm-test/workspace/kata-containers-runtime-16.04-PR/go/src/github.com/kata-containers/runtime'...
07:20:22 /srv/jenkins/workspace/kata-gitvm-test/workspace/kata-containers-runtime-16.04-PR/go/src/github.com/kata-containers/runtime /srv/jenkins/workspace/kata-gitvm-test/workspace/kata-containers-runtime-16.04-PR/go/src/github.com/kata-containers/tests
07:20:22 From https://github.com/kata-containers/runtime
07:20:22  * [new ref]         refs/pull/287/head -> PR_287
07:20:22 Switched to branch 'PR_287'
07:20:23 First, rewinding head to replay your work on top of it...
07:20:23 Applying: enable containers in pod
07:20:23 Using index info to reconstruct a base tree...
07:20:23 M	virtcontainers/pkg/oci/utils.go
07:20:23 Falling back to patching base and 3-way merge...
07:20:23 Auto-merging virtcontainers/pkg/oci/utils.go
07:20:23 CONFLICT (content): Merge conflict in virtcontainers/pkg/oci/utils.go
07:20:23 error: Failed to merge in the changes.
07:20:23 Patch failed at 0001 enable containers in pod
07:20:23 The copy of the patch that failed is found in: .git/rebase-apply/patch
07:20:23 
07:20:23 When you have resolved this problem, run "git rebase --continue".
07:20:23 If you prefer to skip this patch, run "git rebase --skip" instead.
07:20:23 To check out the original branch and stop rebasing, run "git rebase --abort".
07:20:23 
07:20:23 Build step 'Execute shell' marked build as failure

The second build/run tries to re-install docker, after removing the /var/lib/docker directory - but quite often that directory still has something present/open on it, and fails to be removed. This may be a symptom of a 'failed build/run' that has left some parts of the runtime active, or has corrupted docker somehow.

06:38:52 Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
06:38:52 manage_ctr_mgr.sh - INFO: docker_info: version: 
06:38:52 manage_ctr_mgr.sh - INFO: docker_info: default runtime: 
06:38:52 manage_ctr_mgr.sh - INFO: docker_info: package name: docker-ce
06:38:52 <13>Jun  6 13:32:41 manage_ctr_mgr.sh: configure_docker: configuring runc as default docker runtime
06:38:52 Stopping the docker service
06:38:52 Removing /var/lib/docker
06:38:52 rm: cannot remove '/var/lib/docker/overlay2/12cb69d61f68d094752778a553a0bbf4b031e6fc2e9e7840f62f7ff5a0f4bcd8/merged': Device or resource busy
06:38:52 rm: cannot remove '/var/lib/docker/containers/6887c52084764169110e6031ad72f7704efca6a7890905f80992b216a8c5ab81/mounts/shm': Device or resource busy

golang installation check fails due to the previously installed go

07:24:14 /usr/local/go/bin/go
07:24:14 ERROR: go version go1.8.3 linux/amd64 is installed, remove it before install go 1.10
07:24:14 Build step 'Execute shell' marked build as failure

Find a way to trigger a Jenkins run for >1 repos simultaneously

Now that kata-containers/tests#300 and kata-containers/runtime#305 have landed, ideally we need a process to update the gometalinter version in https://github.com/kata-containers/runtime/blob/master/versions.yaml.

To do that, we'll need to raise a runtime PR for the versions.yaml change of course, but that will only cause the new version of gometalinter to check the runtime repo.

What we need is a "special" way to cause the runtime PR to trigger a rebuild of all the code repos (runtime, shim, proxy, agent, throttler).

If this isn't possible, we'll have to just periodically update the version of gometalinter locally and test all repos locally before raising the runtime PR.

/cc @chavafg, @sboeuf.

metrics: VMs: disable apt-daily service

Despite removing the unattended-upgrades package we still occasionally see the apt-lock being held, which causes the CI run to fail.
From some surfing, I believe this is down to the apt-daily systemd service - so, let's disable that in our base VM master creation.

qemu config errors in logs

I was setting up some CI, and I spotted some qemu config errors in my logs. Checking on our life systems, over at say http://kata-jenkins-ci.westus2.cloudapp.azure.com/job/kata-containers-tests-ubuntu-16-04-PR/283/consoleFull I see the same issue:

Install Qemu
Cloning into 'qemu'...
package github.com/kata-containers/packaging: no Go files in /home/jenkins/jenkins_slave/workspace/kata-containers-tests-ubuntu-16-04-PR/go/src/github.com/kata-containers/packaging
~/jenkins_slave/workspace/kata-containers-tests-ubuntu-16-04-PR/go/src/github.com/kata-containers/tests/qemu ~/jenkins_slave/workspace/kata-containers-tests-ubuntu-16-04-PR/go/src/github.com/kata-containers/tests
Switched to a new branch 'stable-2.11'
Branch stable-2.11 set up to track remote branch stable-2.11 from origin.
Build Qemu
./configure-hypervisor.sh: line 273: [: -eq: unary operator expected
./configure-hypervisor.sh: line 316: [: -eq: unary operator expected
./configure: --disable-uuid is obsolete, UUID support is always built

Add Fedora 27 CI jobs

We only currently test with Ubuntu, so add Fedora 27 jobs for distro coverage.

Add standard project files

Bootstrap the repo by adding the standard:

Code of Conduct doc.
Contributing doc.
pullapprove config file.

[RFC] - Consensus on Kata CI

Kata Containers CI

Background - existing working solution from Clear Containers:

On Clear Containers we had a couple of iterations on creating a CI system and found that using Jenkins and 'slave nodes' on Azure, leveraging nested virtualization, gave us the most external visibility and flexibility. Jenkins snapshot can be seen at [1] and actual Jenkins at [2].

Tests leveraged can be seen at [3].

[1] - https://github.com/clearcontainers/jenkins
[2] - http://cc-jenkins-ci.westus2.cloudapp.azure.com
[3] - https://github.com/clearcontainers/tests

Current status for CI in Kata

As far as I can tell, we haven't had an explicit discussion on CI setup for Kata. Currently, engineers on the Intel side have contributed many tests at [4], and this should already contain the existing test cases from runv and cc-runtime. We have run our CI suite locally on Kata (does surprisingly well!) and we are looking to move forward to enable CI.

[4] - https://github.com/kata-containers/tests

Recommended setup for CI

With the help of our architecture committee (thanks @jessfraz), we have credits in place to be able to setup our CI in Azure, mirroring what we have done already for Clear Containers, only %s/Clear/Kata, more or less. I am of the opinion that this meets our needs very well and already has been proven in the Clear Container program and can be brought up very quickly. Having said that, I want to make sure we get feedback on this so we can have consensus and move forward.

cc/ @sboeuf @chavafg @jessfraz @gnawux @jodh-intel @bergwolf @WeiZhang555 @sameo @tallclair @laijs @GabyCT @grahamwhaley

Add scripts to generate VMs for metrics CI builds

We run our metrics (https://github.com/kata-containers/tests/tree/master/metrics) CI on bare metal machines, as we require consistent and reliable results (which you generally cannot get from cloud instances), but this presents a small issue - if (or rather, when) we get a rogue PR, it can corrupt the host installation (for instance corrupting docker or leaving parts of the runtime laying around and consuming resource etc.).
To isolate the PR builds, we are going to run the metrics withing VMs on the bare metal servers.

To that end, we will need some scripts to generate suitable base VMs configured to run the metrics, and perform the VM clone/deletes per-PR build/run. Add those scripts somewhere to this repo.

VMs: .gitconfig file not set up correctly

A copy/paste/typo has meant we ended up writing to the file
/home/USERNAME/bin/gitproxy
rather than the
/home/USERNAME/.gitconfig

we intended to. See
https://github.com/kata-containers/ci/blob/master/VMs/metrics/ubuntu16.04/user-data#L47

It looks like those two files contents are swapped in the proxy file example:
https://github.com/kata-containers/ci/blob/master/VMs/metrics/ubuntu16.04/user-data.proxy#L77-L108
which is likely where the copy error came from.

https://wiki.jenkins.io/display/JENKINS/Pipeline+Stage+View+Plugin

It looks like we can create individual stages for arbitrary commands meaning we can make it as granular as we want ;)

Originally raised as: clearcontainers/jenkins#7

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

kata-containers / ci Goto Github PK

ci's Introduction

Kata Containers

Introduction

License

Platform support

Hardware requirements

Getting started

Documentation

Configuration

Hypervisors

Community

Getting help

Raising issues

Developers

Components

Main components

Additional components

Packaging and releases

General tests

Metrics tests

Glossary of Terms

ci's People

Contributors

Stargazers

Watchers

Forkers

ci's Issues

Coverage

Basic tests

Packages

Operational

Kata Containers CI

Background - existing working solution from Clear Containers:

Current status for CI in Kata

Recommended setup for CI

Recommend Projects

Recommend Topics

Recommend Org