containers / conmon-rs Goto Github PK
View Code? Open in Web Editor NEWAn OCI container runtime monitor written in Rust
License: Apache License 2.0
An OCI container runtime monitor written in Rust
License: Apache License 2.0
We right now use the default logger in CRI-O, which logs to systemd. It would be good to have at least a runtime monitor configuration to also log to stdout. This allows us to collect conmon and CRI-O logs in parallel.
Q: Do we wanna make stdout logging the default? Which benefits do we gain when logging to systemd?
We should actively work on increasing the unit test coverage by using the mockall framework. This should be done in the same way like we did it for the server init:
conmon-rs/conmon-rs/server/src/init.rs
Lines 11 to 12 in e6084da
conmon-rs/conmon-rs/server/src/init.rs
Lines 72 to 92 in e6084da
I was adding a RPC request to return RSS memory and noticed the RSS usage goes up in the server after the first GRPC request... significantly.
Terminal 1:
# target/release/conmon-server --runtime /bin/ls
Terminal 2:
# cat /proc/$(pidof conmon-server)/status|grep RSS
VmRSS: 1084 kB
# target/release/conmon-client
Version: Response { metadata: MetadataMap { headers: {"content-type": "application/grpc", "date": "Wed, 10 Nov 2021 21:55:01 GMT", "grpc-status": "0"} }, message: VersionResponse { version: "0.1.0" }, extensions: Extensions }
# cat /proc/$(pidof conmon-server)/status|grep RSS
VmRSS: 3464 kB
currently, there's a bug where running with multiple exec sync or containers causes conmonrs to leak threads. This is from the streams::read_loop_single_stream reader.read
call. we're doing a normal blocking read, and waiting on I/O that will never come (nor be canceled).
We need to either not block on the IO, use an event based system, or be able to cancel the read when we know there are no more messages
The test case flakes from time to time with the following error:
ConmonClient CreateContainer
should kill created children if being killed
/home/runner/work/conmon-rs/conmon-rs/pkg/client/client_test.go:67
2022-04-20T09:52:35.865Z INFO [conmon::server] Received SIGINT
2022-04-20T09:52:35.866Z INFO [conmon::server] Using stdout logger
2022-04-20T09:52:35.866Z INFO [conmon::server] Set log level to: DEBUG
2022-04-20T09:52:35.866Z INFO [conmon::init] Missing sufficient privileges to adjust OOM score
2022-04-20T09:52:35.867Z DEBUG [conmon::rpc] Got a version request
2022-04-20T09:52:35.869Z DEBUG [conmon::rpc] Got a create container request for id 5e1b73e9b08c3fd32c51213b8a51e9070e872589be2c3071cd766b9a76e72043
2022-04-20T09:52:35.869Z DEBUG [conmon::streams] Creating new IO streams
2022-04-20T09:52:35.869Z DEBUG [conmon::rpc] PID file is /tmp/conmon-client1596343358/pidfile
2022-04-20T09:52:35.869Z DEBUG [conmon::server] Runtime args "--root=/tmp/conmon-client1596343358/root create --bundle /tmp/conmon-client1596343358 --pid-file /tmp/conmon-client1596343358/pidfile 5e1b73e9b08c3fd32c51213b8a51e9070e872589be2c3071cd766b9a76e72043"
2022-04-20T09:52:35.870Z DEBUG [conmon::cri_logger] Initializing CRI logger in path /tmp/conmon-client1596343358/log
2022-04-20T09:52:35.873Z DEBUG [conmon::streams] Start reading from IO streams
2022-04-20T09:52:35.[959](https://github.com/containers/conmon-rs/runs/6092649279?check_suite_focus=true#step:7:959)Z INFO [conmon::server] Received SIGINT
------------------------------
• [FAILED] [10.210 seconds]
ConmonClient
/home/runner/work/conmon-rs/conmon-rs/pkg/client/client_test.go:16
CreateContainer
/home/runner/work/conmon-rs/conmon-rs/pkg/client/client_test.go:48
[It] should kill created children if being killed
/home/runner/work/conmon-rs/conmon-rs/pkg/client/client_test.go:67
Timed out after 10.005s.
Expected
<*errors.errorString | 0xc00045f310>: {
s: "Expected stopped to be a substr of ID PID STATUS BUNDLE CREATED OWNER\n5e1b73e9b08c3fd32c51213b8a51e9070e872589be2c3071cd766b9a76e72043 10637 created /tmp/conmon-client15[963](https://github.com/containers/conmon-rs/runs/6092649279?check_suite_focus=true#step:7:963)43358 2022-04-20T09:52:35.916020963Z runner\n",
}
to be nil
In [It] at: /home/runner/work/conmon-rs/conmon-rs/pkg/client/client_test.go:78
conmon-rs/pkg/client/client_test.go
Lines 67 to 79 in 0308b06
Is there a way to get rid of gettext-sys? We use it for setting the locale, but the compile times with it seem to be quite long.
If you want youki to provide the cgroup crate and more, I'd love to work with you.
We print the active goroutines after each integration test. It looks like that the util.CopyDetachable
still leaves a running goroutine open:
2022-07-20T14:16:05.7893221Z 2 @ 0x44c076 0x45d112 0x4b882e 0x4b92e5 0x654416 0x6dc739 0x47dfc1
2022-07-20T14:16:05.7893648Z # 0x4b882d io.(*pipe).read+0x14d /opt/hostedtoolcache/go/1.18.4/x64/src/io/pipe.go:57
2022-07-20T14:16:05.7894140Z # 0x4b92e4 io.(*PipeReader).Read+0x64 /opt/hostedtoolcache/go/1.18.4/x64/src/io/pipe.go:136
2022-07-20T14:16:05.7895004Z # 0x654415 github.com/containers/common/pkg/util.CopyDetachable+0xb5 /home/runner/go/pkg/mod/github.com/containers/[email protected]/pkg/util/copy.go:16
2022-07-20T14:16:05.7895944Z # 0x6dc738 github.com/containers/conmon-rs/pkg/client.(*ConmonClient).setupStdioChannels.func2+0xb8 /home/runner/work/conmon-rs/conmon-rs/pkg/client/attach.go:201
conmon-rs/pkg/client/attach.go
Lines 198 to 204 in 13d4828
We should find a way to cleanup the goroutine on server shutdown as well.
No running goroutines for attach after the integration tests.
Running the integration tests reveals the issue by printing the goroutines at the end.
No
$ conmonrs --version
version: 0.1.0-dev
tag: none
commit: 68252d0373b8e262bdf7ff8780fb2b5ab6f66c29
build: 2022-07-20 14:13:13 +00:00
rustc 1.62.1 (e092d0b6b 2022-07-16)
Not relevant
No
Given that podman likely wants to import this go client lib as well (containers/podman#14804) this lib should not depend on any podman code. While it can work the circular dependency is just asking for trouble when podman tries to make breaking changes to the code.
I propose that we move the required podman code to c/common and then podman and conmon-rs can use it without trouble.
The critest conformance test runtime should support attach [Conformance]
fails by timing out:
Right now the test got excluded by CRI-O: https://github.com/cri-o/cri-o/blob/db76322414a7888c7642b707e07dbb9905aa969f/.github/workflows/integration.yml#L57
We either have to fix that on the CRI-O or the conmon-rs side. From the logs I see that conmonrs gets the data from the test:
May 11 10:41:06 nixos conmonrs[504862]: Got a attach container request
May 11 10:41:06 nixos conmonrs[504862]: Creating attach socket: /var/lib/containers/storage/overlay-containers/c16e58ee6177513959550004b949d71472b0cca52ca603bd3764195d22590b59/userdata/ee14eb08f450ffda35dd689411663b8b19d3d7947e695844e49ecca5ce32dc0d/attach
May 11 10:41:06 nixos conmonrs[504862]: Start listening on attach socket
May 11 10:41:06 nixos conmonrs[504862]: Got new attach stream connection
May 11 10:41:07 nixos conmonrs[504862]: Read 14 stdin bytes from client
May 11 10:41:07 nixos conmonrs[504862]: fd:23:read 5 bytes
May 11 10:41:08 nixos conmonrs[504862]: Wrote stdout packet to client
I am currently trying to integrate conmon-rs into Podman. I am trying to get the attach functionality working. When I run the command sudo bin/podman run --log-driver=k8s-file --name=demo -d alpine top
, the container successfully starts and the logs display that there are bytes being read:
2022-08-11T18:14:42.317436Z DEBUG backend:create_container{container_id="a186f94a8b5cd7953e3fe191a0bbf2dfbb8e4199c2dd3c1dd3a05c6ac44674f5" uuid="1c4c62f4-c230-485f-895e-aee634fe43a6"}:promise:stdout: conmonrs::container_io: 238: Read 304 bytes
2022-08-11T18:14:47.322829Z DEBUG backend:create_container{container_id="a186f94a8b5cd7953e3fe191a0bbf2dfbb8e4199c2dd3c1dd3a05c6ac44674f5" uuid="1c4c62f4-c230-485f-895e-aee634fe43a6"}:promise:stdout: conmonrs::container_io: 238: Read 304 bytes
2022-08-11T18:14:52.327820Z DEBUG backend:create_container{container_id="a186f94a8b5cd7953e3fe191a0bbf2dfbb8e4199c2dd3c1dd3a05c6ac44674f5" uuid="1c4c62f4-c230-485f-895e-aee634fe43a6"}:promise:stdout: conmonrs::container_io: 238: Read 304 bytes
When I try and attach to the container doing sudo bin/podman attach demo
, it attaches and briefly displays the output of top:
Mem: 15176388K used, 50390388K free, 1378696K shrd, 4192K buff, 9559136K cached
CPU: 1% usr 0% sys 0% nic 98% idle 0% io 0% irq 0% sirq
Load average: 0.26 0.45 0.47 1/1736 1
PID PPID USER STAT VSZ %VSZ CPU %CPU COMMAND
1 0 root R 1604 0% 10 0% top
However, after this output displays, the container immediately detaches even though the logs show that conmon-rs is still reading bytes.
Log output when attaching:
2022-08-11T18:16:01.775594Z DEBUG backend: conmonrs::rpc: 50: Got a version request
2022-08-11T18:16:01.775814Z DEBUG backend:attach_container{container_id="a186f94a8b5cd7953e3fe191a0bbf2dfbb8e4199c2dd3c1dd3a05c6ac44674f5" uuid="fe2068d0-b224-47ac-9a90-a321b23225ef"}: conmonrs::rpc: 238: Got a attach container request
2022-08-11T18:16:01.775826Z DEBUG backend:attach_container{container_id="a186f94a8b5cd7953e3fe191a0bbf2dfbb8e4199c2dd3c1dd3a05c6ac44674f5" uuid="fe2068d0-b224-47ac-9a90-a321b23225ef"}:promise: conmonrs::attach: 116: Creating attach socket: /var/lib/containers/storage/overlay-containers/a186f94a8b5cd7953e3fe191a0bbf2dfbb8e4199c2dd3c1dd3a05c6ac44674f5/userdata/attach
2022-08-11T18:16:01.775897Z DEBUG backend:attach_container{container_id="a186f94a8b5cd7953e3fe191a0bbf2dfbb8e4199c2dd3c1dd3a05c6ac44674f5" uuid="fe2068d0-b224-47ac-9a90-a321b23225ef"}:promise:attach: conmonrs::attach: 158: Start listening on attach socket
2022-08-11T18:16:01.775968Z DEBUG backend:attach_container{container_id="a186f94a8b5cd7953e3fe191a0bbf2dfbb8e4199c2dd3c1dd3a05c6ac44674f5" uuid="fe2068d0-b224-47ac-9a90-a321b23225ef"}:promise:attach: conmonrs::attach: 163: Got new attach stream connection
2022-08-11T18:16:02.402098Z DEBUG backend:create_container{container_id="a186f94a8b5cd7953e3fe191a0bbf2dfbb8e4199c2dd3c1dd3a05c6ac44674f5" uuid="1c4c62f4-c230-485f-895e-aee634fe43a6"}:promise:stdout: conmonrs::container_io: 238: Read 304 bytes
2022-08-11T18:16:02.402554Z DEBUG backend:attach_container{container_id="a186f94a8b5cd7953e3fe191a0bbf2dfbb8e4199c2dd3c1dd3a05c6ac44674f5" uuid="fe2068d0-b224-47ac-9a90-a321b23225ef"}:promise:attach:write_loop: conmonrs::attach: 254: Wrote stdout packet 0/1 to client
2022-08-11T18:16:02.402594Z DEBUG backend:attach_container{container_id="a186f94a8b5cd7953e3fe191a0bbf2dfbb8e4199c2dd3c1dd3a05c6ac44674f5" uuid="fe2068d0-b224-47ac-9a90-a321b23225ef"}:promise:attach:write_loop: conmonrs::attach: 254: Wrote stdout packet 1/1 to client
2022-08-11T18:16:02.402956Z DEBUG backend:attach_container{container_id="a186f94a8b5cd7953e3fe191a0bbf2dfbb8e4199c2dd3c1dd3a05c6ac44674f5" uuid="fe2068d0-b224-47ac-9a90-a321b23225ef"}:promise:attach:read_loop: conmonrs::attach: 203: Stopping read loop because no more data to read
2022-08-11T18:16:07.407616Z DEBUG backend:create_container{container_id="a186f94a8b5cd7953e3fe191a0bbf2dfbb8e4199c2dd3c1dd3a05c6ac44674f5" uuid="1c4c62f4-c230-485f-895e-aee634fe43a6"}:promise:stdout: conmonrs::container_io: 238: Read 304 bytes
2022-08-11T18:16:12.413399Z DEBUG backend:create_container{container_id="a186f94a8b5cd7953e3fe191a0bbf2dfbb8e4199c2dd3c1dd3a05c6ac44674f5" uuid="1c4c62f4-c230-485f-895e-aee634fe43a6"}:promise:stdout: conmonrs::container_io: 238: Read 304 bytes
Matt Heon and I believe there is a synchronization issue somewhere server side.
I expected the container to remain attached and continue outputting the top
data
The code I've been writing to integrate conmon-rs into Podman can be found in this PR: containers/podman#14930
This has been tried with prior versions of conmon-rs and didn't work either.
$ conmonrs --version
version: 0.1.0
tag: none
commit: 55285cb357adbbdf4ebd8d54076ca8db6da69682
build: 2022-08-11 18:26:23 +00:00
rustc 1.62.0 (a8314ef7d 2022-06-27)
# On Linux:
$ cat /etc/os-release
NAME="Fedora Linux"
VERSION="36 (Workstation Edition)"
ID=fedora
VERSION_ID=36
VERSION_CODENAME=""
PLATFORM_ID="platform:f36"
PRETTY_NAME="Fedora Linux 36 (Workstation Edition)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:36"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f36/system-administrators-guide/"
SUPPORT_URL="https://ask.fedoraproject.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=36
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=36
PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy"
VARIANT="Workstation Edition"
VARIANT_ID=workstation
$ uname -a
Linux fedora 5.18.16-200.fc36.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Aug 3 15:44:49 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Client: Podman Engine
Version: 4.2.0-dev
API Version: 4.2.0-dev
Go Version: go1.18.2
Git Commit: 1ada01a038fceaa9e94beb2de6e4593df03be7fa-dirty
Built: Tue Jun 7 12:48:55 2022
OS/Arch: linux/amd64
$ go version
go version go1.18.4 linux/amd64
$ rustup --version
rustup 1.24.3 (ce5817a94 2021-05-31)
$ rustc --version
rustc 1.62.0 (a8314ef7d 2022-06-27)
When logging to stdout then we usually add a full tracing context to each RPC:
2022-05-04T12:22:35.208960Z DEBUG create_container{container_id="326d99c7850ba440f0faafeb30f875ca448eac74706ee9a03fbf45c33e86b5cc" uuid="b5ce0501-c4b3-4fbb-8453-deb02f133ff5"}: conmon::rpc: 72: Got a create container request
This information seems to be lost when using the systemd logger:
May 11 10:41:06 nixos conmonrs[504862]: Got a create container request
We should investigate if we're misugins the tracing API or if it's related to tokio-rs/tracing#2051
Target is to run the integration tests in an environment which is closer to the actual deployment (OpenShift), to have more realistic performance test results.
Relevant:
Refers to #22
We need a logo for conmon-rs to be easily identifiable compared to conmon.
We should consider to use the OpenShift prow instance for this repository to handle our whole development workflow.
the rust logger allows us to attach metadata. it would be useful to track different messages based on container id, or pid if that's not possible, so we know which container did what
CRI-O GitHub actions e2e tests: https://github.com/cri-o/cri-o/actions/runs/3089802460/jobs/4998634317
A bunch of tests asserting container logs:
Summarizing 6 Failures:
[14113](https://github.com/cri-o/cri-o/actions/runs/3089802460/jobs/4998634317#step:5:14114)
[FAIL] [sig-node] Variable Expansion [It] should allow composing env vars into new env vars [NodeConformance] [Conformance]
[14114](https://github.com/cri-o/cri-o/actions/runs/3089802460/jobs/4998634317#step:5:14115)
test/e2e/framework/util.go:773
[14115](https://github.com/cri-o/cri-o/actions/runs/3089802460/jobs/4998634317#step:5:14116)
[FAIL] [sig-node] Downward API [It] should provide pod name, namespace and IP address as env vars [NodeConformance] [Conformance]
[14116](https://github.com/cri-o/cri-o/actions/runs/3089802460/jobs/4998634317#step:5:14117)
test/e2e/framework/util.go:773
[14117](https://github.com/cri-o/cri-o/actions/runs/3089802460/jobs/4998634317#step:5:14118)
[FAIL] [sig-storage] EmptyDir volumes when FSGroup is specified [LinuxOnly] [NodeFeature:FSGroup] [It] new files should be created with FSGroup ownership when container is non-root
[14118](https://github.com/cri-o/cri-o/actions/runs/3089802460/jobs/4998634317#step:5:14119)
test/e2e/framework/util.go:773
[14119](https://github.com/cri-o/cri-o/actions/runs/3089802460/jobs/4998634317#step:5:14120)
[FAIL] [sig-node] Sysctls [LinuxOnly] [NodeConformance] [It] should support sysctls [MinimumKubeletVersion:1.21] [Conformance]
[14120](https://github.com/cri-o/cri-o/actions/runs/3089802460/jobs/4998634317#step:5:14121)
test/e2e/common/node/sysctl.go:114
[14121](https://github.com/cri-o/cri-o/actions/runs/3089802460/jobs/4998634317#step:5:14122)
[FAIL] [sig-node] Pods [It] should contain environment variables for services [NodeConformance] [Conformance]
[14122](https://github.com/cri-o/cri-o/actions/runs/3089802460/jobs/4998634317#step:5:14123)
test/e2e/common/node/pods.go:524
[14123](https://github.com/cri-o/cri-o/actions/runs/3089802460/jobs/4998634317#step:5:14124)
[FAIL] [sig-storage] Projected combined [It] should project all components that make up the projection API [Projection][NodeConformance] [Conformance]
I’m unsure, they passed a while before we merged the attach fixes.
I assume that we have not written all logs on container termination and race between the CRI logger and child reaper. We probably need something to wait for the logs before exiting conmon-rs.
No response
There are some issues with tokio::select, referring to #317 (comment)
We should consider replacing it with other solutions.
Hi, when I want to run this project with cargo run, it failed, even if I specific --bin like below:
cargo run --bin=/home/zk/go/src/github.com/containers/conmon-rs/target/debug
It get this error :
error: no bin target named /home/zk/go/src/github.com/containers/conmon-rs/target/debug/conmonrs
Run successfully
cd containers/conmon-rs
cargo run --bin=/home/zk/go/src/github.com/containers/conmon-rs/target/debug
No response
$ conmonrs --version
# paste output here
# On Linux:
$ cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.3 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.3 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
$ uname -a
inux ubuntu 5.13.0-41-generic #46~20.04.1-Ubuntu SMP Wed Apr 20 13:16:21 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Allow conmon-rs to process container stdout/err when just streaming from the streams. I consider merging the IOStreams implementation with the console and make it configurable.
The builds fails since a couple of commits with the error message:
DEBUG util.py:445: No match for argument: golang-1.18.6-1.fc35.i386
DEBUG util.py:443: Error: Unable to find a match: golang-1.18.6-1.fc35.i386
The build should work like the other ones.
Mostly on every recent commit on https://copr.fedorainfracloud.org/coprs/rhcontainerbot/podman-next/package/conmon-rs/
No response
$ conmonrs --version
# paste output here
# On Linux:
$ cat /etc/os-release
# paste output here
$ uname -a
# paste output here
A bunch of CentOS and epel 8 package builds are recently failing:
https://copr.fedorainfracloud.org/coprs/rhcontainerbot/podman-next/build/4944707/
The main error message seems to be:
Error: Unable to find a match: rust-1.61.0-1.el8.x86_64, golang-1.18.7-1.el8.x86_64
@lsm5 we can switch back to rust 1.58 if you don't mind. We already changed the repo to support that.
CRI-O conmonrs e2e feature tests.
Mentioned in the title, ran via: https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/cri-o_cri-o/6220/pull-ci-cri-o-cri-o-main-ci-e2e-conmonrs/1572188830188965888
Since ever, reproducible with 8e4cec9
I guess it’s something in the attach stdin code.
No response
To satisfy the CRI-O runtime method AttachContainer, a method should be created that requests an attach socket be created for a container or exec session.
there are tons of RPC frameworks to choose from. we've flirted with the idea of grpc, ttrpc, cap n proto. we should probably just pick one. any other ideas?
We should give the README.md some love and reflect the current state of the project.
This is just an idea. I'd like to discuss this with the conmon-rs team.
Maybe you know creating a cgroup takes a cost, actually, it is one of the most time-consuming tasks. in the container runtime following OCI runtime spec.
Youki previously considered creating the cgroup asynchronously with io_uring, but this did not yield very good results.
However, if it is a daemon like a server, there should be enough time to create it in advance. Whereas there, the container runtime should be able to skip the cgroup creation process by creating the process with clone3
. Wdyt?
This idea is inspired from:
Thus reducing the amount of exec calls that must happen in the container engine, and reducing the amount of memory it uses.
We cannot reattach to a container because the attach path already exists:
FATA[0000] attaching running container failed: Internal error occurred: error attaching to container: create result: conmon-rs/common/proto/conmon.capnp:Conmon.attachContainer: create attach endpoint: Attach socket path already exists: /run/containers/storage/overlay-containers/1ac6dc4e679069ba32bfbd4307435da7551de15ad81e5fff6bc54757badafed1/userdata/2469c120ea14e5d8112e3886a514e1b86262841dac76a4011b008a0a79d55072/attach
We should be able to re-attach containers.
Run a workload via CRI-O and crictl and try to attach twice.
No response
# On Linux:
$ cat /etc/os-release
# paste output here
$ uname -a
# paste output here
pinns is a utility called by cri-o to create pod-level namespaces. It would be cool if conmon-rs could create those namespaces for the pod. Most notably, if it could manage a pause
process to hold open the pod's PID namespace, then CRI-O could get rid of the infra container
Minor issue on s390x packaging builds.
Compiling conmonrs v0.1.0 (/builddir/build/BUILD/conmon-rs/conmon-rs/server)
error[E0308]: mismatched types
--> conmon-rs/server/src/oom_watcher.rs:19:48
|
19 | pub const CGROUP2_SUPER_MAGIC: FsType = FsType(libc::CGROUP2_SUPER_MAGIC as i64);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `u32`, found `i64`
For more information about this error, try `rustc --explain E0308`.
error: could not compile `conmonrs` due to previous error
make: *** [Makefile:17: release] Error 101
error: Bad exit status from /var/tmp/rpm-tmp.iFW7as (%build)
I expect s390x packages to build correctly.
N/A
No response
None
$ conmonrs --version
# paste output here
None
# On Linux:
$ cat /etc/os-release
# paste output here
$ uname -a
# paste output here
None
The recent builds fail on each main
commit: https://copr.fedorainfracloud.org/coprs/rhcontainerbot/podman-next/build/4788630/
Every copr build on main
I don't think they ever worked.
No response
No response
cc @rphillips @lsm5
The test fails in cri-o/cri-o#6220 together with other cases, but I assume that the root cause is the same.
Interestingly, on my local machine the cancellation of the execsync token seems to kill the main container PID.
That the test succeeds.
> k8s-test-run --ginkgo.focus "Kubectl Port forwarding With a server listening on localhost that expects a client request should support a client that connects, sends DATA, and disconnects"
…
[BeforeEach] [sig-cli] Kubectl Port forwarding
set up framework | framework.go:158
STEP: Creating a kubernetes client 09/16/22 11:08:56.88
Sep 16 11:08:56.880: INFO: >>> kubeConfig: /var/run/kubernetes/admin.kubeconfig
STEP: Building a namespace api object, basename port-forwarding 09/16/22 11:08:56.881
STEP: Waiting for a default service account to be provisioned in namespace 09/16/22 11:08:56.885
STEP: Waiting for kube-root-ca.crt to be provisioned in namespace 09/16/22 11:08:56.887
[It] should support a client that connects, sends DATA, and disconnects
test/e2e/kubectl/portforward.go:481
STEP: Creating the target pod 09/16/22 11:08:56.888
Sep 16 11:08:56.892: INFO: Waiting up to 5m0s for pod "pfpod" in namespace "port-forwarding-8233" to be "running and ready"
Sep 16 11:08:56.893: INFO: Pod "pfpod": Phase="Pending", Reason="", readiness=false. Elapsed: 1.353931ms
Sep 16 11:08:56.893: INFO: The phase of Pod pfpod is Pending, waiting for it to be Running (with Ready = true)
Sep 16 11:08:58.896: INFO: Pod "pfpod": Phase="Running", Reason="", readiness=false. Elapsed: 2.003699716s
Sep 16 11:08:58.896: INFO: The phase of Pod pfpod is Running (Ready = false)
Sep 16 11:09:00.896: INFO: Pod "pfpod": Phase="Running", Reason="", readiness=false. Elapsed: 4.003966771s
Sep 16 11:09:00.896: INFO: The phase of Pod pfpod is Running (Ready = false)
Sep 16 11:09:02.895: INFO: Pod "pfpod": Phase="Running", Reason="", readiness=true. Elapsed: 6.003036101s
Sep 16 11:09:02.895: INFO: The phase of Pod pfpod is Running (Ready = true)
Sep 16 11:09:02.895: INFO: Pod "pfpod" satisfied condition "running and ready"
STEP: Running 'kubectl port-forward' 09/16/22 11:09:02.895
Sep 16 11:09:02.895: INFO: starting port-forward command and streaming output
Sep 16 11:09:02.895: INFO: Asynchronously running '/home/sascha/go/src/k8s.io/kubernetes/_output/local/bin/linux/amd64/kubectl kubectl --server=https://localhost:6443/ --kubeconfig=/var/run/kubernetes/admin.kubeconfig --namespace=port-forwarding-8233 port-forward --namespace=port-forwarding-8233 pfpod :80'
Sep 16 11:09:02.895: INFO: reading from `kubectl port-forward` command's stdout
STEP: Dialing the local port 09/16/22 11:09:02.951
STEP: Sending the expected data to the local port 09/16/22 11:09:02.952
STEP: Reading data from the local port 09/16/22 11:09:02.952
STEP: Closing the write half of the client's connection 09/16/22 11:09:04.859
STEP: Waiting for the target pod to stop running 09/16/22 11:09:04.859
Sep 16 11:09:04.859: INFO: Waiting up to 5m0s for pod "pfpod" in namespace "port-forwarding-8233" to be "container terminated"
Sep 16 11:09:04.860: INFO: Pod "pfpod": Phase="Running", Reason="", readiness=false. Elapsed: 1.185975ms
Sep 16 11:09:04.860: INFO: Pod "pfpod" satisfied condition "container terminated"
STEP: Verifying logs 09/16/22 11:09:04.86
…
[ hangs for 120s ]
We can see that the port-forwarder is dead within the container:
> kubectl get pods -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-567b6dd84-bt6pf 1/1 Running 0 2m50s
port-forwarding-5552 pfpod 0/2 Error 0 12s
> ps aux | rg agnhost
root 1000821 0.0 0.0 740048 24788 ? Ssl 11:10 0:00 /agnhost netexec
And the conmonrs logs also indicate that it got terminated:
> sudo journalctl -f _COMM=conmonrs --since=now
conmonrs[1028242]: Using systemd/journald logger
conmonrs[1028242]: Set log level to: trace
conmonrs[1028243]: registering event source with poller: token=Token(0), interests=READABLE
conmonrs[1028243]: registering event source with poller: token=Token(1), interests=READABLE | WRITABLE
conmonrs[1028243]: registering event source with poller: token=Token(2), interests=READABLE | WRITABLE
conmonrs[1028243]: Got a version request
conmonrs[1028243]: deregistering event source from poller
conmonrs[1028243]: registering event source with poller: token=Token(16777218), interests=READABLE | WRITABLE
conmonrs[1028243]: Got a version request
conmonrs[1028243]: deregistering event source from poller
conmonrs[1028243]: registering event source with poller: token=Token(33554434), interests=READABLE | WRITABLE
conmonrs[1028243]: Got a create container request
conmonrs[1028243]: Creating new IO streams
conmonrs[1028243]: PID file is /run/containers/storage/overlay-containers/1d9bbf813ae2ed8bd89e64f2160f973e7bfecd1311876c4c8d0f5af1eeb471b7/userdata/pidfile
conmonrs[1028243]: Runtime args "--root=/run/runc create --bundle /run/containers/storage/overlay-containers/1d9bbf813ae2ed8bd89e64f2160f973e7bfecd1311876c4c8d0f5af1eeb471b7/userdata --pid-file /run/containers/storage/overlay-containers/1d9bbf813ae2ed8bd89e64f2160f973e7bfecd1311876c4c8d0f5af1eeb471b7/userdata/pidfile 1d9bbf813ae2ed8bd89e64f2160f973e7bfecd1311876c4c8d0f5af1eeb471b7"
conmonrs[1028243]: Initializing CRI logger in path /var/log/pods/port-forwarding-2789_pfpod_39410d99-a33c-4a12-bd3b-0a47e1fb0d9c/readiness/0.log
conmonrs[1028243]: registering event source with poller: token=Token(3), interests=READABLE | WRITABLE
conmonrs[1028243]: registering event source with poller: token=Token(4), interests=READABLE | WRITABLE
conmonrs[1028243]: registering event source with poller: token=Token(5), interests=READABLE | WRITABLE
conmonrs[1028243]: Start reading from IO streams
conmonrs[1028243]: Running task
conmonrs[1028243]: Using cgroup path: /proc/1028268/cgroup
conmonrs[1028243]: Waiting for exit code
conmonrs[1028243]: deregistering event source from poller
conmonrs[1028243]: Setup cgroup v2 handling
conmonrs[1028243]: registering event source with poller: token=Token(0), interests=READABLE
conmonrs[1028243]: Read 150 bytes
conmonrs[1028243]: Wrote log line of length 120
conmonrs[1028243]: Wrote log line of length 120
conmonrs[1028243]: registering event source with poller: token=Token(50331650), interests=READABLE | WRITABLE
conmonrs[1028243]: Got a create container request
conmonrs[1028243]: Creating new IO streams
conmonrs[1028243]: PID file is /run/containers/storage/overlay-containers/a4100c860bee5c960903153702b486bd4ada8c6c6a77374b8edfc1afd55b872d/userdata/pidfile
conmonrs[1028243]: Runtime args "--root=/run/runc create --bundle /run/containers/storage/overlay-containers/a4100c860bee5c960903153702b486bd4ada8c6c6a77374b8edfc1afd55b872d/userdata --pid-file /run/containers/storage/overlay-containers/a4100c860bee5c960903153702b486bd4ada8c6c6a77374b8edfc1afd55b872d/userdata/pidfile a4100c860bee5c960903153702b486bd4ada8c6c6a77374b8edfc1afd55b872d"
conmonrs[1028243]: Initializing CRI logger in path /var/log/pods/port-forwarding-2789_pfpod_39410d99-a33c-4a12-bd3b-0a47e1fb0d9c/portforwardtester/0.log
conmonrs[1028243]: registering event source with poller: token=Token(6), interests=READABLE | WRITABLE
conmonrs[1028243]: registering event source with poller: token=Token(7), interests=READABLE | WRITABLE
conmonrs[1028243]: registering event source with poller: token=Token(8), interests=READABLE | WRITABLE
conmonrs[1028243]: Start reading from IO streams
conmonrs[1028243]: Running task
conmonrs[1028243]: Using cgroup path: /proc/1028312/cgroup
conmonrs[1028243]: Waiting for exit code
conmonrs[1028243]: deregistering event source from poller
conmonrs[1028243]: Setup cgroup v2 handling
conmonrs[1028243]: registering event source with poller: token=Token(0), interests=READABLE
conmonrs[1028243]: registering event source with poller: token=Token(67108866), interests=READABLE | WRITABLE
conmonrs[1028243]: Got exec sync container request with timeout 60
conmonrs[1028243]: Creating new IO streams
conmonrs[1028243]: Exec args "--root=/run/runc exec -d --pid-file=/run/containers/storage/overlay-containers/f5a4cb30d7219327b1fb12d111a14137ba65b5ebbfaaa50e4ba980070c4ad7bf/userdata/exec_syncAZSUifMpid 1d9bbf813ae2ed8bd89e64f2160f973e7bfecd1311876c4c8d0f5af1eeb471b7 sh -c netstat -na | grep LISTEN | grep -v 8080 | grep 80"
conmonrs[1028243]: registering event source with poller: token=Token(9), interests=READABLE | WRITABLE
conmonrs[1028243]: registering event source with poller: token=Token(10), interests=READABLE | WRITABLE
conmonrs[1028243]: registering event source with poller: token=Token(11), interests=READABLE | WRITABLE
conmonrs[1028243]: Start reading from IO streams
conmonrs[1028243]: Running task
conmonrs[1028243]: Waiting for exit code
conmonrs[1028243]: Using cgroup path: /proc/1028958/cgroup
conmonrs[1028243]: Setup cgroup v2 handling
conmonrs[1028243]: registering event source with poller: token=Token(0), interests=READABLE
conmonrs[1028243]: Read 81 bytes
conmonrs[1028243]: Exited 0
conmonrs[1028243]: TOKEN: CancellationToken { is_cancelled: false }, PID: 1028958
conmonrs[1028243]: Sending done because token cancelled
conmonrs[1028243]: Sending done because token cancelled
conmonrs[1028243]: deregistering event source from poller
conmonrs[1028243]: Loop cancelled
conmonrs[1028243]: deregistering event source from poller
conmonrs[1028243]: Exiting because token cancelled
conmonrs[1028243]: deregistering event source from poller
conmonrs[1028243]: Done watching for ooms
conmonrs[1028243]: Write to exit paths:
conmonrs[1028243]: Sending exit struct to channel: ExitChannelData { exit_code: 0, oomed: false, timed_out: false }
conmonrs[1028243]: Task done
conmonrs[1028243]: deregistering event source from poller
conmonrs[1028243]: registering event source with poller: token=Token(83886082), interests=READABLE | WRITABLE
conmonrs[1028243]: Got exec sync container request with timeout 60
conmonrs[1028243]: Creating new IO streams
conmonrs[1028243]: Exec args "--root=/run/runc exec -d --pid-file=/run/containers/storage/overlay-containers/f5a4cb30d7219327b1fb12d111a14137ba65b5ebbfaaa50e4ba980070c4ad7bf/userdata/exec_synczA8mDmspid 1d9bbf813ae2ed8bd89e64f2160f973e7bfecd1311876c4c8d0f5af1eeb471b7 sh -c netstat -na | grep LISTEN | grep -v 8080 | grep 80"
conmonrs[1028243]: registering event source with poller: token=Token(16777225), interests=READABLE | WRITABLE
conmonrs[1028243]: registering event source with poller: token=Token(16777227), interests=READABLE | WRITABLE
conmonrs[1028243]: registering event source with poller: token=Token(16777226), interests=READABLE | WRITABLE
conmonrs[1028243]: Start reading from IO streams
conmonrs[1028243]: Running task
conmonrs[1028243]: Waiting for exit code
conmonrs[1028243]: Using cgroup path: /proc/1029067/cgroup
conmonrs[1028243]: Setup cgroup v2 handling
conmonrs[1028243]: registering event source with poller: token=Token(0), interests=READABLE
conmonrs[1028243]: Read 81 bytes
conmonrs[1028243]: Exited 0
conmonrs[1028243]: TOKEN: CancellationToken { is_cancelled: false }, PID: 1029067
conmonrs[1028243]: Sending done because token cancelled
conmonrs[1028243]: Loop cancelled
conmonrs[1028243]: Exiting because token cancelled
conmonrs[1028243]: deregistering event source from poller
conmonrs[1028243]: Sending done because token cancelled
conmonrs[1028243]: deregistering event source from poller
conmonrs[1028243]: Done watching for ooms
conmonrs[1028243]: deregistering event source from poller
conmonrs[1028243]: Write to exit paths:
conmonrs[1028243]: Sending exit struct to channel: ExitChannelData { exit_code: 0, oomed: false, timed_out: false }
conmonrs[1028243]: Task done
conmonrs[1028243]: deregistering event source from poller
conmonrs[1028243]: Read 57 bytes
conmonrs[1028243]: Wrote log line of length 72
conmonrs[1028243]: Wrote log line of length 75
conmonrs[1028243]: deregistering event source from poller
conmonrs[1028243]: Stdout read loop failure: send data message: channel closed
conmonrs[1028243]: registering event source with poller: token=Token(16777223), interests=READABLE | WRITABLE
conmonrs[1028243]: Got exec sync container request with timeout 60
conmonrs[1028243]: Creating new IO streams
conmonrs[1028243]: Exec args "--root=/run/runc exec -d --pid-file=/run/containers/storage/overlay-containers/f5a4cb30d7219327b1fb12d111a14137ba65b5ebbfaaa50e4ba980070c4ad7bf/userdata/exec_sync34TJjjHpid 1d9bbf813ae2ed8bd89e64f2160f973e7bfecd1311876c4c8d0f5af1eeb471b7 sh -c netstat -na | grep LISTEN | grep -v 8080 | grep 80"
conmonrs[1028243]: registering event source with poller: token=Token(100663298), interests=READABLE | WRITABLE
conmonrs[1028243]: registering event source with poller: token=Token(33554442), interests=READABLE | WRITABLE
conmonrs[1028243]: registering event source with poller: token=Token(33554441), interests=READABLE | WRITABLE
conmonrs[1028243]: Start reading from IO streams
conmonrs[1028243]: Running task
conmonrs[1028243]: Waiting for exit code
conmonrs[1028243]: Using cgroup path: /proc/1029207/cgroup
conmonrs[1028243]: Setup cgroup v2 handling
conmonrs[1028243]: registering event source with poller: token=Token(0), interests=READABLE
conmonrs[1028243]: Exited 0
conmonrs[1028243]: Read 81 bytes
conmonrs[1028243]: TOKEN: CancellationToken { is_cancelled: false }, PID: 1029207
conmonrs[1028243]: Sending done because token cancelled
conmonrs[1028243]: Sending done because token cancelled
conmonrs[1028243]: deregistering event source from poller
conmonrs[1028243]: Loop cancelled
conmonrs[1028243]: deregistering event source from poller
conmonrs[1028243]: Exiting because token cancelled
conmonrs[1028243]: deregistering event source from poller
conmonrs[1028243]: Done watching for ooms
conmonrs[1028243]: Write to exit paths:
conmonrs[1028243]: Sending exit struct to channel: ExitChannelData { exit_code: 0, oomed: false, timed_out: false }
conmonrs[1028243]: Task done
conmonrs[1028243]: deregistering event source from poller
conmonrs[1028243]: Exited 2
conmonrs[1028243]: TOKEN: CancellationToken { is_cancelled: false }, PID: 1028312
conmonrs[1028243]: Sending done because token cancelled
conmonrs[1028243]: Loop cancelled
conmonrs[1028243]: Exiting because token cancelled
conmonrs[1028243]: deregistering event source from poller
conmonrs[1028243]: Stderr read loop failure: send done message: channel closed
conmonrs[1028243]: deregistering event source from poller
conmonrs[1028243]: Done watching for ooms
conmonrs[1028243]: Write to exit paths: /var/run/crio/exits/a4100c860bee5c960903153702b486bd4ada8c6c6a77374b8edfc1afd55b872d, /var/lib/containers/storage/overlay-containers/a4100c860bee5c960903153702b486bd4ada8c6c6a77374b8edfc1afd55b872d/userdata/exit
conmonrs[1028243]: Creating exit file
conmonrs[1028243]: Creating exit file
conmonrs[1028243]: Writing exit code to file
conmonrs[1028243]: Writing exit code to file
conmonrs[1028243]: Flushing file
conmonrs[1028243]: Flushing file
conmonrs[1028243]: Done writing exit file
conmonrs[1028243]: Done writing exit file
conmonrs[1028243]: Sending exit struct to channel: ExitChannelData { exit_code: 2, oomed: false, timed_out: false }
Interestingly, when running the workload from the yaml then it works:
apiVersion: v1
kind: Pod
metadata:
labels:
name: pfpod
name: pfpod
spec:
containers:
- args:
- netexec
image: registry.k8s.io/e2e-test-images/agnhost:2.40
imagePullPolicy: IfNotPresent
name: readiness
readinessProbe:
exec:
command:
- sh
- -c
- netstat -na | grep LISTEN | grep -v 8080 | grep 80
failureThreshold: 3
initialDelaySeconds: 5
periodSeconds: 1
successThreshold: 1
timeoutSeconds: 60
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-q2nfb
readOnly: true
- args:
- port-forward-tester
env:
- name: BIND_PORT
value: "80"
- name: EXPECTED_CLIENT_DATA
value: abc
- name: CHUNKS
value: "10"
- name: CHUNK_SIZE
value: "10"
- name: CHUNK_INTERVAL
value: "100"
- name: BIND_ADDRESS
value: localhost
image: registry.k8s.io/e2e-test-images/agnhost:2.40
imagePullPolicy: IfNotPresent
name: portforwardtester
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-q2nfb
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
nodeName: 127.0.0.1
preemptionPolicy: PreemptLowerPriority
priority: 0
restartPolicy: Never
schedulerName: default-scheduler
securityContext: {}
serviceAccount: default
serviceAccountName: default
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
volumes:
- name: kube-api-access-q2nfb
projected:
defaultMode: 420
sources:
- serviceAccountToken:
expirationSeconds: 3607
path: token
- configMap:
items:
- key: ca.crt
path: ca.crt
name: kube-root-ca.crt
- downwardAPI:
items:
- fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
path: namespace
> k get pods
NAME READY STATUS RESTARTS AGE
pfpod 2/2 Running 0 24s
Removing the readiness probe in the test makes it pass on my local machine:
https://github.com/kubernetes/kubernetes/blob/02ac8ac4181e179c2f030a9f8b1abef0d9a0b512/test/e2e/kubectl/portforward.go#L77-L87
So it looks like that the the first execsync request kills the portforwardtester
container.
$ conmonrs --version
version: 0.2.0
tag: none
commit: 51eb41deca7402edb0eb8b75b965283790dbf299
build: 2022-09-15 07:32:52 +00:00
target: x86_64-unknown-linux-gnu
rustc 1.63.0 (4b91a6ea7 2022-08-08)
cargo 1.63.0 (fd9c4297c 2022-07-01)
# On Linux:
$ cat /etc/os-release
# paste output here
$ uname -a
# paste output here
to address CVE-2022-1708 we need a way to limit the amount of output conmon-rs processes in an ExecSync request
We do currently use docker on Debian 11 and let the containers log on stdout through the
Splunk logging driver (https://docs.docker.com/config/containers/logging/splunk/) via HEC (https://docs.splunk.com/Documentation/Splunk/latest/Data/UsetheHTTPEventCollector) into Splunk.
In term of docker-compose this gives us the possibility to configure Splunk-logging on a per deployment basis, which is very comfortable as well as independent of any central configuration.
We would like to move to Podman on RHEL8 servers. Everything works fine so far, but we didn't find a way to log stdout of the containers via the HEC interface into Splunk, as the currently available podman version 4 does not provide such a splunk logging driver.
Is there a solution on the part of Podman to log the output of the containers into Splunk.
If not, is it possible to commission a corresponding development against payment?
I opened a feature request in the containers/conmon#340 and I was told to open the request in this repo.
release-static
N/A
Not sure, just noticed this morning
= note: "cc" "-m64" "/home/runner/work/conmon-rs/conmon-rs/target/x86_64-unknown-linux-gnu/release/deps/conmonrs-988be6b2ab5ecfbe.conmonrs-4c05c21a52340583.3ioxwzlr70pbvyba.rcgu.o.rcgu.o" "-Wl,--as-needed" "-L" "/home/runner/work/conmon-rs/conmon-rs/target/x86_64-unknown-linux-gnu/release/deps" "-L" "/home/runner/work/conmon-rs/conmon-rs/target/release/deps" "-L" "src/backend/linux_raw/arch/outline/release" "-L" "/home/runner/work/conmon-rs/conmon-rs/target/x86_64-unknown-linux-gnu/release/build/libgit2-sys-adb848e1e8fe704a/out/build" "-L" "/home/runner/.rustup/toolchains/1.58.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/tmp/rustcOeSN7b/liblibgit2_sys-9c0b940abda9ef93.rlib" "/tmp/rustcOeSN7b/librustix-371ab8aa59bbc384.rlib" "-Wl,--start-group" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-lgcc_eh" "-lgcc" "-Wl,--end-group" "/home/runner/.rustup/toolchains/1.58.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-5667a4a7e2c48d47.rlib" "-Wl,-Bdynamic" "-lc" "-lz" "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-L" "/home/runner/.rustup/toolchains/1.58.1-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/home/runner/work/conmon-rs/conmon-rs/target/x86_64-unknown-linux-gnu/release/deps/conmonrs-988be6b2ab5ecfbe" "-Wl,--gc-sections" "-static" "-no-pie" "-Wl,-zrelro,-znow" "-nodefaultlibs"
= note: /usr/bin/ld: /home/runner/work/conmon-rs/conmon-rs/target/x86_64-unknown-linux-gnu/release/deps/conmonrs-988be6b2ab5ecfbe.conmonrs-4c05c21a52340583.3ioxwzlr70pbvyba.rcgu.o.rcgu.o: in function `<std::sys_common::net::LookupHost as core::convert::TryFrom<(&str,u16)>>::try_from':
/rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b//library/std/src/sys_common/net.rs:191: warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: attempted static link of dynamic object `/lib/x86_64-linux-gnu/libc.so.6'
/usr/bin/ld: attempted static link of dynamic object `/lib64/ld-linux-x86-64.so.2'
/usr/bin/ld: attempted static link of dynamic object `/usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libz.so'
collect2: error: ld returned 1 exit status
ref https://github.com/containers/conmon-rs/actions/runs/3675616095/jobs/6218022389
We are relying on OOM testing within crio. We need to add native OOM testing within this repo.
ref: #365
The issue’s that we disconnect stdout and stderr with non terminal containers in streams.rs. This is causing the stdout logger not work any more since the log crate relies on stdio. We should investigate how to improve that.
CRI-O conmonrs e2e feature tests.
Mentioned in the title, ran via: https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/cri-o_cri-o/6220/pull-ci-cri-o-cri-o-main-ci-e2e-conmonrs/1572188830188965888
Since ever, reproducible with 8e4cec9
No response
No response
#608 is failing due to a possible API change. We will need a manual bump of the package to fix it.
capnproto alpha 5 to work.
N/A
No response
$ conmonrs --version
# paste output here
# On Linux:
$ cat /etc/os-release
# paste output here
$ uname -a
# paste output here
We're already using the tracing crate, which has support for Open Telemetry (OTEL) as well: https://github.com/tokio-rs/tracing/tree/master/tracing-opentelemetry
Using the crate would probably incorporate adding those, too:
Independently how we wanna use it from a configuration perspective, having OTEL support would increase the observability of the runtime stack (together with CRI-O) by gaining much more insights about what is going on in conmon-rs.
the current implementation of conmon listens to the container's cgroup.events
file to check whether a container OOMed after it died. We should do a similar thing in conmonrs
I received some tips for packaging conmon-rs RPMs from @cgwalters. Generally bootupd looks like a good example. The bootupd github releases contain a source tar and a vendor tar. The bootupd.spec uses both tarballs to generate the resulting rpm.
Starting points could be:
None
None
No response
NONE
$ conmonrs --version
# paste output here
# On Linux:
$ cat /etc/os-release
# paste output here
$ uname -a
# paste output here
We can use the Open Telemetry context propagation to link spans between CRI-O and conmon-rs. This would mean that we have to use the global Text Map Propagator on the client side as well as on the conmon-rs server side to inject/extract the spans.
Since capnproto does neither natively support serializing hash maps nor carrying headers, we have to change each RPC to additionally contain the metadata:
struct VersionRequest {
metadata @0 :Data;
verbose @1 :Bool;
}
I assume that we introduce a new metadata type like:
struct MetadataMap<'a>(HashMap<&'a str, &'a str>);
impl<'a> Extractor for MetadataMap<'a> {
fn get(&self, key: &str) -> Option<&str> {
self.0.get(key).map(|x| x.clone())
}
/// Collect all the keys from the MetadataMap.
fn keys(&self) -> Vec<&str> {
self.0.keys().map(|k| k.clone()).collect::<Vec<_>>()
}
}
And then (for each RPC) restore the spans:
let metadata = pry!(req.get_metadata());
let hashmap: HashMap<&str, &str> = pry_err!(serde_json::from_slice(metadata));
let metadata_map = MetadataMap(hashmap);
let parent_cx = global::get_text_map_propagator(|prop| prop.extract(&metadata_map));
self.tracer().start_with_context("version", &parent_cx);
Examples:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.