Does cri-dockerd support journald logging ?? about cri-dockerd HOT 6 CLOSED

used the default cri-docker.service file shipped in the RPM

from cri-dockerd.

If docker is configured with journald logging, it completely bypasses the normal CRI mechanisms.

The best practice way of managing this is to use something like grafana-agent or vector or your log aggregator of choice, and have multiple sinks. Either forwarding to Graylog, Loki, Logstash, or a process which reads from the system and forwards to journald.

In particular, logging to journald localizes logs for any given container/pod to the system they're running on, which is orthogonal to k8s logging (accessible from anywhere). The only reasonable mechanism to have this work would be to have kube-api able to poll from journald on every single worker node, which is not practical for a lot of reasons.

I'm happy to help you find an alternative logging structure which can give something like the best of both worlds.

from cri-dockerd.

Hm, kubernetes’ original Dockershim worked with journald logging though—at least as of 1.23.x.

Did they deprecate more than just the CRI-fication of Docker in 1.24?

from cri-dockerd.

cri-docker runs as a systemd unit, which logs to journald/journalctl (from the daemon itself). The answer to log collection is exactly the same as the comment from Jun 7.

from cri-dockerd.

Sorry, I didn’t mean cri-dockerd’s logs, I mean the Pod/Container logs.

I meant that kubelet <=1.23 with a Docker Runtime with the Logger Driver set to journald was able to somehow provide container logs upon kubectl logs ….

I absolutely understand from your earlier comment that containers logging to journald isn’t conforming to CRI and that there’s several ways to setup logging the CRI way. Docker itself isn’t CRI conformant either, and so shouldn't be used as the underlying Container Runtime, and yet here we are and you guys made an adapter (which I'm really glad exists :)).

So I was just wondering what part about the now-abandoned Dockershim or Docker-compatibility in the kubernetes/kubelet Code was responsible for kubelet logs's magic/un-CRI compatibility to a Docker Runtime using journald as the Logging Engine.
An abandoned part that's beyond cri-dockerd's scope or purpose (because I thought this project wants to provide a full third-party replacement for the Dockershim)? Or am I completely misunderstanding how kubectl logs used to work with Docker writing container logs to journal?

from cri-dockerd.

The now-abandoned Dockershim is this codebase. Upstream k8s was nice enough to extract it and hand it over, and it saw some refactoring to make it more sense as a standalone codebase, but nothing else was removed. I haven't walked back whatever else happened in kubectl/etc, but this is dockershim in 99% of ways.

AFAIK (and judging from the message), kubectl logs isn't doing anything particularly intelligent there, and I wouldn't expect it to, honestly. It's trying to read a symlink (which, after rotation, would probably be /var/lib/docker/containers/xxxxxxx/....) which doesn't point anywhere after rotation (there is likely to be [012345...].log). I would guess, without explicitly testing, that configuring dual logging in docker/daemon.json to log to both json-file (the default) and journald would give the desired behavior.

from cri-dockerd.