Comments (13)
a) does not accept "---" at the top of a YAML document - but docker compose does; presence of --- yields
I've pushed a change that tries to address this.
c) compose support is not cumulative
Hmm, I'm not able to reproduce this, but I'm not sure I'm doing the right thing either. Is the docker://compose/docker-compose.addon.grafana.yaml
just a regular compose file? I can pass two different compose YAML config files and all of the service logs are loaded up.
Thanks for the detailed response!
from lnav.
Thanks for trying it out and giving a heads up! I had made a change after the docker/url work that caused the problem. I've pushed a quick fix to get it going again. Need to think a bit to figure out how to improve the behavior a bit more.
from lnav.
I get the same behaviour when running
lnav -e 'docker logs -f lnav-target'
❯ docker --version
Docker version 24.0.5, build ced0996
from lnav.
The following strategy enables "following" docker logs, but is less convenient ... and seems to be very prone to lnav opening / closing files (the FIFO pipe)?
mkfifo lnav-target
docker logs -f lnav-target >> lnav-target
lnav lnav-target
from lnav.
Many thanks for the fantastically fast fix!
Current HEAD works very well for
docker run -it --rm --name lnav-target-1 bash bash -c 'while true; do sleep 1; echo "$(date -Ins) INFO c1"; done'
docker run -it --rm --name lnav-target-2 bash bash -c 'while true; do sleep 1; echo "$(date -Ins) INFO c2"; done'
lnav docker://lnav-target-1 docker://lnav-target-2
I can now look into doing fancier things: I like setting up local notebook infrastructure with docker compose
(podman-compose
) stacks, without building up a resource-heavy ELK stack. lnav
would probably simplify interacting with this quite a bit.
from lnav.
@shoffmeister Should there be a docker-compose://
URL that loads the logs for the services in the local docker-compose.yml
file?
from lnav.
@tstack while docker-compose://
sounds like a logical extension, I am unclear about what "local docker-compose.yml
file" would mean:
My experiments tend to involve some however arbitrarily named baseline docker-compose.yaml
often enriched with addons, launch immediately or later on: docker compose -f d1.yml -f d2.yml -f d3.yml
One directory right now has
❯ ls *.y*
docker-compose.addon.grafana.yaml docker-compose.addon.splunk.yaml
docker-compose.addon.opentelementry.yaml docker-compose.yml
and that's more the rule than the exception.
I just successfully tried
containers=($(docker container ls --all --format '{{.Names}}')); ./lnav "${containers[@]/#/docker://}"
containers=($(docker ps --format '{{.Names}}')); ./lnav "${containers[@]/#/docker://}"
While this is not what docker-compose://
would do, practically it may be very close? This makes me suspect that documenting approaches cookbook-style to conveniently provisioning a list of log source targets could work well?
Not implementing docker-compose://
would also allow leaving the identity crisis of podman-compose
unaddressed ;)
And, after all that, someone will come up with a request to "hey, lnav
all Deployment
s over all namespaces of that k3s
cluster". I wouldn't open the door for that.
version: "2"
services:
s1:
image: bash
command: bash -c 'while true; do sleep 1; echo "$(date -Ins) INFO s1"; done'
s2:
image: bash
command: bash -c 'while true; do sleep 1; echo "$(date -Ins) INFO s2"; done'
s3:
image: bash
command: bash -c 'for i in {1..5}; do sleep 1; echo "$(date -Ins) WARN s3"; done; exit 99'
from lnav.
I've pushed a change that adds some initial support for docker compose. If you use compose
as the hostname with the path to the configuration file, it will launch docker compose logs
for each service specified in the config file. For example, if there is a compose.yaml
file in the current directory, you can do:
lnav docker://compose/compose.yaml
That will tail all of the service logs separately, so the LOG (or TEXT) view will show the messages associated with a particular container.
Note that the docker compose logs
processes don't seem to exit when lnav exits. I think this is a docker issue.
I don't really use docker much, so I'm not sure how useful this is. Let me know if you have time to try it out and if there are gaps.
Thanks
from lnav.
@tstack Interesting! I gave the "compose" support a whirl with my local stacks and noticed a few things:
a) does not accept "---" at the top of a YAML document - but docker compose does; presence of ---
yields
✘ error: failed to parse YAML content │
reason: check failed: !m_tree->is_stream(id)
Generally, IIRC, docker supports multi-document YAML files, i.e. more than one YAML document in a single file. ---
is the separator between those documents. See https://yaml.org/spec/1.2.2/ - "2.2. Structures"
b) "compose" implementation is too general; in 38e5fa5#diff-8b93f0d06f87b27b049e1357526684516a8ee4382042b4051916ae155f132918R18 very generic support for the "compose" feature is added. This implementation assumes that on a user's system the following works
docker compose -f ...
podman compose -f ...
Alas ...
-
docker compose
is specific to modern(?) versions of docker when they enabled compose functionality through a docker plugin; "Back In The Old Days" this used to bedocker-compose
as a dedicated Python script. And both co-exists, as all they do is "docker". Depending on how far you want to look backwards (LTS distros?),docker-compose
is the way to go -
podman compose
does not exist - there is no compose plugin for podman. There ispodman-compose
which is supposedly CLI-compatible with - you guessed it! -docker-compose
docker-compose
and podman-compose
are separate installations, so docker being available does not guarantee the presence of the -composes.
All this applies, IIRC, to both Linux and Windows.
It's a mess.
Having tried compose support in lnav, I recognize this as quite user-friendly. I am torn. 🤷♂️
c) compose support is not cumulative, does not aggregate, i.e. lnav docker://compose/docker-compose.yml docker://compose/docker-compose.addon.grafana.yaml
only shows the content of the second stack. This is in contrast to lnav docker://zookeeper docker://grafana
which works as expected. docker compose -f ...
for the respective compose files does the expected thing, so it appears to be an issue in the aggregation of the composes in lnav.
Repro yaml for item a)
---
version: "2"
services:
s1:
image: bash
command: bash -c 'while true; do sleep 1; echo "$(date -Ins) INFO s1"; done'
s2:
image: bash
command: bash -c 'while true; do sleep 1; echo "$(date -Ins) INFO s2"; done'
s3:
image: bash
command: bash -c 'for i in {1..5}; do sleep 1; echo "$(date -Ins) WARN s3"; done; exit 99'
from lnav.
a) does not accept "---" at the top of a YAML document - but docker compose does (…)
You mean the support added only 4 days ago in version v2.21.0 (Support for multi-document YAML files (compose-spec/compose-go#451))
from lnav.
Hmm, I have habitually putting ---
at the top of docker-compose.yaml
for many years- and it has been working well?
Now, true multi-document support - yeah, my memory is ... blurred? ... from all those stacks. Historically, I have been assembling these through multiple --file
specs, or through some addon pattern which tack onto existing volumes / networks. All that is for quick and resource-friendly local LEGO fun with containers.
from lnav.
@tstack Sorry for the long delay in getting back on the "compose not cumulative" issue that I reported above!
It would seem as if there is a different problem: lnav simply errors out on the first docker compose file with
✘ error: unknown command - <NULL>
And it is really the content of the compose file which is responsible: debug.zip
To reproduce, simply run the latest CI build with lnav docker://compose/debug.yaml
Note that the file is already trimmed down; the original size is 6 KB.
from lnav.
FWIW, I am on a "modern" Docker installation; because of that, I do not have docker-compose
but only docker compose
.
To get lnav
to work with this setup, having a simple script like the one below on the PATH
is sufficient:
f="docker-compose"; touch $f; chmod u+x $f; cat > $f <<'EOF'
#!/usr/bin/env bash
docker compose "${@}"
EOF
from lnav.
Related Issues (20)
- Prompt existing overwrites when using `lnav -i <file>` HOT 1
- Latest version crashes on Arch Linux HOT 22
- $schema attribute in format file not recognized by provided jsonschema HOT 2
- Temporary folder not recreated if gone for some reason HOT 1
- When printing timestamp the last 3 microsecond digits are always "0". HOT 2
- request a release for the static init issue fix HOT 3
- Wrap text along word boundaries with the command :enable-word-wrap
- Add the ability to add multiple :filter-expr HOT 2
- Add to HIST the ability to display the number of matches based on arbitrary patterns HOT 1
- Add support for 7z archives HOT 3
- Can't open files in /tmp HOT 2
- Hangs/freezes and uses 100% CPU with small log files. HOT 1
- An option to exclude files and directories implied by `-r` (recurse) option HOT 14
- Pretty print regression for json logs HOT 4
- stdin and adb logcat -v time issues HOT 3
- 0.12.2 "top" mode navigation is broken HOT 1
- Custom theme location does not seem to work? HOT 4
- Enable native Caddy JSON log file format HOT 1
- Can it be built without Rust, disabling related module? HOT 2
- Cursor mode selection does not select expected lines HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lnav.