fly-log-local's Introduction

fly-log-local

Local logging at Fly using Vector and Loki.

This app reads the Fly NATS endpoint using Vector which then sends to Loki for the datastore on the attached volume. The logs can later be accessed via Logcli or any other Loki client. In this example the Loki endpoint is running without authentication and only accessible via fly proxy.

Quickstart

Create Fly app

fly apps create <insert-app-name>

Edit fly.toml

Update the app name.

app = "<insert-app-name>"

Create volume

fly volumes create log_data

Create secret

fly secrets set ACCESS_TOKEN=$(fly auth token)

Scale memory ( optional )

fly scale memory 512

Deploy

fly deploy

Install logcli

brew install logcli

Start proxy

fly proxy 3100:3100

Search loki

Tailing all app logs.

~/ logcli query -z UTC -q '{forwarder="vector"}' -t
2022-08-18T14:33:35Z {fly_app="fly-log-local-example", fly_instance="c1046e0d", forwarder="vector"} 2022-08-18 14:33:35,571 INFO success: loki entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2022-08-18T14:33:35Z {fly_app="fly-log-local-example", fly_instance="c1046e0d", forwarder="vector"} 2022-08-18 14:33:35,572 INFO success: vector entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2022-08-18T14:40:51Z {fly_app="bold-star-5487", fly_instance="97200383", forwarder="vector"} Starting instance
2022-08-18T14:40:58Z {fly_app="bold-star-5487", fly_instance="97200383", forwarder="vector"} Configuring virtual machine
2022-08-18T14:40:58Z {fly_app="bold-star-5487", fly_instance="97200383", forwarder="vector"} Pulling container image
2022-08-18T14:40:58Z {fly_app="bold-star-5487", fly_instance="97200383", forwarder="vector"} Unpacking image
2022-08-18T14:40:58Z {fly_app="bold-star-5487", fly_instance="97200383", forwarder="vector"} Preparing kernel init
2022-08-18T14:40:59Z {fly_app="bold-star-5487", fly_instance="97200383", forwarder="vector"} Configuring firecracker
2022-08-18T14:40:59Z {fly_app="bold-star-5487", fly_instance="97200383", forwarder="vector"} Starting virtual machine
2022-08-18T14:40:59Z {fly_app="bold-star-5487", fly_instance="97200383", forwarder="vector"} Starting init (commit: 9b0a951)...
...

Specific app and from time.

~/ logcli query -z UTC -q --from="2022-08-18T14:30:00Z" '{fly_app="fly-log-local-example"}'
2022-08-18T14:33:35Z {} 2022-08-18 14:33:35,572 INFO success: vector entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2022-08-18T14:33:35Z {} 2022-08-18 14:33:35,571 INFO success: loki entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

Additional documentation can be found here: https://grafana.com/docs/loki/latest/tools/logcli/

fly-log-local's People

Contributors

Stargazers

Watchers

fly-log-local's Issues

Deploy error: failed to fetch an image or build from source

I get the following error when deploying the app with fly deploy:

Error: failed to fetch an image or build from source: error building: executor failed running [/bin/sh -c git clone https://github.com/grafana/loki.git     && cd loki     && make loki logcli]: exit code: 2

While building, the process stops with the following:

imports github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/container: build constraints exclude all Go files in /go/loki/vendor/github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/container
#10 17.25 vendor/github.com/grafana/dskit/netutil/netutil.go:6:2: cannot find package "." in:
#10 17.25 	/go/loki/vendor/net/netip
#10 17.26 make: *** [Makefile:158: cmd/loki/loki] Error 1

To be deployed as standalone app or integrated with "main app"?

Please bare with my very basic question, is this supposed to be a different standalone app from the app I am trying to save logs for, or is this to be integrated/deployed into the same main app?

Recommend Projects