Giter VIP home page Giter VIP logo

logbot's Introduction

LogBot

IRC logger and publisher.

https://logbot.info/

About

LogBot is comprised of three main parts:

  • logbot-irc: IRC bot that logs events to a queue
  • logbot-consumer: Processes and stores queued events
  • logbot-web: Mojolicous application for viewing logs online

Maintainer

License

  • MIT

logbot's People

Contributors

disasm avatar globau avatar marked avatar simonlindholm avatar tantek avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

logbot's Issues

Add logs for #focus

#focus is the official channel for Firefox Focus development - it'd be great if we could get logs too!

show the channel's current topic in the web ui

capture the channel's current topic and show it in the web ui.

  • don't show in channel logs
  • create a new table that tracks the current topic
  • show in webui; probably as tooltips

  • listen for RPL_TOPIC and RPL_NOTOPIC messages
  • add new type to jobqueue
  • once a day request topics for channels-in
  • in logbot-nightly tidy up the topics table old channels

Can no longer view a date range

Previously (when the logs were at http://logs.glob.uno), you could view logs for a channel for a range of dates, I think up to 28 days or so.

Now that the logs are at https://mozilla.logbot.info/, that ability seems to be gone - you can only view one day at a time.

I found viewing multiple days at a time to be very useful. Please consider bringing that feature back.

add "invite"

you should be able to invite logbot to a channel. depends on issue 1 being fixed first.

logbot-nightly could overwrite config changes made while it's running

logbot-nightly loads a config when it starts, and if it has to archive a channel it saves. if a channel is joined while logbot-nightly is running, that join will be reverted.

probably need a scope-based lock around the config file, something like

$config = config_update($config_file, sub {
    shift->{channels}->{$channel}->{archived} = 1;
});

this would flock and load the $config_file, passing it into the callback, save the config, release the flock and return the updated $config object.

Can't call method "log_event" on an undefined value at /home/logbot/logbot/lib/LogBot/Bot.pm line 177

Can't call method "log_event" on an undefined value at /home/logbot/logbot/lib/LogBot/Bot.pm line 177.
at /home/logbot/logbot/lib/LogBot/Daemon.pm line 22
LogBot::Daemon::ANON('Can't call method "log_event" on an undefined value at /home...') called at /home/logbot/logbot/lib/LogBot/Bot.pm line 177
LogBot::Bot::_log_event('LogBot::Bot=HASH(0x2a648b0)', 'LogBot::Event=HASH(0x37df118)') called at /home/logbot/logbot/lib/LogBot/Bot.pm line 168
LogBot::Bot::action('LogBot::Bot=HASH(0x2a648b0)', 'logbot', 'greatwarrior', 'kicks logbot') called at /home/logbot/logbot/lib/LogBot/IRC.pm line 153
LogBot::IRC::irc_ctcp_action('LogBot::IRC', 'POE::Session=ARRAY(0x2a6f578)', 'POE::Kernel=ARRAY(0x20cafd8)', 'HASH(0x2a6b3e0)', 'irc_ctcp_action', 'POE::Session=ARRAY(0x2a5ce48)', undef, '/usr/share/perl5/vendor_perl/POE/Component/IRC.pm', 316, ...) called at /usr/share/perl5/vendor_perl/POE/Session.pm line 482
POE::Session::_invoke_state('POE::Session=ARRAY(0x2a6f578)', 'POE::Session=ARRAY(0x2a5ce48)', 'irc_ctcp_action', 'ARRAY(0x37db5f8)', '/usr/share/perl5/vendor_perl/POE/Component/IRC.pm', 316, 'POE::Wheel::ReadWrite(355) -> select read') called at /usr/share/perl5/vendor_perl/POE/Kernel.pm line 1100
eval {...} called at /usr/share/perl5/vendor_perl/POE/Kernel.pm line 1099
POE::Kernel::_dispatch_event('POE::Kernel=ARRAY(0x20cafd8)', 'POE::Session=ARRAY(0x2a6f578)', 'POE::Session=ARRAY(0x2a5ce48)', 'irc_ctcp_action', 1, 'ARRAY(0x37db5f8)', '/usr/share/perl5/vendor_perl/POE/Component/IRC.pm', 316, 'POE::Wheel::ReadWrite(355) -> select read', ...) called at /usr/share/perl5/vendor_perl/POE/Resource/Events.pm line 266
POE::Kernel::_data_ev_dispatch_due('POE::Kernel=ARRAY(0x20cafd8)') called at /usr/share/perl5/vendor_perl/POE/Loop/Select.pm line 315
POE::Kernel::loop_do_timeslice('POE::Kernel=ARRAY(0x20cafd8)') called at /usr/share/perl5/vendor_perl/POE/Loop/Select.pm line 323
POE::Kernel::loop_run('POE::Kernel=ARRAY(0x20cafd8)') called at /usr/share/perl5/vendor_perl/POE/Kernel.pm line 1276
POE::Kernel::run('POE::Kernel=ARRAY(0x20cafd8)') called at /home/logbot/logbot/lib/LogBot/IRC.pm line 22
LogBot::IRC::start('LogBot::IRC') called at /home/logbot/logbot/lib/LogBot/Daemon.pm line 85
LogBot::Daemon::gd_run('LogBot::Daemon=HASH(0xb385a8)') called at /usr/share/perl5/vendor_perl/Daemon/Generic.pm line 190
Daemon::Generic::new('LogBot::Daemon', 'configfile', 'logbot.conf') called at /usr/share/perl5/vendor_perl/Daemon/Generic.pm line 38
Daemon::Generic::newdaemon('configfile', 'logbot.conf') called at /home/logbot/logbot/lib/LogBot/Daemon.pm line 23
LogBot::Daemon::start('LogBot::Daemon') called at ./logbot line 14
at /home/logbot/logbot/lib/LogBot/Daemon.pm line 22
LogBot::Daemon::ANON('Can't call method "log_event" on an undefined value at /home...') called at /usr/share/perl5/vendor_perl/POE/Kernel.pm line 1297
POE::Kernel::_rethrow_kr_exception('POE::Kernel=ARRAY(0x20cafd8)') called at /usr/share/perl5/vendor_perl/POE/Kernel.pm line 1285
POE::Kernel::run('POE::Kernel=ARRAY(0x20cafd8)') called at /home/logbot/logbot/lib/LogBot/IRC.pm line 22
LogBot::IRC::start('LogBot::IRC') called at /home/logbot/logbot/lib/LogBot/Daemon.pm line 85
LogBot::Daemon::gd_run('LogBot::Daemon=HASH(0xb385a8)') called at /usr/share/perl5/vendor_perl/Daemon/Generic.pm line 190
Daemon::Generic::new('LogBot::Daemon', 'configfile', 'logbot.conf') called at /usr/share/perl5/vendor_perl/Daemon/Generic.pm line 38
Daemon::Generic::newdaemon('configfile', 'logbot.conf') called at /home/logbot/logbot/lib/LogBot/Daemon.pm line 23
LogBot::Daemon::start('LogBot::Daemon') called at ./logbot line 14

create network-wide metrics

network-wide metrics could be useful, especially to server operators.
plan here is provide the most of same info that channels have, across all channels.

  • number of active posters (at least one event within the last 7 days)
  • number of events per day
    • by channels (top 10)
    • across all channels
  • network-wide activity plot
  • top 10 users by event count

Add logs for #telemetry

#telemetry is the official channel for telemetry discussions - it'd be great if there were logs. Note that I'm just a consumer of telemetry and not an owner so it may be pertinent to check with that team first if that's something they'd want (e.g. it'd be undesirable for someone to accidentally drop private data in channels and have it stored forever in logs).

explicitly check channel invites are from ops

currently logbot relies on the network's config to prevent non-ops from inviting users. as this may change, logbot should verify that the invite originated from a channel op before logging.

roughly:

  1. upon receiving an invite, add to a pending_invites $state hash, and join the channel
  2. in the joined handler, check if we're joining a pending_invites channel, and...
  3. send a NAMES command
  4. collect ops in RPL_NAMREPLY
  5. on RPL_ENDOFNAMES crosscheck pending_invites with the collected list of channel ops
  6. if invited by an op, log and update the config
  7. if not invited by an op, log, and leave the channel
  8. privmsg the requester with the result: either a url or an error message
  • don't push messages from pending_invites channels for logging

add help to commands

add a help command, and a base method to commands for a one line syntax blurb

log real ip in mojo access logs

because hypnotoad is behind a reverse proxy, the remote ip in the access logs is always localhost. probably need to use x-forwarded-for

HTTPS for logs.glob.uno

Not sure if this is the right place to track this, but it would be nice to have HTTPS and even HSTS preload for the logs.

upgrade digital ocean instance

currently running on 512mb DO instance.

logbot's memory consumption is right on the limit of this, and triggered a few OOM alerts. i've enabled swap as a stop gap, but really need to bump up to the next instance size (1 gb ram) and disable swap.

Add "load previous/next day" links to log viewer

A use case of mine...

Each morning, for each of several IRC rooms:

  1. Visit the log page for the room, eg http://logs.glob.uno/?c=treeherder
  2. Try to read any messages I missed overnight, but the log doesn't go back far enough.
  3. So, click the "..." for the "from date" selector.
  4. Pick the day prior to today (sometimes mis-clicking, meaning I get completely the wrong logs).
  5. Click update
  6. Wait for page to fully reload
  7. Scroll down to where one day turned into the next (ie where the top of the log was at step 2) - because the vast majority of the newly loaded log is redundant (other than the last few hours), since it was during my working day and so I was connected to IRC at that time.
  8. Repeat for {#developers, #ateam, ...}

Problems:

  • Multiple clicks & easy to click the wrong date
  • Tedious to scroll down to the log part that was previously being viewed
  • Page has to reload

Proposal:
At the start and end of the viewed log, add "Load previous day" and "Load next day", which would:

  • Adjust the from/to date fields appropriately and update the page
  • Ideally: Scroll down to the end of the new day / where the previously viewed content was.
  • Optionally: Load the additional log segments via AJAX and not need the page reload.

create mojo access logs

rework logging..

  • ~/logbot/logs/irc-$network.log โ†’ /var/log/logbot/irc_$network.log
  • use Mojolicious::Plugin::AccessLog to log mojo access to /var/log/logbot/access_$network.log
  • configure log rotation for all logs

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.