sentinel's Introduction

Sentinel

A WebSocket-compatible kill & fleet tracker for EVE Online, built on Sails.js. Part of the magic behind the GLOSS project.

Features

Backed by PostgreSQL
Live reporting from zKillboard's WebSocket service
Identifies fleet patterns based on recorded kills
Resolves characters, corporations and alliances via ESI
Resolves shiptypes and systems via the EVE Online SDE
Sophisticated job queue scheduling via Kue

Environment

Install Node >= 8
Install Yarn
Install and configure a database (PostgreSQL preferred)
Install and configure Redis

The app will refuse to run without a valid root-level .env - see the example file.

Boot

$ cd sentinel
$ yarn
$ node app.js

And now you're listening for kills and saving them to the DB in real time.

Provided it's not firewalled, a frontend to the job queue will be available at http://<BASE_URL>:6574.

If you're going to leave this thing running permanently, you should run it with NODE_ENV=production (i.e. npm start).

Support

Got a problem? File a GitHub issue. Please be nice.

Want to support the developer? ISK donations welcome.

sentinel's People

Contributors

Stargazers

Watchers

sentinel's Issues

Investigate the use of zKillboard WSS

https://github.com/zKillboard/zKillboard/wiki/Websocket

Solo kills by structures should not create fleet instances

These are not fleets, so the active fleet records are slowly leaking upward:
https://gloss.space/api/sentinel/fleets/5adb71856793287e2eb82488
https://gloss.space/api/sentinel/fleets/5adb6ff86793287e2eb82363

Furthermore, it appears that it's not being considered in the health check job: https://github.com/dougestey/sentinel/blob/master/config/jobs.js#L39

Job clustering over multiple cores

Memory leak

Slow leak, numbers to consider (keeping in mind that Arbiter is also leaking/operating on the same server instance):

Total RAM: 3.75 GB

38% total memory consumption after running for 6h
Rebooting the app brings us back down to 22-23% memory consumption

Usually Sentinel will eat up all of the available RAM, regardless if Arbiter is running, after a few days or so.

Character records should be updated on a reasonable interval

Currently the caching on a character lookup is infinite. We should bust the cache after a period of time so that corp changes, etc are reflected accurately.

This is an interesting problem, and somewhat unique to Sentinel because of the goal it's trying to accomplish: tracking a fleet's movement. Killboards aren't subjected to this problem since their concept of a "fleet" is limited by the system the ships died in.

Background
Imagine a gatecamp that a battleship falls victim to. Everyone gets on the kill. But his pod is subsequently destroyed by a single fastlocker in the fleet. Now both of these kills get fed to Sentinel, one after another. What happens?

A fleet record is created based on the battleship kill.
When the capsule kill record is created, Sentinel recognizes that the attacker is in an existing fleet already because she was on the battleship kill.
Instead of creating a new fleet record, Sentinel relates the capsule kill to the existing fleet record.

So now we have one fleet with two kills, which makes perfect sense.

Alternate situation for you. What if, just before the battleship, that same fastlocker solo'd a shuttle - he missed the battleship kill because he had to ping off, but then arrived in time for the pod, which the fleet held down for him. Here's the sequence:

Fleet A is created with a solo pilot when the shuttle dies.
Fleet B is created based on the battleship kill. Pilot from fleet A is not in this fleet (yet.)
When the battleship's pod dies, Sentinel detects two possible matches for the fleet, and merges those two before into Fleet C before attributing the kill to Fleet C.

These are the types of odd scenarios Sentinel is designed to handle. And it works well. Except when it doesn't.

Scenario

NPSI fleet occurs in nullsec, they net a bunch of kills and Sentinel tracks them properly.
Fleet stands down but a quarter of the pilots remain active doing other things in the game, including getting kills in highsec ganks, nullsec fleets, small-gang pew, etc.

The Problem
The pilots that get kills within the fleet expiry window (30 minutes on prod) are keeping the whole fleet active on Sentinel, which can cause incorrect fleet reports (not to mention huge fleet sizes that are difficult to perform logic on in a performant manner.)

The identifier service doesn't discriminate based on corporation or alliance when identifying a fleet, and rightly so. It also has reasonably strict requirements around whether a fleet is a correct match for an incoming kill (for example, it scores possible fleet records, sorts by those scores, and then picks the best candidate if a high enough percentage of the fleet members are on the new kill.) What I suspect is happening is that it's correctly identifying new fleets in this situation - but then on the next kill analysis, it's seeing that the fleets have similar characters and is merging them back into one.

The Goal
Identifier is going to need more specific logic that splits these pilots out in to new, smaller fleets, and iterating through previous kills is likely going to be the way to deduce whether they're all still rolling together or not.