Giter VIP home page Giter VIP logo

basiccombatlogparser's Introduction

BasicCombatlogParser

A project with the aim to emulate a subset of the features that https://warcraftlogs.com provides, mainly:

  • parsing dps/hps from combat logs
  • browse through your previous encounters
  • generate ability damage breakdowns (damage done/taken)
  • uptime on buffs

I.e. all the smaller-scale features that are just pushing some numbers around and showing them in the GUI.

Fancier things like graphs are not planned as of now, but would be cool tho...

Shoutout to the fandom wowpedia for providing an abundance of info about the various IDs and events in combat logs.

An example of what it looks like during development right now (for my character Crowfield-Silvermoon):

Character Overview: Encounter breakdown (damage done):

Movement map export (taken from a mythic Terros kill):

Design in Code n stuff

Using SQLite because it seemed like the simplest available option, using it through EFCore because its simply awesome and saves me from a lot of headaches from interacting with SQLite manually. Thanks to whoever came up with ORM.

Metadata

Metadata is data that is generated and stored by the parser and is currently divided into 4 types:

  • Combatlog Metadata
    • Store info about the entire combatlog, such as start time and the file name.
  • Encounter Metadata
    • Stores which combatlog it belongs to, which wow encounter id it has, whether it was successful, and a bit more.
  • Performance Metadata
    • Stores dps/hps results for players on encounters, pretty straight forward
  • Player Metadata
    • Stores a players name, realm, class, and guid.

Parsing

Parsing is done in multiple steps:

  • Create combatlog metadata if the file has not been imported yet.
  • Fetch encounter metadata from the file (basically check for ENCOUNTER_START and ENCOUNTER_END events)
  • Read each individual encounter into memory and generate hps/dps metrics for the players involved, also create player metadata for all not yet stored in the database.

Generated Code

I think source generators are pretty neat, currently trialing it for generating EncounterIds, InstanceIds, and everything directly tied to them. The info that the generator requires is a plaintext list of instances and their encounters.

In an ideal world it would be able to grab that info via the WoW Game API, but dealing with OAuth and all that stuff is something i am not going to bother with.

Trying it

This repo now comes with a sample Combatlog file that is about 30MB large. It contains two encounters from normal Aberrus in the War Within Prepatch. The first fight is Magmorax, a simple single target fight, the second is Echo of Neltharion, which contains a number of adds in the second and third phase. This log has been generated on a german WoW client, which means that all spell and creature names are german aswell. The file has been downsized from 80 to around 29MB and is provided in a .zip compressed format. To use it simply unzip it and import it via the "Import" button in the app. From there you can see the encounters within via the DB View directly, or you navigate to it via searching for players (e.g. Bänger).

Other planned features

See issues here on GitHub. Most if not all features that are in consideration will be mentioned in an issue.

Who made this and why

I (Toreole) wrote (almost) all this code myself, just uhh because it's a lot of fun, somehow.

basiccombatlogparser's People

Contributors

toreole avatar smellilac avatar

Watchers

 avatar

Forkers

smellilac

basiccombatlogparser's Issues

Filter out positions that are "way out there" when exporting encounter movement as image

The issue at hand is at most a handful of events every now and then having positions that are way outside the boundary of where the fight generally is. this leads to a zoom out so far that you cant see anything anymore.

Should be reasonably simple to figure out when doing some kind of median position / standard deviation thing. Figure something out.

Add pagination to all DB View tabs

All of the lists in the DBView tab should have buttons for pagination so you can go through all of the data that is stored in the DB.

Look into DataGrids

The AutoGenerateColumn event could be used to cancel columns before they are added to the data grid, and user-defined columns could be injected (?is that the right term?).
When combining this with custom attributes like for example:

[ColumnHeader("Name"), TextColumn(Align: "Left")]
public string Name { get; }
[ColumnHeader("DPS"), BarColumn]
public Custom.TotalMetric DPS { get; }

There is an option to generate columns based on the listed attributes, instead of having to add them to the .xaml file manually.

The columns need to be created via code, set up with everything bla bla bla, and in the end its probably not going to be all that great.
But it might be worth thinking some more about at a later time.

Add Death List to Encounter View

Prerequisit: ability to parse UNIT_DIED / PLAYER_DIED / whatever the actual event is in the Combatlog.

Deaths should be displayed by WHO died, WHEN they died, and WHAT killed them (name of the Ability, "Environment"/"Falling" if not applicable.)

Do pagination properly (DBView)

Should use the "index" of the entities, not the Id, as that is not future-safe (deleting things should be possible at some point)

Log archive

you should have an option to archive logs. archive status should be included in the LogMetadata entry for it.
Archiving should just be putting a log .txt file in a .zip for simplicity.

This is meant as an alternative to completely deleting logs from the directory / database. .zip'ing sufficiently large log files has the potential to save several hundreds in megabytes of storage space. Which adds up fairly quickly when using this a lot.

Add optional RaidFlag filter for DamageDone calculations

For situations where adds need to die and you have an addon that automatically marks it with the "Skull" marker, its interesting to be able to see who is doing how much damage to the target.

This requires only an additional EventFilter parameter that gets passed a TargetRaidFlagFilter (needs to be created first) with the correct mark set.

More detailed breakdown for healing and damage done

The "done by spell" (per entity) breakdown should include other info like "average hit", "crit rate", and "biggest hit".

Implementation for this requires:

  • replacing the long value of the dictionary that is returned by those methods with an appropriate data type
  • adding new DataGrid columns that display those new fields

Decouple breakdown generation/calculation from GUI code

The code for generating damage / healing / etc. breakdowns should not be in the SingleEncounterView.xaml.cs file.
All of this should be in a different file. Most likely this should be part of the EncounterInfo class, or an extension for it.
Any parameters such as which player or mode is selected should be used as a parameter for the method.

Remove Sparse Arrays if possible

Already did this with strings where i went from 8MB to 1MB of duplicate strings
But there are still 4MB of sparse arrays.
some offenders that are guaranteed to be because of me:

  • CombatlogEvent[] - 610KB - example: 38% unused, array length: 204606

  • AdvancedParamEvent[] - 331KB - example: 25% unused, array length: 163840

  • DamageEvent[] - 84KB
    and other events too of course.

  • PowerType[] - 107KB - (many of these are 4B each, with 100% (1) unused, which would just be Mana, i think that can be safely ignored.)

Better log import UX

The UX when importing a log should be improved.

  • No longer only import on start-up, have a dedicated button for it.
  • Show a progress bar for the individual steps while importing a log.
  • Show a warning popup when a log has already been imported. (Just dismiss, no other actions to be taken)

Remove combatlogs from DB / Re-import feature

You should be able to remove combatlogs from the stored metadata, and/or re-import entire logs (delete old metadata, create new)
this is mostly so that if a version was bugged during import, the data can get updated.
addition: versioning for this app maybe? just an idea

Garbage collect stale combat data

After inspecting an encounter and going back to the DB View, the loaded combat data remains in memory, which can be >20MB each.
Unused combat data should get garbage collected but there is something preventing it from happening automatically

Improve Search

The Searchbar for Players (as a part of MainWindow.xaml right now) has a bunch of problems right now:

  • input breaks, only allows one character, then resets.
  • using the arrow keys to go down should not cause the player to be "selected" for inspection in the PMView.
  • Doesnt display all information (realm, class color)

Besides that it might be a good idea to seperate it out into its own control and provide an event for when a player is "really" selected

Storage preference settings

I want a config option that determines whether log files are stored as is in plaintext, or whether they should by default be compressed as a .zip archive.
Compressing files obviously leads to lower harddrive usage, while increasing load times.

Add more / better SpellSchool colors/brushes

Every basic SpellSchool type should have its own color/brush associated with it.

Additionally it would be really cool if every spell school that was a combination of multiple basic ones had a Gradient Brush across all the basic types as its Brush.
(Optional: adjust the type structure to be equivalent to what was done for ClassColors and their Brushes.)

Relevant classes / files: SpellSchoolColors, ClassColors

Navigate between Encounters in the same Combatlog

You should be able to navigate between encounters in the same combatlog.

This can either be implemented:

  • directly in the SingleEncounterView
  • as a CombatlogView that embeds a SingleEncounterView with the selected encounter.
  • as a CombatlogView that links encounters via changing the MainWindow content to a new SingleEncounterView

The second option is most likely the one that is favoured.

Add Augmentation Support Events to initial parse during import

Augmentation Evokers _SUPPORT events should be taken into consideration when creating performance metadata created by importing a combatlog file.
the way to do it is already implemented in the Single Encounter View code right now, but it has not been copied over to CombatLogParser.ProcessPerformances yet

DB View -> Encounter View link is broken

Havent yet looked into why, but it just leaves the view blank after reading the encounter. very weird
Note that this doesnt happen when navigating to the encounter view via the player page

Akaari's Soul player association unreliable.

In situations where a subtlety rogue uses Secret Technique, the damage dealt by the Akaaris Soul NPC that is spawned is currently not always counted as being done by the rogue.
There can be SPELL_CAST_SUCCESS events associated with these Souls, but they are unreliable when the rogue is far away from the person that is logging. In the case that they do appear however, the Advanced Parameters of the event refer to itself and subsequently provide the GUID of the rogue it belongs to.
For situations where only one rogue (with the relevant talent) is in the raid, the soul can just be linked to the rogue without any additional checks, and the NpcId (144961) should be used for this.

The implementation of this will probably require a bit of a rewrite of things where special rules and such can be allowed.

  • count of classes/specs within raid group
  • talent check (talents in general need to be properly done first, the IDs in the COMBATANT_INFO are difficult to figure out)

Add Breakdown modes for Buffs and Debuffs

Next to Damage done/taken and healing, one of the interesting aspects of fights are buffs and debuffs.
This comes in different varieties for different purposes:

  • Tracking buffs / debuffs across the raid
    • This aids in seeing when someone used a group affecting ability a Bloodlust effect or healing cooldown (e.g. Divine Hymn) and vice versa with large raid wide debuffs applied from the boss
  • Tracking debuffs by certain players on enemies
    • Useful for seeing uptime on DOTs and other damage increasing debuffs

DPS/HPS metrics are fucked

For some (as of now unknown) reason, the dps sums in the single encounter view are very wrong. about 3-4% off from what its supposed to be.
(For all intents and purposes, the overview/import dps is correct - within 0.02% margin of WCL on average)

overview inspect error %
82451,5 79949,7 -3,0343
76142,2 74346,1 -2,3589
75507,7 73597,3 -2,5301
90873,3 90018,7 -0,9404
80070,9 77658,7 -3,0126
83594,3 81377 -2,6525
74519,9 72574,4 -2,6107
78673,3 76187,8 -3,1593

Theres something going wrong in the dps calculation thats done in the encounter view source code.
Needless to say: The goal here is not to temporarily fix the encounterview damage/healing calculation, but to rewrite this into one combined method that works for both import and inspection.
After all, both should start out with the EncounterInfo object that is used to generate the dps totals.

Create a popup displaying an error during parsing a log

When importing a log, the user should be shown a popup giving information about the error that occurred.
This includes but is not necessarily limited to:

  • the line the program is currently parsing
  • the underlying error message with stack trace
  • link to github issues page to create a bug report

Use public get properties for data instead of public fields

As is C# standard, you shouldnt really have public fields exposed and should use get properties instead.
An example of where this currently isnt the case is the DamageEvent class.

Some events are properly using properties, while some arent. This should be fixed to get the codebase all around up to standard.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.