dantevg / webstats Goto Github PK

Instead of just showing nothing when something goes wrong, show an error message in place of the table. Also add a message "Loading..." before any data is loaded.

can i convert offline uuid from uuid column to nickname in MYCUSTOMTABLE table? I configured it like this but nothing worked

database:
hostname: localhost
username: DATABASE USERNAME
password: DATABASE PASSWORD
config:
- database: DATABASE NAME
table: MYCUSTOMTABLE
convert: # Some examples, refer to documentation for explanations
- [uuid, column]

Java Players are availabale, but Bedrock (GeyserMC) players aren't listed

the 'web' folder is empty for me. getting 'No connection to server. Maybe the server is offline, or the 'host' setting in index.html is incorrect.'

(from Unkn0wn3636 at the spigotmc WebStats thread)

Feature

For non-player-specific placeholders, display them on a separate row for the server itself, instead of displaying them for every player.

Implementation

Maybe add a separate global-placeholders config entry. All placeholders specified there will not be displayed on individual players' rows but on the server's own row.

Feature

Add option to serve the webpage from the plugin (like dynmap), so no external web server is required. Meant for people that do not have a web server set up.

Implementation

• Add config option, something like enable-internal-webserver
• Maybe integrate CSS and JS into a single HTML file so no extra http requests are needed

i have the plugin but when im connecting to my server my webstats dont work.

screen :

With this, one can make a nice automatically-updating and interactive graph of players' playtime for instance.

Problem

WebStats does not adjust to rate limiting at all. This results in getting a HTTP 429 response from Discord with this body:

{"code": 0, "message": "You are being blocked from accessing our API temporarily due to exceeding our rate limits frequently. Please read our docs at https://discord.com/developers/docs/topics/rate-limits to prevent this moving forward."}

Resources

• Link from message: https://discord.com/developers/docs/topics/rate-limits

Possible solution

TODO

Specify a custom order for the columns, in config.yml:

columns: [player, money, deaths]

For columns not in this list, either:

• don't display
• display in alphabetical order after all columns that are specified

(came from #5 (comment))

It would be nice if it's able to show top players (for example, top 100) in a selected stat.
Perhaps they can be placed in multiple sheets.

Btw, https can be achieved by reverse proxy.

Since some PlaceholderAPI plugins do not return placeholder data for offline players, store the data for them when they are online so we can use that stored data when they're offline.

• Storage method: database
• When server starts: load all data (to reduce IO)
• When player leaves: save their data
• When server closes: save all data

(came from #5 (comment))

When I apply any filter in the latest build it for some reason like resets the filter and goes back to the default view after just a few moments. I assumed it was the update interval causing it but even after disabling auto updates it still seems to remove the filters on me.

I am using the default web page you made with the default index/styles.

Example below.

s1stats.mineroyalemc.com

Command to reload config without needing to restart the entire server and without using the server's unsupported /reload command

Instead of identifying them by their display name, allow using the internal name. This prevents confusion when two objectives have the same display name.

URL: http://lcorpnet.ddns.net:8094/

Keeping this up so it can be checked to see the result

Config: https://pastebin.com/ckET1DVj

Mostly just added a bunch of placeholders for testing

Doesn't seem to initialize the table. works on a localhost test server fine. when running from a hosted server (Pebblehost) I get a similar result to this but no error in console? Stats.json IS propagated from checking.
#16

Paper: 1.18.2

I'd be happy to offer other info if needed

Problem

When changing the config and removing placeholders, the placeholder storage table still contains entries for these placeholders.

Possible solution

When storing all placeholders into the database, remove any rows that have a placeholder name that is not known.

Folia is a new fork of Paper centred around multithreading, and it seems to still be at an experimental stage right now. Once plugins that WebStats depends on support it, WebStats should also have support for Folia.

However, support for Folia should not come at the expense of support for Paper, as I expect the grand majority of servers will stay running Paper.

Thanks to radeon on Discord for the suggestion!

Stats sources support

• Scoreboard: ❌ not supported

  ALL scoreboard API is considered broken (this is global state that I've not figured out how to properly implement yet)

• Database: ✔️ probably fine
• PlaceholderAPI: ⏳ in-progress branch

Resources

• Note on plugin compatibility and API plans: https://github.com/PaperMC/Folia#plugin-compatibility
• Prior art
  • LuckPerms: https://github.com/LuckPerms/LuckPerms/pull/3615/files
  • Chunky: https://github.com/pop4959/Chunky/pull/256/files
  • EssentialsX: https://github.com/EssentialsX/Essentials/pull/5291/files

To do

• Add folia-supported: true to plugin.yml
• Change scheduler to use one of the Folia schedulers
• Wait for either PlaceholderAPI or the scoreboard API to start working
• ???

Add functionality from https://github.com/Dantevg/WebStatsStorer into the main plugin.
To reduce file size, also only store a statistic for a player if it actually changed.

Currently all color codes are not serialized when imported to json, would be very cool to have this parse and translate the colors, or at the least filter the codes out of the output on the plugin end.

Maybe only request a statistic update when people are online?

Problem

Sometimes randomly the web frontend stops working and stopping the server hangs on "Disabling WebStats". Score gathering still works (WebStatsStorer still saved new stats)

Output of /webstats debug command:

WebStats 1.6.1
Active sources: scoreboard
Thread status: alive
Socket status: open

Possible cause

Something in ServerSocket or HTTPConnection hangs

Possible solution

Look from here where something can hang indefinitely:

Additional solution: don't wait indefinitely on thread join here: (timeout of a couple seconds should be good)

Also maybe use Thread::getState in debug output

Resources

• ServerSocket docs: https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/net/ServerSocket.html
• Thread docs: https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/Thread.html

hey

Is it possible to add Placeholders compatibility? for example add faction power to the table, or mcmmo level, by adding %placeholder% to a .yml config

Can use EssentialsX API to detect whether a player is vanished.

Thanks to Just4Fun on the Spigotmc forums for the suggestion!

(from Unkn0wn3636 at the spigotmc WebStats thread)

Problem

Minecraft version: 1.18.2

Stack trace:

[11:32:01 ERROR]: Error occurred while disabling WebStats v1.6.1 (Is it up to date?)
java.lang.IllegalStateException: zip file closed
at java.util.zip.ZipFile.ensureOpen(ZipFile.java:831) ~[?:?]
at java.util.zip.ZipFile.getEntry(ZipFile.java:330) ~[?:?]
at java.util.jar.JarFile.getEntry(JarFile.java:518) ~[?:?]
at java.util.jar.JarFile.getJarEntry(JarFile.java:473) ~[?:?]
at org.bukkit.plugin.java.PluginClassLoader.findClass(PluginClassLoader.java:163) ~[paper-api-1.18.2-R0.1-SNAPSHOT.jar:?]
at java.lang.ClassLoader.loadClass(ClassLoader.java:587) ~[?:?]
at org.bukkit.plugin.java.PluginClassLoader.loadClass0(PluginClassLoader.java:108) ~[paper-api-1.18.2-R0.1-SNAPSHOT.jar:?]
at org.bukkit.plugin.java.PluginClassLoader.loadClass(PluginClassLoader.java:103) ~[paper-api-1.18.2-R0.1-SNAPSHOT.jar:?]
at java.lang.ClassLoader.loadClass(ClassLoader.java:520) ~[?:?]
at me.vagdedes.ultimatestatistics.b.b.b.onRequest(PlaceHolder.java:46) ~[UltimateStatistics.jar:?]
at me.clip.placeholderapi.replacer.CharsReplacer.apply(CharsReplacer.java:161) ~[PlaceholderAPI-2.11.1.jar:?]
at me.clip.placeholderapi.PlaceholderAPI.setPlaceholders(PlaceholderAPI.java:70) ~[PlaceholderAPI-2.11.1.jar:?]
at nl.dantevg.webstats.placeholder.PlaceholderSource.lambda$getScoresForPlayer$1(PlaceholderSource.java:93) ~[WebStats-1.7.0-pre1.jar:?]
at java.util.LinkedHashMap.forEach(LinkedHashMap.java:721) ~[?:?]
at nl.dantevg.webstats.placeholder.PlaceholderSource.getScoresForPlayer(PlaceholderSource.java:92) ~[WebStats-1.7.0-pre1.jar:?]
at nl.dantevg.webstats.placeholder.PlaceholderStorage.save(PlaceholderStorage.java:121) ~[WebStats-1.7.0-pre1.jar:?]
at nl.dantevg.webstats.placeholder.PlaceholderStorage.saveAll(PlaceholderStorage.java:154) ~[WebStats-1.7.0-pre1.jar:?]
at nl.dantevg.webstats.placeholder.PlaceholderStorage.disconnect(PlaceholderStorage.java:59) ~[WebStats-1.7.0-pre1.jar:?]
at nl.dantevg.webstats.placeholder.PlaceholderSource.disable(PlaceholderSource.java:105) ~[WebStats-1.7.0-pre1.jar:?]
at nl.dantevg.webstats.WebStats.onDisable(WebStats.java:99) ~[WebStats-1.7.0-pre1.jar:?]
at org.bukkit.plugin.java.JavaPlugin.setEnabled(JavaPlugin.java:266) ~[paper-api-1.18.2-R0.1-SNAPSHOT.jar:?]
at org.bukkit.plugin.java.JavaPluginLoader.disablePlugin(JavaPluginLoader.java:399) ~[paper-api-1.18.2-R0.1-SNAPSHOT.jar:?]
at org.bukkit.plugin.SimplePluginManager.disablePlugin(SimplePluginManager.java:538) ~[paper-api-1.18.2-R0.1-SNAPSHOT.jar:?]
at org.bukkit.plugin.SimplePluginManager.disablePlugins(SimplePluginManager.java:515) ~[paper-api-1.18.2-R0.1-SNAPSHOT.jar:?]
at org.bukkit.craftbukkit.v1_18_R2.CraftServer.disablePlugins(CraftServer.java:493) ~[paper-1.18.2.jar:git-Paper-235]
at net.minecraft.server.MinecraftServer.stopServer(MinecraftServer.java:973) ~[paper-1.18.2.jar:git-Paper-235]
at net.minecraft.server.dedicated.DedicatedServer.stopServer(DedicatedServer.java:803) ~[paper-1.18.2.jar:git-Paper-235]
at net.minecraft.server.MinecraftServer.runServer(MinecraftServer.java:1272) ~[paper-1.18.2.jar:git-Paper-235]
at net.minecraft.server.MinecraftServer.lambda$spin$0(MinecraftServer.java:317) ~[paper-1.18.2.jar:git-Paper-235]
at java.lang.Thread.run(Thread.java:833) ~[?:?]

Resources

• This has the same stack trace and a similar plugin setup: https://hub.spigotmc.org/jira/browse/SPIGOT-6621

Possible fixes

• Use https://hub.spigotmc.org/javadocs/spigot/org/bukkit/event/server/PluginDisableEvent.html
  This event gets called any time a plugin is disabled, so kind of a brute-force way
• Save placeholder player data some time earlier (on player join, every couple hours?)
• Maybe this is a placeholderAPI bug, rely on them fixing it (maybe by checking whether a plugin is still loaded somehow?)

Using TypeScript could catch some errors earlier, but it might not be useful

This server is running Purpur version git-Purpur-1449 (MC: 1.18.1) (Implementing API version 1.18.1-R0.1-SNAPSHOT) (Git: 91e3be5 on HEAD)

There's nothing on the page except:
Compact Hide offline players Prev Next

An error appears in the console:
[20:18:56 WARN]: Exception in thread "WebStats" java.lang.IllegalStateException: @NotNull method nl/dantevg/webstats/placeholder/PlaceholderStorer.getScore must not return null
[20:18:56 WARN]: at WebStats-1.6.0.jar//nl.dantevg.webstats.placeholder.PlaceholderStorer.$$$reportNull$$$0(PlaceholderStorer.java)
[20:18:56 WARN]: at WebStats-1.6.0.jar//nl.dantevg.webstats.placeholder.PlaceholderStorer.getScore(PlaceholderStorer.java:158)
[20:18:56 WARN]: at WebStats-1.6.0.jar//nl.dantevg.webstats.placeholder.PlaceholderSource.lambda$getScores$0(PlaceholderSource.java:64)
[20:18:56 WARN]: at java.base/java.util.LinkedHashMap.forEach(LinkedHashMap.java:721)
[20:18:56 WARN]: at WebStats-1.6.0.jar//nl.dantevg.webstats.placeholder.PlaceholderSource.getScores(PlaceholderSource.java:56)
[20:18:56 WARN]: at WebStats-1.6.0.jar//nl.dantevg.webstats.placeholder.PlaceholderSource.getStats(PlaceholderSource.java:90)
[20:18:56 WARN]: at WebStats-1.6.0.jar//nl.dantevg.webstats.Stats.getStats(Stats.java:23)
[20:18:56 WARN]: at WebStats-1.6.0.jar//nl.dantevg.webstats.Stats.getAll(Stats.java:34)
[20:18:56 WARN]: at WebStats-1.6.0.jar//nl.dantevg.webstats.HTTPConnection.route(HTTPConnection.java:50)
[20:18:56 WARN]: at WebStats-1.6.0.jar//nl.dantevg.webstats.HTTPConnection.start(HTTPConnection.java:25)
[20:18:56 WARN]: at WebStats-1.6.0.jar//nl.dantevg.webstats.WebStats.run(WebStats.java:115)
[20:18:56 WARN]: at java.base/java.lang.Thread.run(Thread.java:833)

The plugin gets the information and writes it to SQL

webstats debug
[20:24:11 INFO]: WebStats 1.6.0
Active sources: scoreboard, PlaceholderAPI
Thread status: dead
Socket status: open
Placeholder storer database connection: connected
Loaded placeholders:
81ada1cc-0334-4d7d-8806-0e8e78cc2a7f (Fandorin): balance = 823.3333333333333
81ada1cc-0334-4d7d-8806-0e8e78cc2a7f (Fandorin): displayed name = Fandorin
81ada1cc-0334-4d7d-8806-0e8e78cc2a7f (Fandorin): uuid = 81ada1cc-0334-4d7d-8806-0e8e78cc2a7f

Config:

port: 8989
columns: [uuid, balance]
objectives:

• '*'
  database:
  hostname: localhost:3306
  username: MineFS_WebStat
  password: xxxxxxx
  placeholders:
  '%cmi_user_name%': displayed name
  '%cmi_user_uuid%': uuid
  '%cmi_user_balance%': balance
  store-placeholders-database: MineFS_WebStat

Some values have large numbers and it can be hard to see what they mean. The option to specify per column what the unit of the values is would solve this. Possible units / categories:

• time: milliseconds, ticks(?), seconds, minutes, hours (e.g. 12:34:56.789)
• item count: stacks, items (note: can be confusing for 16-stackables) (e.g. 1st 23)
• distance: meters/blocks, kilometres (e.g. 123m, 12.34km)
• date: formatted, timestamp (regional formatting or ISO)
• ... more?

Example config: (minutes, ticks, items, timestamp are taken as the source units, WebStats will choose best units to display)

column-units:
  Play Minutes: minutes
  Aviate: ticks
  Mine Diamond: items
  Last Seen: timestamp

Possible extra function: show original value on hover ("tooltip")

Add an arrow in the currently sorted column, indicating the sorting direction (ascending/descending).

The arrow can probably be an inline svg triangle.

Make the webpage more accessible:

• Add alt text to images
• Add tabindex=0 to table column headings
• Add onkeydown event where using only onclick now
• ...

(from Unkn0wn3636 at the spigotmc WebStats thread)

Feature

Add possibility to display multiple tables with different scores, on different pages.

Implementation

• Config item tables: list or named list, each item of this list is a list of columns. Mutually exclusive with columns of course.
• Probably specify which table/page with query string stats.json?page=1 or stats.json?page=pagename

Hello, your guide is a bit confusing, could you make a video version or something like that?

(from draexo at the spigotmc WebStats thread)

Feature

Discord webhook that posts a (non-interactive) embed message with the top 10 (or so) sorted on a specified stat. Auto-update every minute (or so).

Implementation

Probably use inline fields. This has a problem: if there are too many columns or player names are too long, overflowing columns get put on a new row, breaking the visual connection to the player. Alternative is to use code blocks for monospaced manual table display.

(image from embed builder)

Resources

• Intro to webhooks: https://support.discord.com/hc/en-us/articles/228383668-Intro-to-Webhooks
• Discord developer portal: https://discord.com/developers/docs/resources/webhook
  • Execute Webhook
  • Edit Webhook Message
  • Embed Object
  • Allowed Mentions
• Embed builder GUI: https://autocode.com/tools/discord/embed-builder/
• Embed builder GUI: https://discohook.org
• Some nice Java discord webhook library?
• Custom Gson serialisation: https://github.com/google/gson/blob/master/UserGuide.md#TOC-Custom-Serialization-and-Deserialization

Feature

The ability to store the placeholders for offline players in a plain text file instead of requiring a database

Implementation

• Storing in yaml file, similar to current player highlight ip-to-names.yml (#19), or
• Storing in csv file, similar to Dantevg/WebStatsStorer. May be more efficient storage-wise, but may be harder to write and read

Config option boolean store-placeholders-in-file, mutually exclusive to store-placeholders-database.
Otherwise, maybe change config to:

• store-placeholders
  • method (database or file)
  • database-name

Add a "bar graph" to each cell: a horizontal filled bar behind each cell, scaled by the relative score of that player for that objective. This will make it easier to compare scores at a glance, rather than having to read the numbers: 6,241 does not look a lot more than 2,243 at first, yet it is.

Just creating this as a separate issue to keep track.

If all stats for a user is 0, the stats table does not load at all, and errors out with a "Cannot read property of undefined" or something similar (the error disppeared as soon as one of the scoreboard-values was updated to 1, so I don't have the exact error message in frnt of me anymore).

If just some values are 0, the table looks a bit strange as 0-values are not shown:

Support serving the stats over https natively, somehow. Without relying on manual configuration.

For the internal web server, this is not needed because everything is unencrypted. For an external web server this is also not needed because you can set up a reverse proxy. Native HTTPS support is only needed for external websites created with services like Wix, Squarespace or Weebly, where you cannot set up a reverse proxy.

It is possible (and easy) to generate a self-signed certificate and use that. But, browsers will give you a big fat warning.
So, we need to somehow get a validated certificate from something like letsencrypt. The problem is that they use DV (domain validation), which seems to not be available for IPs, only domains.

Challenge types: https://letsencrypt.org/docs/challenge-types/

The HTTP-01 and TLS-ALPN-01 challenge types use port 80 or 443 to verify the certificate, which you do not have access to unless you self-host.

DNS-01 challenge

We can use the DNS-01 challenge in manual mode. Downside: need to manually renew the certificate every few months.
How to get a letsencrypt certificate with DNS validation: [IN PROGRESS]
https://eff-certbot.readthedocs.io/en/stable/using.html#manual

sudo certbot certonly --manual --preferred-challenges dns

sudo openssl pkcs12 -export -out webstats.p12 -in /etc/letsencrypt/live/tlstest.equinoxmc.nl/fullchain.pem -inkey /etc/letsencrypt/live/tlstest.equinoxmc.nl/privkey.pem -name webstats

sudo keytool -importkeystore -srckeystore webstats.p12 -srcstoretype pkcs12 -srcalias webstats -destke
ystore webstats.jks -deststoretype jks -deststorepass webstats -destalias webstats

DNS-01 with DuckDNS

DuckDNS is a free DNS provider with an API to change A and TXT records. This means you can use a DuckDNS subdomain as the domain for the WebStats plugin on your server.

DNS TXT record URL format: (https://www.duckdns.org/spec.jsp)

https://www.duckdns.org/update?domains=<SUBDOMAIN>.duckdns.org&token=<DUCKDNS_TOKEN>&txt=<DNS_CHALLENGE>

Java ACME challenge library

To automatically renew the certificate, the plugin needs to use a Java library. Acme4J is listed in the letsencrypt list of Java libraries. [IN PROGRESS]

ZeroSSL

ZeroSSL seems to be an alternative to letsencrypt that does support TLS certificates for IP addresses: https://zerossl.com/documentation/api/create-certificate/. This still does not work because we would need to get a certificate for only the WebStats port.

Resources

• DNS check: https://toolbox.googleapps.com/apps/dig
• Certbot manual: https://eff-certbot.readthedocs.io/en/stable/using.html
• ACME clients: https://letsencrypt.org/docs/client-options/#other-client-options
• ACME libraries: https://letsencrypt.org/docs/client-options/#libraries
• Reddit thread with pointers: https://www.reddit.com/r/selfhosted/comments/zxf4ns/is_it_possible_to_enable_tls_without_a_domain/
• Tutorial with extra steps: https://gist.github.com/ioqy/5a9a03f082ef81f886862949d549ea70

Every time I initialize the server after a few seconds it isn't accessible anymore. If I reload the plugin it comes back but then it turns off again.

Hi, I am trying to understand how to setup this exactly,

The help to setup isnt easy for me to understand (Prolly just me), but I cant seem to understand how to add in multiple columns and link them to their correct placeholder/scoreboard objective.

And as for a webpage, I do not quite understand how to set it up correctly :/

Is there an example of any of these that can help me understand it all better?

A source map is already generated, but the plugin does not serve it. Letting the plugin serve the source map allows for better debugging and minimisation of WebStats-dist.js

The plugin should reconnect automatically after the connection is closed.
https://pastebin.com/eN1LkPM8

An option to only display players that are online, with a button/checkbox to toggle

(came from #5 (comment))

It could just be me, but I can't get the column-ordering to work.

I have changed config.yml and added:

tables:
    #columns: []
    columns: ['Level', 'XP', 'Total Kills', 'Death Count']

These are the display names, but I have also tried the objective name (if I understand this correctly). E.g:

> scoreboard objectives add deathCount deathCount "Death Count"

And then use deathCount in the config:

    columns: ['level', 'XP', 'totalKills', 'deathCount']

But it always displays them in alphabetical order:

Another little issue there is that it would be nice if empty columns could show up. It was very confusing when I started setting up this plugin that I did not see anything.

hey there, again.
Some improvements! congratulations!

Here is some of my doubts:

• how to config the database saving on the 'store-placeholders-database: WebStats_placeholders'?
• neither 'hide offline players' or compact works
• can u give me an example how to sort the collums? #columns: [money, playername, jumps] ?

Hey, would you ever consider making this MySQL compatible? I love the idea here, but I don't use the Scoreboard and store all my Data in MySQL.

Feature

Like Dynmap can show player names for messages sent from the browser, WebStats should be able to highlight the row for your player.

Implementation

Probably by storing players' IPs when they are online and matching that against the request IP.

Since this is my only way to contact you, can you help me with the implementation?
We have our server (purpur) and website (laravel on xampp) running on a laptop but im kinda noob with this stuff.
Is the following code right? https://pastebin.com/tQcApvkK

From console:

Nag author(s): '[RedPolygon]' of 'WebStats' about their usage of System.out/err.print. Please use your plugin's logger instead (JavaPlugin#getLogger).

Happens when closing WebStats browser tab.

Error: https://pastebin.com/8TsUXPMQ