monk-afk / filterplus Goto Github PK

Only detects Latin characters.

• make it so the word-list files are loaded first, then the mod storage.

• only save list items added by the /filter commands to mod_storage, don't save words already in the *list.lua files.

• update /reload_filter to this change

	if #message > max_caps then
		lower(message)
	end

should be on line 130

would be useful to have a /blacklist dump to print the contents of the blacklist in a lua table

Hello! i have a feature request so that other mods can use integrate filterplus by exposing
an API for them to use.
the API im suggesting is very similar to how https://content.minetest.net/packages/sofar/filter/
does things, but its very old and dosent provide a sane default wordlist. the main functions i'd expect are

filterplus.check_text(text)
checks the text for swears
the first return value is a bool indicating whether the message is allowed
the second return value should be the positing of where the bad word starts,
possibly return a list of number pairs that show where each swear begins/ends instead

example

local success, swears = filterplus.check_text("frick you witch")
-- success = false
-- swears = 
-- {
--   {1, 5},   -- where "frick" starts and begins
--   {11, 15}, -- where "witch" starts and begins
-- }

also add a way for mods to register callbacks for when players violate

filterplus.on_violation = {}
filterplus.register_on_violation(function(player, message)

end)

other APIs could be provided as well (adding/removing words to blacklist etc), but these are the most important ones

i think the best way to go about doing this is by splitting this core functionality into its own mod
filterplus_lib since currently filterplus provides much more than just chat filtering, which can get in the way
of mod makers (as it does in my usecase). and make filterplus_lib a dependency of filterplus

filterplus_lib should only implement the APIs, here are some things it shouldnt do

• filter chat messages, its up to mods that depend on filterplus_lib to do this
• register any punishment callbacks, its up to the mod to do that
• ranks/tags
• mentions

not sure how checking for link URLs and all caps messages fit into this.
they should be configurable at least

for sake of completeness, here is how my usecase looks like.
im making a mod that completely reimplements everything that has to do with the chat.
its comes with a filly featured interpreted language inspired by POSIX shell and minecraft syntax
for chat commands, that provides a base API for defining chat commands that are more powerful and easier to write.
with that base im also implementing all the commands a user would ever want, reimplementing most builtin commands to use my much more powerful API, this includes moderation tools like muting/banning/blocking etc and i would like to optionally include chat filtering, but i dont want to reinvent the wheel and filterplus adds duplicate features like muting that would break things for me

i cant personally help implement this but i can provide feedback from the perspective of a modder if you decide to implement this

sidenote: on contentDB filterplus has the "API/liberary" tag and says that it provides an API in the description, even though it dosent. my guess is that this was planned but later forgotten

entering one space will send empty message

UTF reference codes aren't saved to mod_storage as encoded. The string is converted to reference (\u00f)
This causes the filter to bug out

see image

if player name is "monk":

• "monk" is highlighted green
• "Monk" is not
• "MONK" is not
• etc etc

to filter commonly used evasion technique, possibly merge the proceeding word with the current one after removing symbol. for example:

"this is a test" would check for: "thisis", "isa" "atest", as well as the individual words.

this will cause some false-positives, the main one being "puta", example: "I put a block here". In tests using str.find without spaces "puta" was found 1081 time out of the 4890 blacklist hits, more than "fuck" which was found 1024 times.

chat messages:       4205567
words total:         13178334
blacklist items:     413
filter hits:         4890
clock time:          53.435444

if the added checks aren't too costly, here is the most recent profiler dump:

2024-02-11 07:29:17: ACTION[Server]: Values below show absolute/relative times spend per server step by the instrumented function.
A total of 2306385 samples were taken

 instrumentation           |   min µs |   max µs |   avg µs | min % | max % | avg %
-------------------------- | -------- | -------- | -------- | ----- | ----- | ------
filterplus:                |        0 |    72973 |       16 |   0.0 |  92.3 |   0.3
  - on_leaveplayer[1] .... |        1 |      788 |        4 |   0.0 |  19.1 |   0.1
  - on_chat_message[1] ... |        5 |    72973 |      648 |   0.0 |  92.3 |  13.0
  - /mute ................ |      289 |      361 |      332 |   1.2 |  18.9 |  10.5
  - on_joinplayer[1] ..... |        1 |     2779 |        4 |   0.0 |   8.1 |   0.0

another string manipulation needed it removing repeating characters. no english word (that i know of) uses more than two repeating letters, so removing the additional repeats would be handy.

a quick writeup which mostly works:

local function remove_repeating(message)
	return message:gsub("([%S]+)([%S])%1", "%2")  -- keep doubled chars
end

print(
	remove_repeating("hi assume this teeeeeesssssttttt aggregate okkkk")
)

bug with the above script is: for odd numbered repeats: 55555 it leaves one character, and with even numbered repeats: 4444 it leaves two characters, so many word variants would have to be added to blacklist, ex: one, oone, oonnee, onnee, etc.. this is not efficient but it's a starting point anyway

/ignore_all to ignore the public chats, but still can receive direct message

/ignore <playername> to ignore specific player

if these functions are enabled, the player shouldn't be able to send to public chat/ignored player

if minetest.check_player_privs(name, {server = true}) then

should be

if minetest.check_player_privs(name, {blacklist = true}) then