Deals with most of back-end so you can focus on creation of functionality.

• Local environment for each guild saved on remote server
• AES encryption of data
• Compression of data
• Custom (tree based) parser for commands
• Object-oriented
• Aliases (macros) to shorten frequently used commands
• Bundles (send multiple commands in one)
• Error handling (logging)
• Status check (test integration with third-party services)
• Translation and language detection
• Automatic loading of .py files in .features/ subdirectory
• Code simple enough so every programmer can figure it out and tune
• Translation using emojis as example what it can be used for :)

Created and tested for Python 3.11
All python packages required are listed in requirements.txt, install them using pip install -r requirements.txt

Tokens must be included in .env file in working directory, containing:

# Required. Your discord's bot token, You can get one at https://discord.com/developers/applications
DISCORD_TOKEN="your token here"
# Required. Your token for dropbox's app. You can get one at https://www.dropbox.com/developers/documentation/python
DROPBOX_TOKEN="your token here"
# Optional. Token for better language detection library. You can get one at https://detectlanguage.com/
DETECT_LANGUAGE_TOKEN="your token here"
# Optional. Number of minutes between saves of database, if unset it will default to 30 minutes
DATA_SAVE_INTERVAL_MIN=number_of_minutes_between_database_save

Discord bot must have... all intents enabled.

Engine requires files .salt.dump and .aeskey.dump for encryption. If they don't exist, it will generate them automatically. Don't lose those files or you won't be able to load your data from storage!

To start, run python executable_main.py while in the main directory. Project first loads and sets-up all relevant scripts from engine/, then imports all scripts from .features/. Both those directories are added to sys.path so it's as-if they were in the same directory.

Discord.py is awesome library but it lacks storage for permanent, guild-specific data. If you want to have separate settings for every guild (server) or guild member, you must implement it on your own. Simplest implementation would be dict() variable with key=guild.id used as storage, dumped on shutdown and loaded on startup using pickle. Zeke does precisely that for you, and on top of that provides encryption, periodic backup and saving on remote server (Dropbox).

Furthemore, Zeke has custom command parser, granting you better control over it. While the code is complex, for the most part it uses only pure python and basic libraries, so you should be able to work out how features work and modify them to fulfill your needs.

This bot requires message content priviledged intent to create its' own command parser, which does work and it's much more convenient for me than Discord's commands, BUT as far as I've heard Discord DOESN'T verify bots (and you need to verify your bot once it goes big, i THINK it's over 100 servers) that use message content only for command parsing. So if you decide to use Zeke as your backend, you will either need to reimplement parser to use Discord's command or make a reasonable feature using message content intent.

Custom parser of Zeke has feature called alias, which allows you to replace any set of commands with a keyword. Example:

zeke alias add $add_alias zeke alias add

now by using add_alias you can call zeke alias add. Character $ is used before keyword when you want to add or remove alias, but not when actually using this alias (it's done for implementation reason: aliases are replaced before feeding command into parser, meaning removal would be impossible cuz name would be immediately replaced with full command). Alias can be used to change to "change" bot's prefix: zeke alias add $zk zeke allows you to use zk instead of zeke.

Note that you might accidently or purposely break the bot with aliases. For example, zeke alias add $zeke broken would replace zeke (which is main command prefix) with broken (which is just text, not any command) and parser would ignore this message, meaning using any command would be impossible. To mitigate this problem, I've implemented no-alias mode. You can block aliases by starting command with $ character, allowing you to remove problematic aliases: $zeke alias remove $zeke.

Bundle is another feature of parser, allowing you to bundle multiple commands into one. Usage zeke bundle command1 ; command2 ; command3. ; is used as separator for commands.

Custom parser of Zeke has also support for admin roles - Discord roles that allow user to use any command regardless of their permissions on the server. Every Parser object has it's own admin role and it propagates to all Parsers under it (so adding role as priviledged role for MainParser will allow users to use literally any command there's)

Admin roles can be configured with admrole command, included in every Parser object. In case of MainParser: zeke admrole <mention_role>.

To add a feature, you should create .py file in .features/ directory. It will be automatically loaded on startup and you can "attach" your own code to the main process using Triggers. It may sound complicated so let me show an example

At first, let's load all of important modules. We won't use all of them, but here's import-recipe for you.

from triggers import objectTriggers as Triggers
from triggers import objectTimers as Timers
from triggers import objectGlobalTimers as GlobalTimers
from tools import objectTools as Tools
from tools import objectTranslateTools as TranslateTools
from database import objectDatabase as Database
from cmdparser import objectMainParser as MainParser
from cmdparser import Parser, Command
from envvars import objectEnvVars as EnvVars
from discordbot import DiscordBot

Usually first thing within the script is to add some data to default environment of guilds. Database.Default (type GuildEnv) is default environment which is used to update every existing guild environment. TL:DR if you add something to Database.Default it will appear in every guild environment, including existing ones.

Let's take simplified code of translate.py:

Database.Default.Settings.AddDefault("reaction_translator", dict()) # dict[emoji] = tgt_lang

Here we create new dictionary dict() and store it in Env (from guildenv.py), inside Settings part, under name reaction_translator. Env has three attributes: Settings, Data, Temporary. They're the same except Temporary is lost on shutdown. All data stored in Database must be pickle-able and MUST NOT contain any reference loops.

Dictionary we've created can be used to store any kind of data. In this scenarion, we will use emoji (type string) as key and language code as value.

def PartialToFullReaction(PartialEmoji, message):
    for reaction in message.reactions:
        if str(PartialEmoji) == str(reaction): return reaction
    return None

async def on_raw_reaction_add(local_env, payload, PartialEmoji, member, guild, message):
    if message.author.bot: return # Skip if reaction added to bot's message
    if len(message.content) < 4: return # Skip if it's nearly empty message
    reaction = PartialToFullReaction(PartialEmoji, message) # Get full version of partial reaction (those are discord types)
    if reaction and reaction.count > 1: return # Skip if there're more such reactions already added
    emoji = str(PartialEmoji) # Stringify emoji - key for our dictionary
    text = message.content # Get text of the message
    reaction_translator = local_env.Settings.Get("reaction_translator") # get reference to dictionary object
    # local_env.Settings.Set("reaction_translator", reaction_translator) # If your data can't be accessed with a refence (python number, for example) you can set this value later with Env.Set
    if emoji in reaction_translator: # if emoji is programmed to translate message
        tgt_lang = reaction_translator[emoji] # get target language
        (src_lang, _, translated) = TranslateTools.Translate(text, tgt_lang) # Get src_lang, tgt_lang and translated text
        await message.add_reaction(PartialEmoji) # Add reaction with the same emoji, so users can't spam-translate the same message
        await Tools.DcReply(message, translated, postprocess = Tools.DcWrapCode) # send message with translation
Triggers.Get("on_raw_reaction_add").Add(on_raw_reaction_add)

This makes on_raw_reaction_add run every time anyone adds a reaction, even if corresponding message isn't loaded by the bot. Triggers are Zeke's way of expanding functionality; no need to modify engine, you just append your function and Zeke does the rest. More about available triggers and their syntax in separate chapter.

Above code allows to translate messages with reaction, but there's still missing part: we need to fill reaction_translator dictionary with data, so it knows which emojis it should react to. Of course, it could be hardcoded but I prefer to make commands so users can fill their translation database on their own.

async def cmd_add(ctx, args, trail):
    if len(args) != 2: raise RuntimeError("Incorrect arguments. Expected: <emoji> <language>")
    local_env = Database.GetGuildEnv(ctx.guild.id) # Get GuildEnv for this guild
    emoji = args[0]
    lang = args[1]
    reaction_translator = local_env.Settings.Get("reaction_translator")
    if emoji in reaction_translator: raise RuntimeError(f"This emoji is already occupied by '{reaction_translator[emoji]}' language")
    reaction_translator[emoji] = lang

Zeke's custom command parser requires function with header async func(ctx, args, trail). ctx is of discord's type Context, but both args and trail are lists. It works like this: zk3 translate add 🇵🇱 pl -> args = ["🇵🇱", "pl"], trail = ["zk3", "translate", "add"]. Trail usually isn't very useful, used mostly by built-in command "help".

Commands are considered to succeed if executed properly. If cmd_add doesn't return or returns None/False, then Zeke automatically adds 👍 reaction; this can be prevented by returning True. If you wish to give error feedback, raise any exception and Zeke will forward it to the user.

Once function is created, we must add it to MainParser somehow. Typically, for features I'm making new instance of Parser which i fill with commands for given feature, then I add this parser as new command to MainParser

parser = Parser("translate")
parser.Add( Command("add", cmd_add, Help = "Add emoji translation for language.", LongHelp = "Add emoji translation for language.\nSyntax: TRAIL <emoji> <language>") )
#parser.Add( Command("remove", cmd_remove, Help = "Remove emoji translation for language.", LongHelp = "Remove emoji translation for language.\nSyntax: TRAIL <emoji>") )
#parser.Add( Command("list", cmd_list, Help = "Display list of current emojis used in translations.", LongHelp = "Display list of current emojis used in translations.\nSyntax: TRAIL") )
#parser.Add( Command("custom", cmd_custom, Help = "Display list of available custom languages.", LongHelp = "Display list of available custom languages.\nNot real languages obviously.\nSyntax: TRAIL") )
MainParser.Add( Command(parser.Name(), parser, Help = "Setup translation feature", StaticPerms=discord.Permissions.all()) )

Syntax for commands and parsers will be described more in another chapter.

Triggers are Zeke's way of expanding functionality - if you want your functionality to react to message being sent, you create such function (coroutine, actually) and add it to corresponding trigger. It's very similar to discord events, but Zeke usually passes additional arguments to trigger calls.

Here's recipe how to add every type of trigger.

Triggers.Get("on_dm").Add(func) # async func(message)
Triggers.Get("on_message").Add(func) # async func(local_env, message)
Triggers.Get("on_reaction_add").Add(func) # async func(local_env, reaction, user)
Triggers.Get("on_reaction_remove").Add(func) # async func(local_env, reaction, user)
Triggers.Get("on_raw_reaction_add").Add(func) # async func(local_env, payload, emoji, member, guild, message)
Triggers.Get("on_raw_reaction_remove").Add(func) # async func(local_env, payload, emoji, member, guild, message)
Triggers.Get("on_member_join").Add(func) # async func(local_env, member)
Triggers.Get("on_member_remove").Add(func) # async func(local_env, member)
Triggers.Get("on_guild_join").Add(func) # async func(local_env, guild)
Triggers.Get("on_guild_remove").Add(func) # async func(local_env, guild)
Triggers.Get("on_ready").Add(func) # async func()

All triggers and parameters are self explanatory or taken directly from discord.py so I won't describe them. local_env is GuildEnv of server (guild) within which this trigger was called. Also it's worth noting that Zeke ignores messages and reactions sent in DM or by other bots. You can get DMs with on_dm trigger but given guild-oriented focus of this bot, there's little support for that (f.e. Database only works with guilds, not DMs)

There are also strictly Zeke's triggers

Triggers.Get("Initialisation").Add(func) # async func()
Triggers.Get("Status").Add(func) # async func(), returns (name:string, result:bool, err_mess:string)

Initialisation is called after the entire engine (especially EnvVars) is initialised. Status deserves own chapter, but in short: it's used to create status check for given feature (useful to detect and debug problems with 3rd party integrations)

Timers are functions that get triggered every X seconds. They can called for each guild separately (Timers) or called only once (GlobalTimers).

Syntax:

• Timers.Add(time, func) # async func(local_env, guild, second)
• GlobalTimers.Add(time, func) # async func(second)

where second is current value of counter that increments each second. time is given in seconds, use Tools.ToSeconds(seconds=0, minutes=0, hours=0, days=0, weeks=0) for easy and readable conversion.

There's a arbitrary limitation; internal counter is resetted every constant.SECONDS_COUNTER_RESET seconds, so you can't really schedule any task longer than that. If you do, this task will be executed every time counter resets (28 days by default).

Zeke comes with custom-made command parser to give programmer more control. It uses two important classes: Command and Parser. Command represents singular function (leaf in parser tree), while Parser controls control flow through the branches. I think it will make more sense when explained with examples:

zeke help

In this case, control flows into MainParser (root of the parser tree, the main Parser of bot) which then calls help Command.

zeke translate add <emoji> <lang>

In this case, control flows into MainParser which calls translate Command. This command contains another instance of Parser, meaning it passes the (flows) control forward, to add Command. <emoji> and <lang> are args for this command.

Tree-like structure allows to group commands together and makes it overall easier to deal with when there's a lot of them. Custom Parser also supports static and dynamic permissions.

Example of code how to create local parser and attach it to MainParser.

parser = Parser("translate")
parser.Add( Command("add", cmd_add, Help = "Add emoji translation for language.", LongHelp = "Add emoji translation for language.\nSyntax: TRAIL <emoji> <language>") )
#parser.Add( Command("remove", cmd_remove, Help = "Remove emoji translation for language.", LongHelp = "Remove emoji translation for language.\nSyntax: TRAIL <emoji>") )
#parser.Add( Command("list", cmd_list, Help = "Display list of current emojis used in translations.", LongHelp = "Display list of current emojis used in translations.\nSyntax: TRAIL") )
#parser.Add( Command("custom", cmd_custom, Help = "Display list of available custom languages.", LongHelp = "Display list of available custom languages.\nNot real languages obviously.\nSyntax: TRAIL") )
MainParser.Add( Command(parser.Name(), parser, Help = "Setup translation feature", StaticPerms=discord.Permissions.all()) )

Parser(name)
    name: string - Name of the parser, used f.e. in built-in help. Should be the same as name of "overlying" Command.

Command(name, obj, [Help, LongHelp, StaticPerms, DynamicPerms])
    name: string - Name of command (used to call it)
    obj: async func(ctx,args,trail) | Parser - command-function or another parser
    Help: string - optional, short (one line) help for this command
    LongHelp: string - optional, long help for this command, it's the same as Help if unset
    StaticPerms: discord.Permissions - optional, minimal permissions required from user to use this command
    DynamicPerms: func(ctx) - optional, return True if user is allowed to use given command in given context

DynamicPerms can be used to f.e. make sure that user is connected to voice chat when issuing command. For a user to be able to use the command, it must pass a permission check: Check = DynamicCheck and (StaticPerms or AdminRole)

In both Help and LongHelp keyword TRAIL can be used, it gets replaced with total trail of commands (in above example, TRAIL = zeke translate add)

Zeke provides GuildEnv for every guils (server) bot is connected to. Each GuildEnv has three Envs within it: Data, Settings, Temporary. Those are exactly the same, except Temporary is cleaned on reboot. All saved data is encrypted (check chapter Security).

Env provides three most imporant functions: Get(keyword), Set(keyword, value), AddDefault(keyword, value).

Within GuildEnv there's also dictionary with Envs for every member of guild. You can access those environments by using Database.GetUserEnv(GuildEnv, user_id)

Here's recipe for some things that can be done with Database object.

#Database.Default # type: GuildEnv
Database.Default.Settings.AddDefault("keyword", 0)
async def cmd(ctx, args, trail):
    local_env = Database.GetGuildEnv(ctx.guild.id) # type: GuildEnv
    #local_env.Data # type: Env
    #local_env.Settings # type: Env
    #local_env.Temporary # type: Env
    user_env = Database.GetUserEnv(local_env, ctx.author.id) # type: Env
    val = local_env.Settings.Get("keyword")
    local_env.Settings.Set("keyword", val + 1)

By default, Database uses Dropbox to store files, however it's designed to be compatible with abstract class Storage so if you implement all methods declared there, you can easily swap dropbox for something else - for example, local storage.

Very important limitation: all variables in Database must be pickle-able and contain no self-referencing loops.

There's custom module for environmental variables but do not use that. The whole idea is poor and worked out below expectations.

.env is loaded at startup of the engine so you can use os.getenv(key). Don't bother figuring out envvars.py part of the engine, it's really not worth it.

Status check is feature of engine that simplifies debugging of 3rd party integrations. Translator and language detection in tools.py are a good example here.

    async def translator_status(self):
        pl_text = "Dzisiaj jest piękny dzień. W dni takie jak te, dzieci twojego pokroju..."
        try:
            self.__rawTranslate(pl_text, 'pl', 'en')
            return ( "Translation integration", True, "Unknown" )
        except Exception as e:
            return ( "Translation integration", False, str(e) )
    async def detector_status(self):
        pl_text = "Dzisiaj jest piękny dzień. W dni takie jak te, dzieci twojego pokroju..."
        detect = True if self.DetectLanguage(pl_text) == 'pl' else False
        return ( "Detect language integration", detect, "Unknown" )
Triggers.Get("Status").Add(objectTranslateTools.translator_status)
Triggers.Get("Status").Add(objectTranslateTools.detector_status)

Status is a Trigger that receives no arguments async func(), but must return values: (name: string, isOk: bool, errMess: string). The last returned value - errMess - is used only if isOk = False.

Feature for those who need to save something on local machine of bot, make some operations on it, then discard.

import os
from temp import objectTempManager as TempManager
reference = TempManager.New()
dirpath = reference.Path()
os.makedirs(dirpath)
filepath1 = os.path.join(dirpath, "some_text.txt")
filepath2 = os.path.join(dirpath, "some_vid.mp4")

Code snippet above generates random, unique filepath within temporary directory and claims it until the variable (reference) it is stored in gets destroyed.

Random filepath has no extension. TempManager doesn't create ANY files, it only manages filenames. However if you save file to this location and then reference is destroyed, the file will be removed aswell. The same applies to directories.

You can use random filepath to store any single file you want (but you can't use extensions then) or create a directory and save multiple files inside. Once the reference is destroyed, whole directory will be purged.

First of all, I've no education in computer security. I've tried to add encryption to all long-term storage data, but i don't have expertise to verify if the approach taken is actually secure. That being said, let me explain what's in the code.

Database stores data in dictionary in form dict[hash(guild_id)] = GuildEnv(). Any request for environment of guild not present in this dictionary warrants a request for Storage to load data from remote storage. If there's no file with data for this guild, new enviroment (copy of Database.Default) is created and returned.

Data of guild is stored in file named hash(guild_id). hash (in all cases) means PBKDF2, which uses guild_id AND random salt (automatically generated and stored in .salt.dump 24-byte long string of random bytes). This ensures that, even if connection or the remote storage itself is compromised, you can't even correlate those files to particular discord guilds.

The data itself is compressed and then encrypted with AES algorithm using 24-bytes long key (also automatically generated and stored in .aeskey.dump). Encryption requires pickle-dumping of data, converting whole GuildEnv (except Temporary part) into binary string. SHA256 is used to compute hash of this binary string before encryption. Then, Initialisation Vector (IV, 16-bytes long) is randomly generated. Finally, binary string of data is encrypted into cipher. Tuple of all three (IV, cipher, hash) is then again pickle-dumped into binary string - which is returned.

Decryption pretty much reverses this process, pickle-loading, decrypting and verifying hash. Note that pickle-dumping and pickle-loading is considered INSECURE in python and may run any code on server's machine if your storage is hacked.

Zeke was designed to allow expansion without ANY change to engine code. All python file (*.py) from .features directory are automatically imported after the engine's setup. By using Triggers, Database, Parser and others you can achieve pretty much anything and if Zeke doesn't have support for particular thing you need, you can access DiscordBot (type discord.ext.commands.Bot) from discordbot.py and expand functionality as you want.

Features are loaded in unordered manner.

Examples of features are stored in sample_features. You must move them to .features if you want to test them out. Typically, you want .features to be another git repository (private one). You can "comment out" (disable) feature by preceeding its' name with comma (so levels.py is active, while .levels.py is disabled)

There is also important limitation - it's not always possible to remove feature from bot safely. If there's any custom class object within Database, then you CAN NOT remove feature of this class from bot, otherwise Database will become unusable. By custom class, i mean any class declared within features. If you want to purge feature out of your repository, you must first write code to remove instances of such classes from database. This limitation does not apply for built-in python types.