configuration system improvements (#143)

This close #139 and #138.

- environ variables are no longer available from the `Config` object.
- all configuration keys have been lower-cased
- new config value `extension` has been added, to select the loaded
extensions

.github/CONTRIBUTING.md

@@ -102,6 +102,54 @@ If you are unfamiliar with alembic, [`here is some information`](/alembic/README
 
 In order to run the bot without any issue, there are some prerequisites.
 
+First, a `.env` file with the following values:
+| Key                   | Requirement | Description                                                                    |
+|-----------------------|-------------|--------------------------------------------------------------------------------|
+| `POSTGRES_USER`       | Required    | Used to create the database                                                    |
+| `POSTGRES_PASSWORD`   | Required    | Used to create the database                                                    |
+| `POSTGRES_DB`         | Required    | Used to create the database                                                    |
+| `MYBOT_TOKEN`         | Required    | [Create a bot](https://discord.com/developers/applications) and copy the token |
+| `TOPGG_AUTH`          | Optional    | Used to sync top.gg                                                            |
+| `TOPGG_TOKEN`         | Optional    | Used to sync top.gg                                                            |
+| `MS_TRANSLATE_KEY`    | Optional    | Required if "microsoft" is set in `TRANSLATOR_SERVICES`                        |
+| `MS_TRANSLATE_REGION` | Optional    | Required if "microsoft" is set in `TRANSLATOR_SERVICES`                        |
+| `LOG_WEBHOOK_URL`     | Optional    | Used to send bot logs using a webhook                                          |
+
+
+Then, create a `config.toml` ([TOML](https://toml.io/en/)) with the following values:
+| Key                   | Description                                                                                                                       |
+|-----------------------|-----------------------------------------------------------------------------------------------------------------------------------|
+| `support_guild_id`    | The bot needs to be member and administrator of this guild                                                                        |
+| `bot_id`              | Used for top.gg (if enabled)                                                                                                      |
+| `bot_name`            | Used for the webhook logs                                                                                                         |
+| `owners_ids`          | Grant permissions to these users (e.g. eval command, extensions reloading...) reload...                                           |
+| `translator_services` | A list of translations services to enable. Names will be imported from [`cogs.translate.adapters`](/src/cogs/translate/adapters/) |
+| `extensions`          | A list of extensions to enable. Names will be imported from [`cogs`](/src/cogs/)                                                  |
+
+## Extra informations
+
+In the project structure, `main.py` serves as the entry point executed by Docker. It provides a compact CLI application with various options that can be used with pre-created shell files in the `bin/` directory.
+`mybot.py` is the base of MyBot, containing the `MyBot` class, instantiated once at launch and available in many places in the code.
+
+The `MyBot` class has some utility functions like `getch_user`, `getch_channel`, and `get_or_create_db`. Refer to their docstring for more information.
+
+The `core` directory contains internally used code for MyBot, while `cogs` contains the implementation of features exposed by MyBot. Additionally, `libraries` holds wrappers for external APIs and tools used by the project.
+
+### i18n
+
+All strings should be passed to the `_` function (available in module `core.i18n`), to have them being translated automatically, e.g. `_("Hello World")`. The function also supports format options like the `str.format()` function (e.g., `_("Hello {}", name)`).
+
+`_` allows `msgfmt` to extract the strings from the code automatically, but it will also ✨ magically ✨ translate the strings in the correct language by walking through the callstack to find an `Interaction` object (this is generally not recommended, but in this case it is justified in my opinion).  
+Consequently, using `_` outside a command callback will not retrieve the language. You should then specify it using `_locale` argument. Set `_locale` to `None` if the string should not be translated at this time in the execution.
+
+Additionally, `_` also accepts a `_l` parameter to set a maximum size, to avoid any bugs due to translations being too long.
+
+Strings in command parameters, descriptions, etc., should be surrounded by the `discord.app_commands.locale_str` (aliased as `__`), to make `discord.py` send the translations directly to Discord.
+
+## Configuration and environnement
+
+In order to run the bot without any issue, there are some prerequisites.
+
 First, a `.env` file with the following values:
 | Key                   | Requirement | Description                                                                    |
 |-----------------------|-------------|--------------------------------------------------------------------------------|

src/cogs/admin.py

@@ -17,7 +17,7 @@
 
 class Admin(ExtendedCog):  # TODO(airo.pi_): add checkers
     @app_commands.command()
-    @app_commands.guilds(config.SUPPORT_GUILD_ID)
+    @app_commands.guilds(config.support_guild_id)
     async def reload_extension(self, inter: Interaction, extension: str):
         await self.bot.reload_extension(extension)
         await inter.response.send_message(f"Extension [{extension}] reloaded successfully")
@@ -26,12 +26,12 @@ async def reload_extension(self, inter: Interaction, extension: str):
     async def extension_autocompleter(self, inter: Interaction, current: str) -> list[app_commands.Choice[str]]:
         return [
             app_commands.Choice(name=ext, value=f"cogs.{ext}")
-            for ext in self.bot.extensions_names
+            for ext in self.bot.config.extensions
             if ext.startswith(current)
         ]
 
     @app_commands.command()
-    @app_commands.guilds(config.SUPPORT_GUILD_ID)
+    @app_commands.guilds(config.support_guild_id)
     async def sync_tree(self, inter: Interaction):
         await inter.response.defer()
         await self.bot.sync_tree()

src/cogs/api.py

@@ -43,7 +43,7 @@ def __init__(self, bot: MyBot):
         self.app.add_routes(self.routes)
 
     async def cog_load(self):
-        if not config.EXPORT_MODE:
+        if not config.export_mode:
             self.bot.loop.create_task(self.start())
 
     async def start(self) -> None:

src/cogs/translate/__init__.py

@@ -194,7 +194,7 @@ def __init__(self, bot: MyBot):
         self.tmp_user_usage = TempUsage()
 
         self.translators: list[TranslatorAdapter] = []
-        for adapter in self.bot.config.TRANSLATOR_SERVICES.split(","):
+        for adapter in self.bot.config.translator_services:
             adapter_module = importlib.import_module(f".adapters.{adapter}", __name__)
             self.translators.append(adapter_module.get_translator()())
 
@@ -355,7 +355,7 @@ async def check_user_quotas(self, user_id: int, strategy: SendStrategy) -> bool:
             ui.Button(
                 style=discord.ButtonStyle.url,
                 label=_("Vote for the bot", _locale=None),
-                url=f"https://top.gg/bot/{self.bot.config.BOT_ID}/vote",
+                url=f"https://top.gg/bot/{self.bot.user.id}/vote",  # pyright: ignore[reportOptionalMemberAccess]
             )
         )
         await strategy(

src/cogs/translate/adapters/microsoft.py

@@ -1,8 +1,8 @@
+import os
 from collections.abc import Sequence
 
 from lingua import Language as LinguaLanguage, LanguageDetectorBuilder
 
-from core import config
 from libraries.microsoft_translation import MicrosoftTranslator
 
 from ..languages import Language, Languages, LanguagesEnum
@@ -31,9 +31,7 @@
 
 class Translator(TranslatorAdapter):
     def __init__(self) -> None:
-        if config.MS_TRANSLATE_KEY is None or config.MS_TRANSLATE_REGION is None:
-            raise ValueError("Missing Microsoft Translator configuration")
-        self.instance = MicrosoftTranslator(config.MS_TRANSLATE_KEY, config.MS_TRANSLATE_REGION)
+        self.instance = MicrosoftTranslator(os.environ["MS_TRANSLATE_KEY"], os.environ["MS_TRANSLATE_REGION"])
         self.detector = (
             LanguageDetectorBuilder.from_languages(*lingua_to_language.keys())
             .with_low_accuracy_mode()

src/core/_config.py

@@ -16,20 +16,28 @@
 
 
 class Config:
-    SUPPORT_GUILD_ID: int = 332209340780118016
-    BOT_ID: int = 500023552905314304  # this should be retrieved from bot.client.id, but anyway.
-    OWNERS_IDS: ClassVar[list[int]] = [341550709193441280, 329710312880340992]
-    POSTGRES_USER: str = "postgres"
-    POSTGRES_DB: str = "mybot"
-    POSTGRES_PASSWORD: str | None = None
-    EXPORT_MODE: bool = False
-    TOPGG_TOKEN: str | None = None
-    TOPGG_AUTH: str | None = None
-    MS_TRANSLATE_KEY: str | None = None
-    MS_TRANSLATE_REGION: str | None = None
-    # comma separated list of services to use for translation. Corresponding files should be in cogs/translate/adapters.
-    TRANSLATOR_SERVICES: str = "libretranslate"
-    LOG_WEBHOOK_URL: str | None = None
+    """Get any configuration information from here.
+
+    This class is a singleton. You can get the configurations info from `bot.config`, or import the instance `config`
+    from this module, or even use `Config()` as they are all the same instance.
+
+    To ensure the config keys are accessed after being defined, the `define_config` function should be called when the
+    config is ready to be used. This will set the `_defined` attribute to True, and any access to the config before this
+    will raise a warning.
+
+    The values assigned bellow are the default values, and can be overwritten by the `define_config` function.
+    Everything present in the `config.toml` file will be added to the config instance (even if it is not defined here).
+    But please make sure to define the config keys here, for autocompletion.
+
+    Refer to [`.github/CONTRIBUTING.md`](.github/CONTRIBUTING.md) for more information.
+    """
+
+    support_guild_id: int = 332209340780118016
+    owners_ids: ClassVar[list[int]] = [341550709193441280, 329710312880340992]
+    translator_services: ClassVar[list[str]] = ["libretranslate"]
+    extensions: ClassVar[list[str]] = []
+    bot_id: int | None = None
+    export_mode: bool = False
 
     _instance: ClassVar[Self] | None = None
     _defined: ClassVar[bool] = False
@@ -40,7 +48,7 @@ def __new__(cls, *args: Any, **kwargs: Any) -> Self:
         return cls._instance
 
     @classmethod
-    def define(cls):
+    def set_as_defined(cls):
         Config._defined = True
 
     def __init__(self, **kwargs: Any):
@@ -66,7 +74,7 @@ def define_config(config_path: Path | str | None = None, **kwargs: Any):
             kwargs |= tomllib.load(f.buffer)
 
     Config(**kwargs)  # it is a singleton, so it will directly affect the instance.
-    Config.define()
+    Config.set_as_defined()
 
 
 config = Config()

src/core/_logger.py

@@ -105,10 +105,10 @@ def emit(self, record: logging.LogRecord):
         if getattr(record, "ignore_discord", False):
             return
 
-        if config.LOG_WEBHOOK_URL is None:
+        if os.getenv("LOG_WEBHOOK_URL") is None:
             return
 
-        self.send_to_discord(record, config.LOG_WEBHOOK_URL)
+        self.send_to_discord(record, os.environ["LOG_WEBHOOK_URL"])
 
 
 class _ColorFormatter(logging.Formatter):

src/core/checkers/base.py

@@ -97,4 +97,4 @@ def inner(user_id: int) -> bool:
     return inner
 
 
-is_me_bool = allowed_users_bool(*config.OWNERS_IDS)
+is_me_bool = allowed_users_bool(*config.owners_ids)

src/main.py

@@ -3,11 +3,11 @@
 import os
 from os import environ
 from pathlib import Path
-from typing import Annotated, Any
+from typing import Annotated
 
 import typer
 
-from core._config import define_config
+from core._config import config, define_config
 from core._logger import create_logger
 from features_exporter import features_exporter
 
@@ -35,30 +35,23 @@ def run(
         typer.Option("--sync", help="Sync slash command with Discord."),
     ] = False,
 ):
+    define_config(config_path)
     required_env_var = {"POSTGRES_USER", "POSTGRES_PASSWORD", "POSTGRES_DB", "MYBOT_TOKEN"}
-    optional_env_var = {
-        "TOPGG_TOKEN",
-        "TOPGG_AUTH",
-        "MS_TRANSLATE_KEY",
-        "MS_TRANSLATE_REGION",
-        "TRANSLATOR_SERVICES",
-        "LOG_WEBHOOK_URL",
-    }
-    kwargs: dict[str, Any] = {}
 
     if missing_env_var := required_env_var - set(os.environ):
-        raise RuntimeError(f"The following environment variables are missing: {", ".join(missing_env_var)}")
-
+        logger.critical("The following environment variables are missing: {}", ", ".join(missing_env_var))
+        raise SystemExit(1)
     if len({"MS_TRANSLATE_KEY", "MS_TRANSLATE_REGION"} & set(os.environ)) == 1:
-        raise RuntimeError("MS_TRANSLATE_KEY and MS_TRANSLATE_REGION should be either both defined or both undefined.")
+        logger.critical("MS_TRANSLATE_KEY and MS_TRANSLATE_REGION should be either both defined or both undefined.")
+        raise SystemExit(1)
     if len({"TOPGG_TOKEN", "TOPGG_AUTH"} & set(os.environ)) == 1:
-        raise RuntimeError("TOPGG_TOKEN and TOPGG_AUTH should be either both defined or both undefined.")
-
-    present_env_var = set(os.environ) & (required_env_var | optional_env_var) - {"MYBOT_TOKEN"}
-    for env_var in present_env_var:
-        kwargs[env_var] = os.environ[env_var]
-
-    define_config(config_path, **kwargs)
+        logger.critical("TOPGG_TOKEN and TOPGG_AUTH should be either both defined or both undefined.")
+        raise SystemExit(1)
+    elif "TOPGG_TOKEN" in os.environ and config.bot_id is None:
+        logger.critical(
+            "Due to a limitation, you have to manually specify the bot_id in the configuration when using top.gg."
+        )
+        raise SystemExit(1)
 
     from mybot import MyBot  # MyBot is imported after the config is defined.
 
@@ -71,7 +64,7 @@ def run(
 def export_features(
     filename: Annotated[Path, typer.Argument(help="The json filename for the output.")] = Path("./features.json"),
 ):
-    define_config(EXPORT_MODE=True)
+    define_config(export_mode=True)
     asyncio.run(features_exporter(filename=filename))
 
 

src/mybot.py

@@ -1,8 +1,8 @@
 from __future__ import annotations
 
 import logging
+import os
 import re
-import sys
 from typing import TYPE_CHECKING, Any, cast
 
 import discord
@@ -50,14 +50,14 @@ def __init__(self, startup_sync: bool = False) -> None:
         self._invite: discord.Invite | None = None
 
         self.error_handler = ErrorHandler(self)
-        if config.TOPGG_TOKEN is not None:
-            self.topgg = topggpy.DBLClient(config.TOPGG_TOKEN, default_bot_id=config.BOT_ID)
+        if os.getenv("TOPGG_TOKEN") is not None:
+            self.topgg = topggpy.DBLClient(os.environ["TOPGG_TOKEN"], default_bot_id=config.bot_id)
             self.topgg_webhook_manager = topggpy.WebhookManager()
             (
                 self.topgg_webhook_manager.endpoint()
                 .route("/topgg_vote")
                 .type(topggpy.WebhookType.BOT)
-                .auth(config.TOPGG_AUTH or "")
+                .auth(os.environ["TOPGG_AUTH"] or "")
                 .callback(self.topgg_endpoint)
                 .add_to_manager()
             )
@@ -82,21 +82,6 @@ def __init__(self, startup_sync: bool = False) -> None:
         )
 
         # Keep an alphabetic order, it is more clear.
-        self.extensions_names: list[str] = [
-            "admin",
-            # "api",
-            # "calculator",
-            "clear",
-            "config",
-            # "game",
-            "help",
-            "poll",
-            # "ping",
-            # "restore",
-            "stats",
-            "eval",
-            "translate",
-        ]
         self.config = config
         self.app_commands = []
 
@@ -139,12 +124,8 @@ async def get_topgg_vote(self, user_id: int) -> bool:
         return self.topgg_current_votes.get(user_id, False)
 
     async def connect_db(self):
-        if config.POSTGRES_PASSWORD is None:
-            logger.critical("Missing environment variable POSTGRES_PASSWORD.")
-            sys.exit(1)
-
         self.db_engine = create_async_engine(
-            f"postgresql+asyncpg://{config.POSTGRES_USER}:{config.POSTGRES_PASSWORD}@database:5432/{config.POSTGRES_DB}"
+            f"postgresql+asyncpg://{os.environ["POSTGRES_USER"]}:{os.environ["POSTGRES_PASSWORD"]}@database:5432/{os.environ["POSTGRES_DB"]}"
         )
         self.async_session = async_sessionmaker(self.db_engine, expire_on_commit=False)
 
@@ -162,10 +143,13 @@ async def on_ready(self) -> None:
         activity = discord.Game("WIP!")
         await self.change_presence(status=discord.Status.online, activity=activity)
 
-        tmp = self.get_guild(self.config.SUPPORT_GUILD_ID)
+        tmp = self.get_guild(self.config.support_guild_id)
         if not tmp:
-            logger.critical("Support server cannot be retrieved")
-            sys.exit(1)
+            logger.critical("Support server cannot be retrieved. Set the correct ID in the configuration file.")
+            raise SystemExit(1)
+        if tmp.me.guild_permissions.administrator is False:
+            logger.critical("MyBot doesn't have the administrator permission in the support server.")
+            raise SystemExit(1)
         self.support = tmp
         await self.support_invite  # load the invite
 
@@ -247,7 +231,7 @@ async def on_message(self, message: discord.Message) -> None:
                 label="Invite link",
                 style=discord.ButtonStyle.url,
                 emoji="🔗",
-                url=f"https://discord.com/api/oauth2/authorize?client_id={config.BOT_ID}&scope=bot%20applications.commands",  # NOSONAR noqa: E501
+                url=f"https://discord.com/api/oauth2/authorize?client_id={self.user.id}&scope=bot%20applications.commands",  # NOSONAR noqa: E501
             )
         )
         view.add_item(
@@ -274,7 +258,7 @@ async def support_invite(self) -> discord.Invite:
         return self._invite
 
     async def load_extensions(self) -> None:
-        for ext in self.extensions_names:
+        for ext in self.config.extensions:
             if not ext.startswith("cogs."):
                 ext = "cogs." + ext