Cleanup the codebase, add documentation, explanations, etc... (#135)

This pull request updates and upgrades the entire project from both a
documentation and codebase perspective. Its aim is to enhance the
project's contribution-friendliness by removing unnecessary elements,
renaming components in a more convenient manner, and adding extensive
general explanations.

Here's a non-exhaustive list of changes:

- Removed the "beta"-tagged compose file. This can now be manually
adjusted during bot deployment, eliminating the need for a separate
file.
- Replaced pylint (which wasn't enabled anyway), isort, bandit, and
black with ruff. Ruff offers significant speed improvements and
additional functionalities.
- Ensured compliance with new rules specified in pyproject.toml
throughout the entire codebase.
- Implemented pip-tools to streamline dependency management.
- Updated versions of tools used in Github Actions.
- Added useful information in CONTRIBUTING.md
- And more...

.github/CONTRIBUTING.md

@@ -1,33 +1,122 @@
+# Contribution Guidelines
+
 Feel free to contribute to MyBot.
-There is mainly three ways to contribute :
- 1.  Source code contributions
- 2.  Translations contributions
- 3.  Financial contributions
+There is mainly three ways to contribute:
+
+1.  Source code contributions
+2.  Translations contributions
+3.  Financial contributions
 
 ## Source code contributions
 
-If you have any suggestion for the bot, and you think you have the ability to do it yourself, start by contacting me through Discord!
-It would be a pleasure to count you from the Mybot's contibutors!
+If you have any suggestions for the bot and you think you have the ability to do it yourself, consider contacting me through [Discord](https://discord.gg/GRsAy4aUgu)!
+It would be a pleasure to count you to the Mybot's contributors!
 Then, fork this project, make the changes you want, and open a Pull Request on `master`.
-Before open a Pull Request, you can start by running `tox`. This will ensure your code respect the style, etc...
+
+## Good practices and requirements
+
+### Dependencies
+
+The project is using [pip-tools](https://github.com/jazzband/pip-tools) to manage its dependencies. The requirements are declared in [requirements.in](/requirements.in) and developers requirements are in [requirements.dev.in](/requirements.dev.in).
+
+Install the dependencies using `pip-sync` or simply `pip install -r requirements.txt`, as well as the developer dependencies with `pip install -r requirements.dev.txt`.
+
+If you add or change dependencies, edit the corresponding `.in` file, then use `pip-compile` (`pip-compile requirements.dev.in` for developer deps).
+
+### Lint, formatting...
+
+The project use [pyright](https://github.com/microsoft/pyright) for static type checking and [ruff](https://github.com/astral-sh/ruff) for general formatting, import sorting, security scans and static code analyses.
+
+Please use these tool to avoid Github Actions failure. [tox](https://github.com/tox-dev/tox) can be used to run every checks before committing by running `tox`.
+
+## Docker watch
+
+The [compose file](/compose.yml) implements [watch](https://docs.docker.com/compose/file-watch/) to help debugging the code. By running `docker compose watch`, the bot will be executed while observing for files changes or deps updates. This will allow extensions reload, and speed up restarts (it avoids rebuilds).
+
+## Run the code on debug version
+
+You can use [debugpy](https://github.com/microsoft/debugpy) easily when running MyBot locally, for dev and debug purposes.  
+Replace `docker compose` with `docker compose -f compose.yml -f compose.debug.yml` when using Docker Compose. You can then use debugpy with the port `5678`.
+
+Additionally, this will setup `config.DEBUG` to `True` from the code perspective, which will also set the logger level to `DEBUG`, and expose the PostgresSQL port to the host (`5432`).
+
+### VSCode debug config
+
+As an example, here is a json configuration that can be added inside your local `.vscode/launch.py` to use the integrated debugger:
+```json
+{
+    "name": "debug",
+    "type": "debugpy",
+    "request": "attach",
+    "connect": {
+        "host": "localhost",
+        "port": 5678
+    },
+    "pathMappings": [
+        {
+            "localRoot": "${workspaceFolder}/src",
+            "remoteRoot": "/app"
+        }
+    ]
+}
+```
+
+After you run the code in debug mode, click on the "play" icon inside VSCode to attach the debug console. You can then use breakpoints, etc.  
+To make the restart button actually restart the bot and not just re-attach the debugger, you can add pre&post tasks:
+```json
+    ...
+    "preLaunchTask": "bot up",
+    "postDebugTask": "bot restart",
+```
+And in `.vscode/tasks.json`, add the tasks:
+```json
+{
+    "label": "bot up",
+    "type": "shell",
+    "presentation": {
+        "reveal": "silent"
+    },
+    "command": "docker compose -f compose.yml -f compose.debug.yml up -d",
+},
+{
+    "label": "bot restart",
+    "type": "shell",
+    "presentation": {
+        "reveal": "silent"
+    },
+    "command": "docker-compose restart mybot"
+}
+```
+
+If the bot is executed with the up task, you should then use the `watch` command with `--no-up`.
+More information here: https://code.visualstudio.com/docs/python/debugging
+
+## Database revisions
+
+The project use [alembic](https://github.com/sqlalchemy/alembic) to manage database revisions.  
+When the bot is started using Docker, `alembic upgrade head` is [automatically executed](https://github.com/mybot-organization/mybot/blob/cleanup/Dockerfile#L30).
+To create revisions, you can use [`alembic.sh`](bin/alembic.sh) in the `bin` directory. This script allow you to use the alembic CLI inside the container, and will mount the [`/alembic`](/alembic/) directory.
+
+If you are unfamiliar with alembic, [`here is some information`](/alembic/README). Check also the [documentation](https://alembic.sqlalchemy.org/en/latest/tutorial.html#create-a-migration-script) as well.
 
 ## Translations contributions
 
-MyBot is a multi-language bot! The codebase is in english, which is then translated in several languages.
+MyBot is a multi-language bot! The codebase is in English, which is then translated in several languages.
 
 Currently, MyBot is translated in :
-  -  French
+
+- French
 
 If you know one of these languages, you can contribute to translations here:
 https://crowdin.com/project/mybot-discord
 
-If you are able to add a new language to the bot, please contact me on Discord! It would be a pleasure to add a new language to MyBot!
+If you want to add a new language to the bot, please contact me on Discord! It would be a pleasure to add a new language to MyBot!
 However, this language must also be available on the Discord application.
 
-## Financial contributions 
+## Financial contributions
 
-You can make voluntary donation at https://www.buymeacoffee.com/airopi
+You can make voluntary donations at https://www.buymeacoffee.com/airopi
 
-----
+---
 
 Thanks !

Dockerfile

@@ -11,11 +11,13 @@ RUN --mount=type=cache,target=/var/cache/apk/ \
     --mount=type=bind,source=./bin/msgfmt.py,target=./msgfmt.py \
     : \
     && apk add gcc musl-dev linux-headers \
+    && pip install -U pip \
     && pip install -U -r requirements.txt \
     && python ./msgfmt.py ./locale/**/LC_MESSAGES/*.po \
     && :
 
 FROM python:3.12.0-alpine as base
+# https://docs.docker.com/reference/dockerfile/#copy---parents
 COPY --parents --from=build /opt/venv /app/locale/**/LC_MESSAGES/*.mo /
 WORKDIR /app
 COPY ./src ./
@@ -24,7 +26,7 @@ ENV PATH="/opt/venv/bin:$PATH"
 ENV PYTHONUNBUFFERED=0
 
 
-FROM base as prod
+FROM base as production
 CMD ["/bin/sh", "-c", "alembic upgrade head && python ./main.py bot --sync -c ./config.toml"]
 
 

bin/pot-generation.sh

@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 pybabel extract \
-    --msgid-bugs-address="[email protected]" \
+    --msgid-bugs-address="[email protected]" \
     --project="MyBot" \
     --version="1.0" \
     -k "_ __" \

compose.debug.yml

@@ -0,0 +1,13 @@
+version: '3.4'
+
+services:
+  mybot:
+    build:
+      target: debug
+    restart: "no"
+    ports:
+      - 5678:5678  # for debugpy; see .github/CONTRIBUTING/md
+
+  database:
+    ports:
+      - 5432:5432  # publish database port to the host

docker-compose.yml → compose.yml

@@ -4,14 +4,15 @@ services:
   mybot:
     image: airopi/mybot:stable
     build:
-      context: .
       dockerfile: ./Dockerfile
-      target: prod
+      target: production
     develop:
       watch:
         - action: sync
           path: ./src
           target: /app
+        - action: rebuild
+          path: ./requirements.txt
     env_file:
       - .env
     tty: true

pyproject.toml

@@ -1,16 +1,3 @@
-[tool.bandit]
-skips = ["B101"]
-
-
-[tool.black]
-line-length = 120
-
-[tool.isort]
-profile = "black"
-combine_as_imports = true
-combine_star = true
-line_length = 120
-
 [tool.tox]
 legacy_tox_ini = """
 [tox]
@@ -19,50 +6,79 @@ skipsdist = true
 [testenv]
 deps = 
     -r requirements.txt
-    -r requirements-dev.txt
+    -r requirements.dev.txt
 commands =
     # pytest
-    black --check src/
-    bandit -r src/ tests/ -c pyproject.toml
-    isort ./src/ --check
+    ruff format --check src
+    ruff check src
     pyright src/
 """
 
-[tool.pylint]
-max-line-length = 120
-allow-reexport-from-package = true
-ignore-patterns = [".*.pyi"]
-jobs = 4
-unsafe-load-any-extension = false
-disable = [
-    "missing-module-docstring",
-    "missing-class-docstring",
-    "missing-function-docstring",
-    "abstract-method",
-    # "arguments-differ",
-    # "attribute-defined-outside-init",
-    # "duplicate-code",
-    # "eq-without-hash",
-    # "fixme",
-    # "global-statement",
-    # "implicit-str-concat",
-    # "import-error",
-    # "import-self",
-    # "import-star-module-level",
-    # "inconsistent-return-statements",
-    # "invalid-str-codec",
+[tool.ruff]
+line-length = 120
+indent-width = 4
+target-version = "py312"
+src = ["src"]
+exclude = ["bin/msgfmt.py", "alembic/**"]
+
+[tool.ruff.lint]
+select = [
+    "E",     # pycodestyle Error
+    "F",     # pyflakes
+    "UP",    # pyupgrade
+    "SIM",   # flake8-simplify
+    "I",     # imports
+    "S",     # bandit (security)
+    "N",     # pep8-naming
+    "ASYNC", # flake8-async
+    "C4",    # flake8-compehensions
+    "FA",    # flake8-future-annotations
+    "ISC",   # flake8-implicit-str-concat
+    "ICN",   # flake8-import-conventions
+    "G",     # flake8-logging-format
+    "PIE",   # flake8-pie
+    "PYI",   # flake8-pyi
+    "RSE",   # flake8-raise
+    "SLOT",  # flake8-slots
+    "INT",   # flake8-gettext
+    "TRY",   # tryceratops
+    "FLY",   # flynt
+    "PERF",  # Perflint
+    "FURB",  # refurb
+    "LOG",   # flake8-logging
+    "RUF",   # Ruff-specific-rules
+    # "ERA",   # locate commented codes
+    # "FIX",   # locate TODOs and FIXME
+    # "PTH",   # flake8-use-pathlib (maybe todo ?)
+    # "TID",   # flake8-tidy-imports (maybe todo ?)
+    # "SLF",   # flake8-self  (managed by pyright)
+    # "RET",   # flake8-return
+    # "Q",     # flake8-quotes
+    # "T20",   # flake8-print
+    # "DTZ",   # flake8-datetimez (TODO)
+    # "B",     # flake8-bugbear
+]
+
+ignore = [
+    "E501",   # line too long (we relate on the formater)
+    "N818",   # Error suffix for exceptions names
+    "PIE796", # Enum contains duplicate value
+    "TRY003", # Avoid specifying long messages outsides the exception class
+    "ISC001", # To avoid conflicts with the formatter
 ]
-argument-rgx = "^[a-z_][a-z0-9_]{0,30}$"
-attr-rgx = "^[a-z_][a-z0-9_]{0,30}$"
-function-rgx = "^[a-z_][a-z0-9_]{0,30}$"
-method-rgx = '[a-z_][a-z0-9_]{0,30}$'
-variable-rgx = '[a-z_][a-z0-9_]{0,30}$'
-ignore-long-lines = '''(?x)(
-^\s*(\#\ )?<?https?://\S+>?$|
-^\s*(from\s+\S+\s+)?import\s+.+$)'''
-indent-string = '    '
-dummy-variables-rgx = '^\*{0,2}(_$|unused_|dummy_)'
-callbacks = ['cb_', '_cb']
+dummy-variable-rgx = '^\*{0,2}(_$|__$|unused_|dummy_)'
+
+[tool.ruff.lint.pyflakes]
+extend-generics = ["core.Menu"]
+
+[tool.ruff.format]
+quote-style = "double"
+indent-style = "space"
+skip-magic-trailing-comma = false
+
+[tool.ruff.lint.isort]
+combine-as-imports = true
 
-# [tool.pyright]
-# reportImportCycles = false
+[tool.setuptools.dynamic]
+dependencies = { file = ["requirements.in"] }
+optional-dependencies.dev = { file = ["requirements.dev.in"] }

requirements.dev.in

@@ -0,0 +1,5 @@
+tox
+pyright
+pip-tools
+debugpy
+ruff