Merge branch 'main' of https://github.com/galaxyproject/galaxy_codex …

…into pr-post-lab-link

.github/workflows/lint_and_unittest.yaml

@@ -0,0 +1,42 @@
+name: Code best practice and unittesting
+
+on:
+  pull_request:
+    branches:
+      - main
+    paths:
+      - 'bin/**'  # This will trigger the workflow only if files in the 'bin' folder are modified.
+
+concurrency:
+  group: lint-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  lint:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v4
+    - uses: actions/setup-python@v5
+      with:
+          python-version: '3.11'
+    - name: Install tox
+      run: python -m pip install 'tox>=1.8.0'
+    - name: Lint
+      run: tox -e lint
+
+  unittest:
+    runs-on: ubuntu-20.04
+    strategy:
+      matrix:
+        python-version: ['3.11']
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Install requirements
+        run: python -m pip install -r requirements.txt
+      - name: Run unittests
+        run: |
+          PYTHONPATH=bin python -m unittest discover -s bin/tests

.github/workflows/run_tests.yaml

@@ -1,9 +1,47 @@
 name: Run tests
-on: [push, pull_request]
+
+# This GitHub Actions workflow is designed to automate testing for the project.
+# It triggers under two conditions:
+# 1. Pull requests targeting the main branch, ensuring that internal changes
+#    are validated before merging.
+# 2. Pushes to the dev branch, allowing for continuous integration
+#    as changes are made directly to the development branch.
+#
+# The workflow consists of three main jobs:
+# - **test-tools**: Runs tests for tools extracted from the repository.
+# - **test-tutorials**: Runs tests for community tutorials.
+# - **test-workflows**: Runs tests for workflow scripts.
+#
+# Each job includes the following steps:
+# - Checkout the repository code.
+# - Set up the Python environment using the specified version.
+# - Install required dependencies.
+# - Execute scripts for extraction and filtering of tools, tutorials, or workflows.
+#
+# This ensures that the code remains reliable and functional as new changes are introduced.
+
+on:
+  pull_request:
+    branches:
+      - main  # Trigger on pull requests targeting the main branch
+  push:
+    branches:
+      - dev  # Trigger on pushes to the dev branch
 
 jobs:
   test-tools:
     runs-on: ubuntu-20.04
+    # This job runs tests for tools.
+    # It checks for internal pull requests targeting the main branch,
+    # as well as pushes to the dev branch.
+    # The workflow performs the following steps:
+    # 1. Checkout the repository code.
+    # 2. Set up Python environment using the specified version.
+    # 3. Install required Python packages.
+    # 4. Extract tools using a provided script.
+    # 5. Filter community tools using a provided script.
+    # 6. Format tools into an interactive table and generate a word cloud.
+    if: github.event.pull_request.head.repo.full_name == github.repository || github.ref == 'refs/heads/dev'
     strategy:
       matrix:
         python-version: ['3.11']
@@ -14,7 +52,7 @@ jobs:
       - uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python-version }}
-      - name: Install requirement
+      - name: Install requirements
         run: python -m pip install -r requirements.txt
       - name: Tool extraction
         run: | 
@@ -26,9 +64,20 @@ jobs:
           bash sources/bin/get_community_tools.sh test
       - name: Create interactive table and wordcloud
         run: | 
-          bash sources/bin/format_tools.sh
+          bash sources/bin/format_tools.sh test
+
   test-tutorials:
     runs-on: ubuntu-20.04
+    # This job runs tests for tutorials.
+    # Similar to the tools job, it checks for internal pull requests
+    # to the main branch and pushes to the dev branch.
+    # The workflow performs the following steps:
+    # 1. Checkout the repository code.
+    # 2. Set up Python environment using the specified version.
+    # 3. Install required Python packages.
+    # 4. Extract tutorials using a provided script.
+    # 5. Filter community tutorials using a provided script.
+    if: github.event.pull_request.head.repo.full_name == github.repository || github.ref == 'refs/heads/dev'
     strategy:
       matrix:
         python-version: ['3.11']
@@ -38,7 +87,7 @@ jobs:
       - uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python-version }}
-      - name: Install requirement
+      - name: Install requirements
         run: python -m pip install -r requirements.txt
       - name: Tutorial extraction
         run: | 
@@ -48,8 +97,13 @@ jobs:
       - name: Tutorial filtering
         run: | 
           bash sources/bin/get_community_tutorials.sh test
+
   test-workflows:
     runs-on: ubuntu-20.04
+    # This job runs tests for workflows.
+    # It performs the same checks as the other jobs, ensuring
+    # quality for workflow scripts.
+    if: github.event.pull_request.head.repo.full_name == github.repository || github.ref == 'refs/heads/dev'
     strategy:
       matrix:
         python-version: ['3.11']
@@ -59,7 +113,7 @@ jobs:
       - uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python-version }}
-      - name: Install requirement
+      - name: Install requirements
         run: python -m pip install -r requirements.txt
       - name: Workflow extraction
         run: | 

README.md

@@ -1,9 +1,64 @@
 Galaxy Codex
 ============
 
-Galaxy Communities Dock aka Galaxy Codex is a catalog of Galaxy resources (tools, training, workflows) that can be filtered for any community.
+Galaxy Communities Dock (Galaxy CoDex) is a catalog of resources for Galaxy communities.
 
-This repository stores the sources to build this catalog. The catalog is automatically updated every week.
+Currently, we have two main resources:
 
+1. **Galaxy Community Catalog**: from tools, training & workflows
+2. **Galaxy Labs**: (led by Galaxy Australia!) main and tool panel content for communities to spin up subdomain pages
 
-Any Galaxy community can be added to this project and benefit from the dedicated resources, including interactive tables that can be embedded into subdomains and website via an iframe. **Learn [how to add your community](https://training.galaxyproject.org/training-material//topics/dev/tutorials/community-tool-table/tutorial.html) in the dedicated GTN tutorial**. 
+This repository stores the sources to build all of this content. The catalog is automatically updated every week.
+
+# Join the CoDex
+Any Galaxy Community can be added to this project and benefit from the dedicated resources.
+**Learn [how to add your community](https://training.galaxyproject.org/training-material/topics/community/faqs/codex.html)** in the dedicated GTN tutorial.
+
+# Galaxy Community Catalog
+To generate interactive tables that can be embedded into Galaxy Labs (subdomains) and websites via an iframe. 
+**Learn [how to generate the Galaxy Community Catalog](https://training.galaxyproject.org/training-material//topics/dev/tutorials/community-tool-table/tutorial.html)** in the dedicated GTN tutorial. 
+
+# Galaxy Lab
+To generate GalaxyLab content, you can follow our documentation (to be added!).
+To spin up a GalaxyLab on your server, you can follow our documentation (to be added!).
+
+# Galaxy Community Catalog content
+
+## Tool table
+
+Column | Description
+--- | ---
+Suite ID | ID of Galaxy suite
+Tool IDs | List of Galaxy tool IDs
+Description | Description of the suite
+Homepage | Homepage for tool
+Suite version | Version of the Galaxy suite
+Suite Conda package | Conda package used as requirement for the suite 
+Latest suite conda package version | Latest Conda package version on anaconda
+Suite version status | Update status derived by comparing the suite version to the latest conda package version
+ToolShed categories | 
+EDAM operations | EDAM operations extracted using bio.tools
+EDAM reduced operations | EDAM operations where only the most specific terms are kept, i.e. all terms that are superclasses of other terms are removed
+EDAM topics | EDAM topics extracted using bio.tools
+EDAM reduced topics | EDAM topics where only the most specific terms are kept, i.e. all terms that are superclasses of other terms are removed
+Suite owner | Owner of the Galaxy suite
+Suite source | Path to the Galaxy suite
+bio.tools ID | ID of the bio.tools entry corresponding to the tool
+bio.tools name | Name of the bio.tools entry corresponding to the tool
+bio.tools description | Description in bio.tools entry corresponding to the tool
+bii ID | ID of BII entry corresponding to the tool
+Number of tools available on ...  | Number of tools available on given Galaxy server
+Suite users (last 5 years) on ...  | Number of users of the suite in the last 5 years on given Galaxy server
+Suite users on ...  | Number of users of the suite on given Galaxy server
+Suite runs (last 5 years) on ...  |  Number of runs of the suite tools in the last 5 years on given Galaxy server
+Suite runs on ...  | Number of runs of the suite tools on given Galaxy server
+Suite users (last 5 years) on main servers  |  Number of users of the suite in the last 5 years on all UseGalaxy servers
+Suite users on main servers  | Number of users of the suite on all UseGalaxy servers
+Suite runs (last 5 years) on main servers  | Number of runs of the suite tools in the last 5 years on all UseGalaxy servers
+Suite runs on main servers  | Number of runs of the suite tools on all UseGalaxy servers
+Deprecated | Deprecation status after review by a domain expert
+To keep | Status to add to a community after review by a domain expert
+
+# Vocabulary
+**Galaxy Lab**: Formerly known as Subdomains, Galaxy Australia built a new method for generating the main and tool panels for a Galaxy subdomain, known as a Galaxy Lab.
+**Galaxy Community**: Galaxy Community of Practice. You can see a full list of Special Interest Groups in the [SIG Directory](https://galaxyproject.org/community/sig).