Merge pull request #12 from bebatut/exclude-curl

Modify one argument, add curl for microgalaxy files in bash script

README.md

@@ -51,26 +51,10 @@ Galaxy Tool extractor
         --api <GitHub API key> \
         --output <Path to output file> \
         [--categories <Path to ToolShed category file>] \
-        [--excluded <Path to excluded tool file category file>]\
+        [--exclude <Path to excluded tool file category file>]\
         [--keep <Path to to-keep tool file category file>]
     ```
 
-    For microGalaxy, a Bash script in `bin` can used by:
-    1. Exporting the GitHub API key as an environment variable:
-
-        ```
-        $ export GITHUB_API_KEY=<your GitHub API key>
-        ```
-
-    2. Running the script
-
-        ```
-        $ bash bin/extract_microgalaxy_tools.sh
-        ```
-
-        It will take the files in the `data/microgalaxy` folder and export the tools into `microgalaxy_tools.csv`
-
-
 The script will generate a CSV file with each tool found in the list of GitHub repository and several information for these tools:
 
 1. Galaxy wrapper id
@@ -90,4 +74,26 @@ The script will generate a CSV file with each tool found in the list of GitHub r
 15. Conda id
 16. Conda version
 17. Reviewed
-18. To keep
+18. To keep
+
+## For microbial related tools
+
+For microGalaxy, a Bash script in `bin` can used by:
+
+1. Exporting the GitHub API key as an environment variable:
+
+    ```
+    $ export GITHUB_API_KEY=<your GitHub API key>
+    ```
+
+2. Running the script
+
+    ```
+    $ bash bin/extract_microgalaxy_tools.sh
+    ```
+
+    It will:
+    1. Update the files in the `data/microgalaxy` folder
+    2. Export the tools into `microgalaxy_tools.csv`
+
+

bin/extract_galaxy_tools.py

@@ -358,7 +358,7 @@ def export_tools(tools, output_fp):
     parser.add_argument('--api', '-a', required=True, help="GitHub access token")
     parser.add_argument('--output', '-o', required=True, help="Output filepath")
     parser.add_argument('--categories', '-c', help="Path to a file with ToolShed category to keep in the extraction (one per line)")
-    parser.add_argument('--excluded', '-e', help="Path to a file with ToolShed ids of tools to exclude (one per line)")
+    parser.add_argument('--exclude', '-e', help="Path to a file with ToolShed ids of tools to exclude (one per line)")
     parser.add_argument('--keep', '-ek', help="Path to a file with ToolShed ids of tools to keep (one per line)")
     args = parser.parse_args()
 
@@ -369,7 +369,7 @@ def export_tools(tools, output_fp):
 
     # get categories and tools to exclude
     categories = read_file(args.categories)
-    excl_tools = read_file(args.excluded)
+    excl_tools = read_file(args.exclude)
     keep_tools = read_file(args.keep)
 
     # parse tools in GitHub repositories to extract metada, filter by TS categories and export to output file

bin/extract_microgalaxy_tools.sh

@@ -1,8 +1,18 @@
 #!/usr/bin/env bash
 
+curl \
+        -L \
+        "https://docs.google.com/spreadsheets/d/1Nq_g-CPc8t_eC4M1NAS9XFJDflA7yE3b9hfSg3zu9L4/export?format=tsv&gid=1533244711" \
+        -o "data/microgalaxy/tools_to_keep"
+
+curl \
+        -L \
+        "https://docs.google.com/spreadsheets/d/1Nq_g-CPc8t_eC4M1NAS9XFJDflA7yE3b9hfSg3zu9L4/export?format=tsv&gid=672552331" \
+        -o "data/microgalaxy/tools_to_exclude"
+
 python bin/extract_galaxy_tools.py \
         --api $GITHUB_API_KEY \
         --output microgalaxy_tools.csv \
         --categories "data/microgalaxy/categories" \
-        --excluded "data/microgalaxy/tools_to_exclude" \
+        --exclude "data/microgalaxy/tools_to_exclude" \
         --keep "data/microgalaxy/tools_to_keep"

data/microgalaxy/tools_to_exclude

@@ -345,6 +345,7 @@ transit
 trinity
 trinotate
 trycycler
+tsebra
 ucsc_blat
 umi_tools
 verkko

data/microgalaxy/tools_to_keep

@@ -128,6 +128,7 @@ krocus
 legsta
 lineagespot
 lorikeet
+m6anet
 maaslin2
 maker
 mapseq