Get dynamic status type when generating plot "OBO Principle by status…

…" in analysis page (#118)

* move analysis notebook to py code first draft

* Define dynamic status type to generate dataframe

The code expected to have the four status types in the dashboard
results. This is not always the case, so the code was updated to check
for the available status types before reshaping the data.

* Script and jinja2 template to generate analysis.html

The script processes the dashboard results and prepares it for analysis.
It generates plots and tables for the analysis page. The script replaces
the dashboard_analysis.ipynb Jupyter notebook with some improvements.

* replaces rule to generate analysis.html with python script

The analysis.html page is now generated by a python script instead of
the jupyter notebook.

---------

Co-authored-by: Anita Caron <[email protected]>

Makefile

@@ -158,6 +158,8 @@ test:
 
 tr: util/create_report_html.py dashboard/bfo/robot_report.tsv dependencies/obo_context.jsonld util/templates/report.html.jinja2
 	python3 $^ "ROBOT Report - bfo" dashboard/bfo/robot_report.html $(REPORT_LENGTH_LIMIT)
-
-dashboard/analysis.html:
-	jupyter nbconvert dashboard_analysis.ipynb  --no-input --execute --to html --output $@
+
+.PRECIOUS: dashboard/analysis.html
+dashboard/analysis.html: util/dashboard_analysis_html.py util/templates/analysis.html.jinja2
+	python3 $< --dashboard-results $(DASHBOARD_RESULTS) --template util/templates/analysis.html.jinja2 --output $@
+

dashboard_analysis.ipynb

@@ -127,12 +127,14 @@
     "dfx.index\n",
     "dfx.reset_index(inplace=True)\n",
     "dfx.fillna(0,inplace=True)\n",
-    "dfx = pd.melt(dfx, id_vars='check', value_vars=['ERROR', 'INFO', 'PASS','WARN'])\n",
+    "errcats = ['ERROR', 'INFO', 'WARN', 'PASS']\n",
+    "value_vars = [status for status in errcats if status in dfx.columns]\n",
+    "dfx = pd.melt(dfx, id_vars='check', value_vars=value_vars)\n",
     "dfx['value']=dfx['value'].astype(int)\n",
     "dfx['status'] = dfx['status'].astype('category')\n",
     "errcats = ['PASS', 'INFO','WARN','ERROR']\n",
     "errcats.reverse()\n",
-    "dfx['status'] = dfx['status'].cat.reorder_categories(errcats)\n",
+    "dfx['status'] = dfx['status'].cat.reorder_categories(value_vars)\n",
     "dfx.sort_values(['check','status'], ascending=False, inplace=True)\n",
     "\n",
     "height = 300+(len(df_results_agg)*10)\n",
@@ -351,7 +353,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.8.3"
+   "version": "3.9.16"
   }
  },
  "nbformat": 4,