-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
0 parents
commit c103353
Showing
88 changed files
with
7,485 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
# Sphinx build info version 1 | ||
# This file records the configuration used when building these files. When it is not found, a full rebuild will be done. | ||
config: 0ff19c6169af8b736c9cce6b548eb9af | ||
tags: 645f666f9bcd5a90fca523b33c5a78b7 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
# Sphinx build info version 1 | ||
# This file records the configuration used when building these files. When it is not found, a full rebuild will be done. | ||
config: fe7f2e826a7128f3d0dcd039f5e45d66 | ||
tags: 645f666f9bcd5a90fca523b33c5a78b7 |
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
# API Reference | ||
|
||
## example | ||
|
||
```{eval-rst} | ||
.. automodule:: example | ||
:members: | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,42 @@ | ||
.. Gesundai documentation master file, created by | ||
sphinx-quickstart on Fri Oct 18 13:25:00 2024. | ||
You can adapt this file completely to your liking, but it should at least | ||
contain the root `toctree` directive. | ||
GESUND AI 👋 | ||
----------------------- | ||
|
||
A CRO platform for clinical-grade AI | ||
Train. Validate. Secure clearance. | ||
Gesund.ai orchestrates the AI as-a-Medical Device lifecycle, providing privacy-centered access to diverse yet standardized medical data sources, and a unique analytical toolbox that fosters clinical validation, regulatory clearance and effective marketing | ||
|
||
How it works? | ||
---------------- | ||
Standardized, unified and diversified data customized for your ML needs and regulatory requirements Gesund assesses model validation needs and provides a suitable mix of high-quality data from its multiple and diverse clinical partner sites | ||
Model owner shares clinical study with Gesund for curation of appropriate dataset(s), and uploads their model onto Gesunds federated validation platform, which resides on hospital premise or private cloud. | ||
Model runs against a previously unseen validation data set that has been curated on the hospital side. | ||
Model accuracy metrics are produced and displayed on the Gesund platform for further examination with respect to patient characteristics, scenario analyses and stress testing. | ||
The model insights are exported into a report for the model owner to supplement their regulatory submission. | ||
|
||
.. toctree:: | ||
:maxdepth: 2 | ||
:caption: Contents: | ||
|
||
user_guide.md | ||
validation_metrics.md | ||
platform_integrations.md | ||
api.md | ||
|
||
Apache 2.0 License | ||
-------------------- | ||
Licensed under the Apache License, Version 2.0 (the "License"); | ||
you may not use this file except in compliance with the License. | ||
You may obtain a copy of the License at | ||
|
||
http://www.apache.org/licenses/LICENSE-2.0 | ||
|
||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
limitations under the License. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
# Platform Integrations | ||
|
||
## Datasets | ||
### Upload | ||
### Download | ||
### Delete | ||
### PACs Curate | ||
### Analysis | ||
|
||
## Models | ||
### Upload | ||
### Download | ||
### Delete | ||
### Batch Predictions | ||
### Model Extensions | ||
|
||
## Validations | ||
### Run Validations | ||
### Compare | ||
### Export | ||
|
||
## NLP | ||
### NER | ||
### Comparison | ||
|
||
## Fairness and Equity | ||
### Statistical Parity | ||
### Equalized Odds | ||
### Equal Opportunity |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
# User Guide | ||
## Installation | ||
## Getting-Started | ||
|
||
# Classification | ||
|
||
# Object Detection | ||
|
||
# Segmentation | ||
|
||
# Scalar Measurements | ||
|
||
# Sub-cohort Analysis | ||
|
||
# Platform Integrations | ||
## Datasets | ||
### Upload | ||
### Download | ||
### Delete | ||
### PACs Curate | ||
### Analysis | ||
## Models | ||
### Upload | ||
### Download | ||
### Delete | ||
### Batch Predictions | ||
### Model Extensions | ||
## Validations | ||
### Run Validations | ||
### Compare | ||
### Export | ||
## NLP | ||
### NER | ||
### Comparison | ||
## Fairness and Equity | ||
### Statistical Parity | ||
### Equalized Odds | ||
### Equal Opportunity |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,183 @@ | ||
# User Guide | ||
|
||
`gesundai-sdk` is python package that enables you to connect with [Gesund.ai](https://gesund.ai/) and utilize majority of the features and capabilities that you as a developer could integrate in your customized workflows to design and execute validation your datasets and model. | ||
|
||
`gesund-val-library` is a python package that explicitly enables you to design and execute your customized machine learning validation workflows with our validation tools. | ||
It is a set of tools that allow developers and researchers to measure the performance of AI models using specific evaluation metrics. In healthcare, these libraries are essential for determining whether AI models can safely and accurately function in real-world clinical environments. Validation libraries ensure models are not only statistically sound but also clinically relevant, addressing key considerations like patient fairness, accuracy across different demographics, and performance in handling different medical conditions. | ||
|
||
This user guide provides a comprehensive overview of `gesundai-val-library`, including installation instructions, basic usage examples, advanced features, and customization options. Whether you're a beginner or an experienced developer, this guide will help you get started and make the most of our package. | ||
|
||
|
||
|
||
## Installation | ||
|
||
**A. Create virtual environment** | ||
|
||
1. Create a virtual environment using any of the following choices | ||
|
||
- [Conda](https://www.anaconda.com/download) | ||
- [Pyenv](https://github.com/pyenv/pyenv) | ||
- [Poetry](https://python-poetry.org/) | ||
- [PDM](https://pdm-project.org/en/latest/) | ||
- [Virtualenv](https://virtualenv.pypa.io/en/latest/) | ||
- [Venv](https://docs.python.org/3/library/venv.html) | ||
|
||
|
||
Activate the environment before the next step | ||
|
||
2. Install the requirements from the requirements text file | ||
|
||
*TBA* | ||
|
||
|
||
**B. Install the library** | ||
|
||
|
||
Installation could be done either using one of the following | ||
|
||
1. Pip | ||
|
||
``` | ||
pip install --upgrade gesund-val-library | ||
``` | ||
|
||
2. Github | ||
|
||
``` | ||
git clone https://github.com/gesund-ai/gesund_val_library.git | ||
cd gesund_val_library | ||
python -m setup.py | ||
``` | ||
|
||
|
||
|
||
## Getting-Started | ||
|
||
|
||
### Basic usage | ||
|
||
``` | ||
# import the required libraries | ||
from gesund_val_library.validation import run_metrics | ||
import pprint | ||
|
||
# provide the json files for respective values | ||
args = { | ||
'annotations_json_path': '/path/to/annotations.json', | ||
'predictions': '/path/to/predictions.json', | ||
'class_mappings': '/path/to/class_mappings.json', | ||
'problem_type': 'object_detection', | ||
'format': 'json_format', | ||
'write_results_to_json': True | ||
} | ||
|
||
# execute the validation metrics | ||
result = run_metrics(args) | ||
|
||
# display the results; the results are in dictionary format | ||
pprint.pprint(result) | ||
|
||
``` | ||
|
||
The supported `format` values are `coco`, `yolo`, `gesund_custom_format`. This indicates that the json file is structured in the mentioned format. | ||
Under the hood, other formats are converted to `gesund_custom_format` and are used. | ||
|
||
The result is comprised of | ||
- The `result` dictionary containing key-value pairs of validation metrics | ||
- The resultant plots are stored in the local as `.png` files | ||
- The `output` directory consisting of results stored as `.json` files. The results are only produced if `write_results_to_json: True` | ||
|
||
|
||
|
||
## Examples | ||
|
||
### Run Validation | ||
|
||
**1. Classification** | ||
|
||
Inorder to the run classification specific validation metrics, set `problem_type: classification` in the args dictionary and provide the respective path. | ||
|
||
``` | ||
# import the required libraries | ||
from gesund_val_library.validation import run_metrics | ||
import pprint | ||
|
||
# provide the json files for respective values | ||
args = { | ||
'annotations_json_path': '/path/to/annotations.json', | ||
'predictions': '/path/to/predictions.json', | ||
'class_mappings': '/path/to/class_mappings.json', | ||
'problem_type': 'classification', | ||
'format': 'json_format', | ||
'write_results_to_json': True | ||
} | ||
|
||
# execute the validation metrics | ||
result = run_metrics(args) | ||
|
||
# display the results; the results are in dictionary format | ||
pprint.pprint(result) | ||
|
||
``` | ||
|
||
|
||
|
||
**2. Segmentation** | ||
|
||
|
||
Inorder to the run segmentation specific validation metrics, set `problem_type: segmentation` in the args dictionary and provide the respective path. | ||
|
||
``` | ||
# import the required libraries | ||
from gesund_val_library.validation import run_metrics | ||
import pprint | ||
|
||
# provide the json files for respective values | ||
args = { | ||
'annotations_json_path': '/path/to/annotations.json', | ||
'predictions': '/path/to/predictions.json', | ||
'class_mappings': '/path/to/class_mappings.json', | ||
'problem_type': 'segmentation', | ||
'format': 'json_format', | ||
'write_results_to_json': True | ||
} | ||
|
||
# execute the validation metrics | ||
result = run_metrics(args) | ||
|
||
# display the results; the results are in dictionary format | ||
pprint.pprint(result) | ||
|
||
``` | ||
|
||
**3. Object Detection** | ||
|
||
Inorder to the run object detection specific validation metrics, set `problem_type: object_detection` in the args dictionary and provide the respective path. | ||
|
||
``` | ||
# import the required libraries | ||
from gesund_val_library.validation import run_metrics | ||
import pprint | ||
|
||
# provide the json files for respective values | ||
args = { | ||
'annotations_json_path': '/path/to/annotations.json', | ||
'predictions': '/path/to/predictions.json', | ||
'class_mappings': '/path/to/class_mappings.json', | ||
'problem_type': 'object_detection', | ||
'format': 'json_format', | ||
'write_results_to_json': True | ||
} | ||
|
||
# execute the validation metrics | ||
result = run_metrics(args) | ||
|
||
# display the results; the results are in dictionary format | ||
pprint.pprint(result) | ||
|
||
``` | ||
|
||
|
||
### Sub Cohort Analysis | ||
|
||
*TBA* |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
# Validation Metrics | ||
|
||
|
||
This section provides an explanation on the implemented metrics used for analysis of the model predictions vs ground truth and comparison of annotations to derive | ||
agreement score between two or more annotators. | ||
|
||
## Classification | ||
|
||
*TBA* | ||
|
||
## Segmentation | ||
|
||
*TBA* | ||
|
||
## Object Detection | ||
|
||
*TBA* | ||
|
||
## Annotation Agreements | ||
|
||
*TBA* |
Oops, something went wrong.