Skip to content

Commit

Permalink
Merge pull request #79 from Infectious-Disease-Modeling-Hubs/enhancem…
Browse files Browse the repository at this point in the history
…ent/handle-null-taskids

Enhancement/handle null taskids
  • Loading branch information
annakrystalli authored Apr 3, 2024
2 parents 1c03e97 + 79a6d3b commit f6b9eb2
Show file tree
Hide file tree
Showing 15 changed files with 663 additions and 6 deletions.
2 changes: 2 additions & 0 deletions .github/workflows/R-CMD-check.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ on:
branches: [main, master]
pull_request:
branches: [main, master]
schedule:
- cron: '0 0 * * *'

name: R-CMD-check

Expand Down
7 changes: 4 additions & 3 deletions DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,8 @@ Imports:
dplyr,
fs,
gh,
hubAdmin,
hubData,
hubAdmin (>= 0.1.0),
hubData (>= 0.1.0),
hubUtils (>= 0.0.1),
jsonlite,
jsonvalidate,
Expand All @@ -63,7 +63,8 @@ Remotes:
Infectious-Disease-Modeling-Hubs/hubUtils,
Infectious-Disease-Modeling-Hubs/hubData,
Infectious-Disease-Modeling-Hubs/hubAdmin,
assignUser/octolog
assignUser/octolog,
apache/arrow/[email protected]
Config/testthat/edition: 3
Config/Needs/website: pkgdown, Infectious-Disease-Modeling-Hubs/hubStyle
Encoding: UTF-8
Expand Down
7 changes: 7 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
# hubValidations 0.0.1

* Release stable 0.0.1 version
* Enforce minimum dependence on latest `hubData` (0.1.0) & `hubAdmin` (0.1.0). This allows for successful validation of submissions to hubs with multiple model tasks, where a given model task might contain non relevant task IDs and both `required` and `optional` properties have been set to `null` in `tasks.json` (#75). See the [relevant section in `hubDocs` documentation](https://hubdocs.readthedocs.io/en/latest/quickstart-hub-admin/tasks-config.html#required-and-optional-elements) for more details.
* Improve formatting of current time print in `validate_submission_time()` message by removing decimal seconds and including local time zone.


# hubValidations 0.0.0.9008

* Added new articles on:
Expand Down
4 changes: 3 additions & 1 deletion R/check_submission_time.R
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,9 @@ check_submission_time <- function(hub_path, file_path, ref_date_from = c(
details <- NULL
} else {
details <- cli::format_inline(
"Current time {.val {Sys.time()}} is outside window {.val {submission_window}}."
"Current time {.val {format(Sys.time(),
format = '%Y-%m-%d %H:%M:%S',
usetz = TRUE)}} is outside window {.val {submission_window}}."
)
}

Expand Down
9 changes: 9 additions & 0 deletions README.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,15 @@ You can install the development version of hubValidations like so:
remotes::install_github("Infectious-Disease-Modeling-Hubs/hubValidations")
```

> ##### 💡 TIP
> `hubValidations` has a dependency on the `arrow` package. For troubleshooting `arrow` installation problems, please consult the [`arrow` package documentation](https://arrow.apache.org/docs/r/#installation).
>
> You could also try installing the package from the [Apache R Universe repository](https://apache.r-universe.dev) with:
>
> ```r
> install.packages("arrow", repos = c("https://apache.r-universe.dev", "https://cran.r-project.org"))
> ```
***
## Code of Conduct
Expand Down
14 changes: 14 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,20 @@ You can install the development version of hubValidations like so:
remotes::install_github("Infectious-Disease-Modeling-Hubs/hubValidations")
```

> ##### 💡 TIP
>
> `hubValidations` has a dependency on the `arrow` package. For
> troubleshooting `arrow` installation problems, please consult the
> [`arrow` package
> documentation](https://arrow.apache.org/docs/r/#installation).
>
> You could also try installing the package from the [Apache R Universe
> repository](https://apache.r-universe.dev) with:
>
> ``` r
> install.packages("arrow", repos = c("https://apache.r-universe.dev", "https://cran.r-project.org"))
> ```
------------------------------------------------------------------------
## Code of Conduct
Expand Down
2 changes: 1 addition & 1 deletion hubValidations.Rproj
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ AlwaysSaveHistory: Default

EnableCodeIndexing: Yes
UseSpacesForTab: Yes
NumSpacesForTab: 4
NumSpacesForTab: 2
Encoding: UTF-8

RnwWeave: Sweave
Expand Down
268 changes: 267 additions & 1 deletion tests/testthat/_snaps/validate_submission.md

Large diffs are not rendered by default.

43 changes: 43 additions & 0 deletions tests/testthat/test-validate_submission.R
Original file line number Diff line number Diff line change
Expand Up @@ -158,3 +158,46 @@ test_that("validate_submission fails when csv cannot be parsed according to sche
c("check_error", "hub_check", "rlang_error", "error", "condition")
)
})

test_that("File containing task ID with all null properties validate correctly", {
expect_snapshot(
str(
validate_submission(
hub_path = test_path("testdata/hub-null"),
file_path = "team-model/2023-11-26-team-model.parquet",
skip_submit_window_check = TRUE
)
)
)
expect_true(
suppressMessages(
check_for_errors(
validate_submission(
hub_path = test_path("testdata/hub-null"),
file_path = "team-model/2023-11-26-team-model.parquet",
skip_submit_window_check = TRUE
)
)
)
)
expect_snapshot(
str(
validate_submission(
hub_path = test_path("testdata/hub-null"),
file_path = "team-model/2023-11-19-team-model.parquet",
skip_submit_window_check = TRUE
)
)
)
expect_true(
suppressMessages(
check_for_errors(
validate_submission(
hub_path = test_path("testdata/hub-null"),
file_path = "team-model/2023-11-19-team-model.parquet",
skip_submit_window_check = TRUE
)
)
)
)
})
13 changes: 13 additions & 0 deletions tests/testthat/testdata/hub-null/hub-config/admin.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"schema_version": "https://raw.githubusercontent.com/Infectious-Disease-Modeling-Hubs/schemas/main/v2.0.0/admin-schema.json",
"name": "US CDC FluSight",
"maintainer": "US CDC",
"contact": {
"name": "Joe Bloggs",
"email": "[email protected]"
},
"repository_host": "GitHub",
"repository_url": "https://github.com/cdcepi/Flusight-forecast-data",
"file_format": ["csv", "parquet", "arrow"],
"timezone": "US/Eastern"
}
128 changes: 128 additions & 0 deletions tests/testthat/testdata/hub-null/hub-config/model-metadata-schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"title": "Schema for Modeling Hub model metadata",
"description": "This is the schema for model metadata files, please refer to https://github.com/covid19-forecast-hub-europe/covid19-forecast-hub-europe/wiki/Metadata for more information.",
"type": "object",
"properties": {
"team_name": {
"description": "The name of the team submitting the model",
"type": "string"
},
"team_abbr": {
"description": "Abbreviated name of the team submitting the model",
"type": "string",
"pattern": "^[a-zA-Z0-9_+]+$",
"maxLength": 16
},
"model_name": {
"description": "The name of the model",
"type": "string"
},
"model_abbr": {
"description": "Abbreviated name of the model",
"type": "string",
"pattern": "^[a-zA-Z0-9_+]+$",
"maxLength": 16
},
"model_version": {
"description": "Identifier of the version of the model",
"type": "string"
},
"model_contributors": {
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {
"type": "string"
},
"affiliation": {
"type": "string"
},
"email": {
"type": "string",
"format": "email"
},
"orcid": {
"type": "string",
"pattern": "^\\d{4}\\-\\d{4}\\-\\d{4}\\-[\\dX]{4}$"
}
},
"additionalProperties": false,
"required": ["name", "affiliation", "email"]
}
},
"website_url": {
"description": "Public facing website for the model",
"type": "string",
"format": "uri"
},
"repo_url": {
"description": "Repository containing code for the model",
"type": "string",
"format": "uri"
},
"license": {
"description": "License for use of model output data",
"type": "string",
"enum": [
"CC0-1.0",
"CC-BY-4.0",
"CC-BY_SA-4.0",
"PPDL",
"ODC-by",
"ODbL",
"OGL-3.0"
]
},
"designated_model": {
"description": "Team-specified indicator for whether the model should be eligible for inclusion in a Hub ensemble and public visualization. A team may designate up to two models.",
"type": "boolean"
},
"citation": {
"description": "One or more citations for this model",
"type": "string",
"examples": ["Gibson GC , Reich NG , Sheldon D. Real-time mechanistic bayesian forecasts of Covid-19 mortality. medRxiv. 2020. https://doi.org/10.1101/2020.12.22.20248736"]
},
"team_funding": {
"description": "Any information about funding source for the team or members of the team.",
"type": "string",
"examples": ["National Institutes of General Medical Sciences (R01GM123456). The content is solely the responsibility of the authors and does not necessarily represent the official views of NIGMS."]
},
"data_inputs": {
"description": "List or description of data inputs used by the model",
"type": "string"
},
"methods": {
"description": "A brief (200 char.) description of the methods used by this model",
"type": "string",
"maxLength": 200
},
"methods_long": {
"description": "A full description of the methods used by this model. Among other details, this should include whether spatial correlation is considered and how the model accounts for uncertainty.",
"type": "string"
},
"ensemble_of_models": {
"description": "Indicator for whether this model is an ensemble of any separate component models",
"type": "boolean"
},
"ensemble_of_hub_models": {
"description": "Indicator for whether this model is an ensemble specifically of other models submitted to this Hub",
"type": "boolean"
}
},
"additionalProperties": true,
"required": [
"team_name",
"team_abbr",
"model_name",
"model_abbr",
"model_contributors",
"license",
"data_inputs",
"methods",
"methods_long",
"ensemble_of_models",
"ensemble_of_hub_models"
]
}
Loading

0 comments on commit f6b9eb2

Please sign in to comment.