Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Round 18 submission for USC-SIkJalpha #277

Merged
merged 5 commits into from
May 9, 2024

Conversation

scc-usc
Copy link
Contributor

@scc-usc scc-usc commented May 8, 2024

Description

Round 18 submission


If this section does not apply to you, please delete it in your PR

If this is a new team submission, please include the following details :-

  • Team name:
  • Model Name:
  • Institution:

If this section does not apply to you, please delete it in your PR

If you are adding new scenarios to an existing model, please include the following details:-

  • Team name:
  • Model name that is being updated:

Checklist

  • Specify a proper PR title with your team name.
  • All validation checks ran successfully on your branch. Instructions to run the tests locally is present here.

Copy link

github-actions bot commented May 8, 2024

No projection submission file in the standard SMH file format found in the Pull-Request. No validation was run.

Copy link

github-actions bot commented May 8, 2024

If the submission contains projection file(s) with quantile projection, a pdf containing visualization plots of the submission is available and downloadable in the GitHub actions. Please click on 'details' on the right of the 'Validate submission' checks. The pdf is available in a ZIP file as an artifact of the GH Actions. For more information, please see here

@LucieContamin
Copy link
Collaborator

Hi @scc-usc ,

Thanks for your submission! Just to add more information from the automatic message, the validation was not run because the submission file was not recognized, it needs to be named: "2024-04-28-USC-SIkJalpha".

Please let me know if any issues or questions,
Best, Lucie

Copy link

github-actions bot commented May 8, 2024

Run validation on files: 2024-04-28-USC-SIkJalpha.gz.parquet

Columns:

No errors or warnings found on the column names and numbers

Scenarios:

No errors or warnings found on scenario name and scenario id columns

Origin Date Column:

❌ Error 302: 'origin_date' should contains 1 unique date. The file contains multiple `origin_date' values: '2024-04-28, 2024-03-01'.
❌ Error 303: 'origin_date' is not corresponding to the expected value, the 'origin_date' date value should correspond to: '2024-04-28'.

Value and Type Columns:

🟡 Warning 505: Some location/target/scenario groups have a unique value for the whole projection period. 215 unique groups have been identified with this issue. For example:
group: 2024-04-28, A-2024-03-01, 31, inc death, 0-64;
group: 2024-04-28, B-2024-03-01, 31, inc death, 0-64;
group: 2024-04-28, C-2024-03-01, 31, inc death, 0-64;
etc.

Target Columns:

🟡 Warning 606: The projection contains more projected week than expected. The additional weeks might not be included in the Ensembles and/or visualizations
❌ Error 607: At least one target week is missing in the time series. 196465 unique groups have been identified with this issue. For example:
group: 2024-04-28, A-2024-03-01, 54, inc hosp, 1, 0-64, sample, 112;
group: 2024-04-28, C-2024-03-01, 54, inc hosp, 1, 0-64, sample, 112;
group: 2024-04-28, E-2024-03-01, 54, inc hosp, 1, 0-64, sample, 112;
etc.
🟡 Warning 602: No value found associated with the targets: inc death (optional), inc hosp (optional), cum death (optional), cum hosp (optional); output_type: quantile, mean.

Locations:

No errors or warnings found on Location

Sample:

❌ Error 905: All the groups should contains the same number of trajectories per group. Please verify.

Quantiles:

No errors or warnings found on quantiles values and format

Age Group:

No errors or warnings found on Age_group

Copy link

github-actions bot commented May 9, 2024

Run validation on files: 2024-04-28-USC-SIkJalpha.gz.parquet

Columns:

No errors or warnings found on the column names and numbers

Scenarios:

No errors or warnings found on scenario name and scenario id columns

Origin Date Column:

No errors or warnings found on the column 'origin_date'

Value and Type Columns:

❌ Error 5041: All values should be greater or equal to 0 the data frame contains some error, please verify.
🟡 Warning 505: Some location/target/scenario groups have a unique value for the whole projection period. 215 unique groups have been identified with this issue. For example:
group: 2024-04-28, A-2024-03-01, 31, inc death, 0-64;
group: 2024-04-28, B-2024-03-01, 31, inc death, 0-64;
group: 2024-04-28, C-2024-03-01, 31, inc death, 0-64;
etc.

Target Columns:

🟡 Warning 606: The projection contains more projected week than expected. The additional weeks might not be included in the Ensembles and/or visualizations
🟡 Warning 602: No value found associated with the targets: inc death (optional), inc hosp (optional), cum death (optional), cum hosp (optional); output_type: quantile, mean.

Locations:

No errors or warnings found on Location

Sample:

No errors or warnings found on Sample

Quantiles:

No errors or warnings found on quantiles values and format

Age Group:

No errors or warnings found on Age_group

Copy link

github-actions bot commented May 9, 2024

Run validation on files: 2024-04-28-USC-SIkJalpha.gz.parquet

Columns:

No errors or warnings found on the column names and numbers

Scenarios:

No errors or warnings found on scenario name and scenario id columns

Origin Date Column:

No errors or warnings found on the column 'origin_date'

Value and Type Columns:

🟡 Warning 505: Some location/target/scenario groups have a unique value for the whole projection period. 215 unique groups have been identified with this issue. For example:
group: 2024-04-28, A-2024-03-01, 31, inc death, 0-64;
group: 2024-04-28, B-2024-03-01, 31, inc death, 0-64;
group: 2024-04-28, C-2024-03-01, 31, inc death, 0-64;
etc.

Target Columns:

🟡 Warning 606: The projection contains more projected week than expected. The additional weeks might not be included in the Ensembles and/or visualizations
🟡 Warning 602: No value found associated with the targets: inc death (optional), inc hosp (optional), cum death (optional), cum hosp (optional); output_type: quantile, mean.

Locations:

No errors or warnings found on Location

Sample:

No errors or warnings found on Sample

Quantiles:

No errors or warnings found on quantiles values and format

Age Group:

No errors or warnings found on Age_group

@LucieContamin
Copy link
Collaborator

Hi @scc-usc ,

Thank you for your submission, I just wanted to verify that I understand correctly the sample ID numbering: there is no stochasticity, every models runs has a different run_grouping "grouped" by age_group, location, horizon, scenario_id, and, target. Is that correct?

Please let me know if any issues or questions,
Best, Lucie

@scc-usc
Copy link
Contributor Author

scc-usc commented May 9, 2024

Each trajectory is generated by a predetermined set of hyperparameters. Once these hyperparameters are fixed, the trajectory is unique. "run_grouping" is the trajectory ID.

@LucieContamin LucieContamin merged commit ea1cb0b into midas-network:master May 9, 2024
1 check passed
@LucieContamin
Copy link
Collaborator

Hi @scc-usc , thank you for the answer and additional answer!

Best, Lucie

@LucieContamin
Copy link
Collaborator

Hi again,

I apologize for the multiple message, but I notice some small issues in your submission. It's ok for this round, but would it be possible for the next updates, submissions to provide the horizon, run_grouping, stochastic_run and value column in a numeric format, please? Please find here, some documentation on the expected format of the submission.

Please let me know if any issues or questions,
Best, Lucie

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants