Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Experiment updates and more flexible cross-validation for Paper revisions #321

Merged
merged 41 commits into from
Nov 9, 2024

Conversation

odunbar
Copy link
Collaborator

@odunbar odunbar commented Oct 3, 2024

Purpose

Adjusts examples to fit referee reviews.

  • this adds parameter sweeps over n_features in G function example
  • this adds tests to sweep over ranks of different nonseparable and separable feature distributions for L63 and UQ for EDMF
  • this updates and adds better plotting

Content

Additionally the following src code changes were also made

  • movement towards a more concrete cross-validation setup (n_cross_val_sets can be set 0 for no CV, or an integer k to perform k splits with the train-test fraction)
  • added the beta / scaled_coeffs penalty back into the loss function
  • moved some default values of ensemble size to be 10*n_params capped at 100 inline with our EKP recommendations
  • made some unit-tests and online docs compatible with julia v1.11

MISC

  • there is a bad Scalar RF test that I will report from line 344
    @test isapprox.(norm(μs - new_outputs), 0, atol = tol_μ)
    Often the problem zeros during optimization (possibly identity issue and should be protected against this)

  • I have read and checked the items on the review checklist.

Copy link

codecov bot commented Oct 3, 2024

Codecov Report

Attention: Patch coverage is 81.90476% with 19 lines in your changes missing coverage. Please review.

Project coverage is 88.85%. Comparing base (acf1051) to head (b636787).
Report is 27 commits behind head on main.

Files with missing lines Patch % Lines
src/VectorRandomFeature.jl 67.30% 17 Missing ⚠️
src/ScalarRandomFeature.jl 95.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #321      +/-   ##
==========================================
+ Coverage   88.17%   88.85%   +0.67%     
==========================================
  Files           8        7       -1     
  Lines        1184     1274      +90     
==========================================
+ Hits         1044     1132      +88     
- Misses        140      142       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@odunbar odunbar force-pushed the orad/revision-rf-updates branch from 3013066 to 270010e Compare November 7, 2024 23:37
@odunbar odunbar changed the title [WIP] Plotting and other changes for paper revisions [WIP] Experiment updates and more flexible cross-validation for Paper revisions Nov 7, 2024
@odunbar odunbar merged commit 6464f2a into main Nov 9, 2024
5 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant