Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor and extend RF interface for more flexible kernels. #220

Merged
merged 1 commit into from
Oct 6, 2023
Merged

Refactor and extend RF interface for more flexible kernels. #220

merged 1 commit into from
Oct 6, 2023

Conversation

odunbar
Copy link
Collaborator

@odunbar odunbar commented Jul 11, 2023 •
edited
Loading

Purpose

Addresses points in #215 In particular:

Content

  • Adds MCMC stage into examples/GCM and improves running interface
  • Adds Shrinkage estimation into cross-validation for RF hyperparameter optimization.
  • Adds Kernel interface for RF, where one specifies either a Separable or Nonseparable Kernel followed by the structure of the covariance to be used to sample the features.
    • Nonseparable(cov): define one covariance structure on the p * d-dim space,
    • Separable(in_cov,out_cov): define one d-dim covariance for inputs and one p-dim covariance for outputs
  • Choose covariance structures for cov,in_cov,out_cov from
    • OneDimFactor(): Must be selected for 1D covariances
    • DiagonalFactor(eps): Diagonal covariance plus eps*I
    • CholeskyFactor(eps): Cholesky representation L*L^T plus eps*I`
    • LowRankFactor(rank, eps): Symmetric factorization W*W^T where W = 1+UDU' D diag, U rectangular, plus eps*I
    • HierarchicalLowRankFactor(rank, eps): Symmetric factorization W*W^T where W = 1+UXU' U rectangular, plus eps*I X cholesky factored
  • Rebalancing the loss function seems to get far more robust results for vector RF
  • I have read and checked the items on the review checklist.

@odunbar odunbar changed the title [WIP] Add "S" to GCM example, and methodology improvements to improve results [WIP] Refactor and extend RF interface for prior covariance structures. Jul 26, 2023
@odunbar odunbar force-pushed the orad/rf-mcmc-for-gcm branch 2 times, most recently from 950ecf6 to 1f1394c Compare July 28, 2023 22:26
@odunbar odunbar changed the title [WIP] Refactor and extend RF interface for prior covariance structures. [WIP] Refactor and extend RF interface for more flexible kernels. Aug 11, 2023
@odunbar odunbar force-pushed the orad/rf-mcmc-for-gcm branch 2 times, most recently from 7c1abd6 to 9cb5e89 Compare October 6, 2023 01:57
@odunbar
Copy link
Collaborator Author

odunbar commented Oct 6, 2023

bors try

bors bot added a commit that referenced this pull request Oct 6, 2023
@bors
Try #220:
cd24abe
@bors
Copy link
Contributor

bors bot commented Oct 6, 2023

try

Build succeeded!

The publicly hosted instance of bors-ng is deprecated and will go away soon.

If you want to self-host your own instance, instructions are here.
For more help, visit the forum.

If you want to switch to GitHub's built-in merge queue, visit their help page.

@odunbar
add Ledoit Wolf shrinkage estimator for cross-validation covariance
2b49075 
different priors

change prior again

add reg into cov-mat for optimization

example tweaks

add reg to scalar method and widen default prior

fix lorenz

fix lorenz

major refactor of RF prior cov

fix lorenz

format

refactored user interface

examples work with new interface

format

resolve merge

format

bug-fix n_hp calc

updates to prior - remove scalings

src for new kernel structure

updated Emulator and Lorenz example with Kernel interface

bugfix symbol checks

Cholesky factorization of K

remove unnecessary warning

needed branch in if-else

error message typo

better nugget term for LRF

sensible results for RF emulator exmaple

bypass JLD2 errors

make low-rank into more of a diagonal decomp

bugfix scatter bounds

added learnable scaling

Lorenz 63 example learning update map

gets l63 pdf

moved L63 into Emulator examples

remove duplicate svd

rebalance loss function (sqrt complexity, zero out scaled coeffs, remove approx_sig2 from noise), greatly improved robustness

example tweaks and fixes for plotting, convergence etc.

clima format

tests pass

clima format

change plots to makie, and small changes

change plots to makie, and small changes

format

docs fix

test shrinkage

format
@odunbar odunbar force-pushed the orad/rf-mcmc-for-gcm branch from c155682 to 2b49075 Compare October 6, 2023 02:54
@odunbar
Copy link
Collaborator Author

odunbar commented Oct 6, 2023

bors r+

@odunbar odunbar changed the title [WIP] Refactor and extend RF interface for more flexible kernels. Refactor and extend RF interface for more flexible kernels. Oct 6, 2023
@bors
Copy link
Contributor

bors bot commented Oct 6, 2023

Build succeeded!

The publicly hosted instance of bors-ng is deprecated and will go away soon.

If you want to self-host your own instance, instructions are here.
For more help, visit the forum.

If you want to switch to GitHub's built-in merge queue, visit their help page.

@bors bors bot merged commit d9f4d91 into main Oct 6, 2023
5 checks passed
@bors bors bot deleted the orad/rf-mcmc-for-gcm branch October 6, 2023 03:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Sample stage for GCM application SVD truncation for vector RF interface for defining user-priors
1 participant
@odunbar