GradParams feature as constraint

[goyalpike]

Great toolbox! Thanks for providing it.

I am interested in using GradParams as a constraint so that coeffs can be optimized along with the network. I was wondering if it is as simple as just replacing constraint = LeastSquares() by constraint = GradParams(inp2, inp2). But I believe there would be a need to define an optimizer for coeffs as well which is not provided. All the demo files use least-squares as a constraint, which is solved by QR, which would not be the case when the coeffs are optimized along with the network.

So a question is: are these optimization steps for coeffs are taken care of by the toolbox as soon as we use constraint = GradParams(inp2, inp2). Also, it would be nice to have a demo hinting how to use GradParams.

Any guidance would be highly appreciated.

Many thanks!

[GJBoth]

Happy to hear you're finding it useful!
Indeed it is that simple: GradParams creates a pytorch parameterlist (see line 43-45 in constraint.py), so that coefficients get added to the model parameters and are returned when you call model.parameters(). This means that the coefficients and the network share the same optimizer (and settings like learning rate), so if you want separate optimizers you'd need to manually specify which of the model.parameters() you want to optimize for each optimizer.

Good point about a demo, I'll add it to the examples!

[goyalpike]

Thank you for the clarification. I appreciate.