Question on calculating square root of the FIM

[vimmoos]

I am currently writing a method to perform forgetting in RL. To do so I add to the model some noise based on the FIM.
A snippet for such an approach is:

 def calculate_scrub_noise(self):
        gaus_noise = obj.vector.random_pvector(
            lc.LayerCollection.from_model(self.model), device=self.device
        )
        inv = self.FIM.inverse().mv(gaus_noise)
        return self.scrub_scale * inv

However, I would like to take the square root of the $F^{-1}*noise$ .
So i would like to implement the following formula: $(scale*F^{-1}*noise)^{1/2}$
What would be the best way to implement it within your library?

Side Note: Great library! compared to other options I found your library to be the most mathematically accurate for the FIM !

Thanks for the response

[tfjgeorge]

Hello, I am not so sure what you want to achieve, you mean to sqrt every component of the vector and return a new PVector ? How do you deal with negative reals, with complex numbers ? Best, Thomas

…

On Mon, Sep 30, 2024 at 9:32 AM Massimiliano Falzari < ***@***.***> wrote: I am currently writing a method to perform forgetting in RL. To do so I add to the model some noise based on the FIM. A snippet for such an approach is: def calculate_scrub_noise(self): gaus_noise = obj.vector.random_pvector( lc.LayerCollection.from_model(self.model), device=self.device ) inv = self.FIM.inverse().mv(gaus_noise) return self.scrub_scale * inv However, I would like to take the square root of the $F^{-1}*noise$ . So i would like to implement the following formula: $(scale*F^{-1}*noise)^{1/2}$ What would be the best way to implement it within your library? Side Note: Great library! compared to other options I found your library to be the most mathematically accurate for the FIM ! Thanks for the response — Reply to this email directly, view it on GitHub <#85>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AALTMWISZWSATHE3W6QI423ZZD5ALAVCNFSM6AAAAABPCUTY42VHI2DSMVQWIX3LMV43ASLTON2WKOZSGU2TKOBRHAYTMNI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[vimmoos]

Thanks for the fast reply!
I am trying to apply the method described in Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks in the RL context. Section 4 of that paper describes the final formulation. I want to take the square root of the inverse of the FIM and not it's individual component (i.e. $X*X=F^{-1}$ where F is the FIM). My initial idea was to get the flat representation from the PVector, perform the square root using Pytorch (as described here: Pytorch sqrt) and then convert it back to a PVector so that I can add it to the model (using the add_model method) In the Pytorch forum linked above they suggest to treat negative eigenvalues as 0.
That said, I use the KFAC representation for the FIM so maybe there is a better/ more efficient / more idiomatic way to do it within your library.
Moreover, I am not sure what would mean geometrically to take the square root of the components besides simply scaling them. Do you have more insights on this ? And do you think that taking the square root of the components would have a similar effect compared to taking the square root of the matrix?
I hope this answer questions, if not please let me know

Best

[tfjgeorge]

This PR #87 should work using EKFAC matrices.

sqrt_inv_M = M_ekfac ** -0.5

In the case of KFAC, I am not sure what to implement since KFAC was only defined for computing the inverse in the paper. There is some weird side effect of the effect of the diagonal regularizer added before computing the inverse in KFAC. I also added a self.pow(n) method, but I don't think it works with non-integer n so this should not be relevant to your use case. I don't know what they actually do in the paper that you link.

[vimmoos]

I made an error in the previous message, I do indeed use the EKFAC, not the KFAC. Sorry for the inconvenience.
Thank you for the new PR. This looks like exactly what i was looking for.