Hello,

Good question, I didn't try to pretrain lstm yet... and currently there is not test for that, so I cannot assure you it will work out of the box. But i'm interested by your results ;)

Is there anything i need to keep in mind when I train the LSTM policy directly?

Pre-training only means supervised learning, so if you are training on a sequence of (observation, action) you should keep track of the hidden state between each step. I don't have other advice more than that for now.

Btw, you should be aware that we are currently working on refactoring the way recurrent policies are define, see PR #244.

Thanks for the answer
I have looked into a bit more and I have some more questions.

1. So there are two special placeholders I need to feed data in when using a LSTM policie. The initials hidden state tensor, and a mask tensor.
   The states have the shape (envs_per_batch, n_hidden * 2), and masks is a boolean list with the shape (envs_per_batch * self.n_steps) and it restest the hidden state when ever is True. What I don't understand is how the one shape is translated in to the other.

2. In the Docks there are some pre train examples using a pretrained function, but none of the learners seem to have this function. so would it remove?
   EDIT:
   Stupid me, forgot to check the version I am currently in.

I don't understand is how the one shape is translated in to the other.

The LSTM code is quite misleading, I think you can have some hints by reading @erniejunior issue: #158

Side note: state_shape = [n_lstm * 2] dim because of the cell and hidden states of the LSTM

In the Docks there are some pre train examples using a pretrained function, but none of the learners seem to have this function. so would it remove?

Yes, the online doc correspond to the "master" version. Pre-training was added in v2.5.0 that was released last week.

@XMaster96 Did you make pretrain() work with LSTM policies? Or did you had to tweak the code? (Here I'm not talking about recording expert data, which is your current PR ;))

@araffin sorry haven't seen your reply apparently i had the Illusion that I would get a notification.

Yes I am still working on it, the problem is just that the original solution is quite hack and not something you put in a PR. unfortunately I am also stretched a bit for time at the moment, but I shud have a early PR done in the next couple of days.

Edit:
I had some wrong notifications settings, now I should get notified