EnvSteps vs CumSteps for ATC

[bmazoure]

Upon running the training script for UL+RL link, I get two kinds of step metrics: CumSteps (eg 23,000) and EnvSteps (184,000 for the corresponding CumSteps). The reward at this snapshot roughly corresponds to the plot from ATC for 200k steps, from which I infer that all axes reported in the ATC paper are reported in EnvSteps - is this correct?

[KarlXing]

Hi @bmazoure , I'm trying to run the same script and meet an error of ModuleNotFoundError: No module named 'rlpyt.adam'. I cannot find the module of rlpyt.adam in the repository. May I know how did you solve this when running to training script for UL+RL. Thank you!