RLBench learning speed

[cambel]

Hi, I am checking this repository, I was able to install everything without apparent problem.

I am testing the run_rlzoo.py script using RLBench with the ReachTarget. I run it but it is quite slow. One episode takes about 2~3 minutes. I wonder if you have seen the same behavior when training or somehow is my configuration. At first, the episode took about 5 min, then I realized that TensorFlow was not working with my GPU, I fixed that and well now is twice as fast, but still, 3 minutes per episode is quite slow, especially if it plans to run for 1000 episodes.

It is that the normal speed using R-VEP? is there anything I can do to train using faster-than-real-time simulation with RLBench?

[quantumiracle]

Hi,
Great to hear that you can run it without problem.
It's true that RLBench env is slow compared with other toy tests, because of the physical simulation. Several tips:
(1) change headless=True within build_rlbench_env.py to disable the visualization for training;
(2) enlarge action_range and choose smaller max_steps in the defaulty.py in one algorithm if you want;
(3) as RLBench env mostly depends on image observation, you should expect a long training time.
Moreover, if you check the RLBench env, the observation should contain both high-dim images and low-dim vectors to get full information, but right now we just provide the image as input. So you may not ideally train the robot to achieve some goals yet without any modification of codes. As most algorithms we provided assume the observation to be an array instead of a tuple or dict, you may need to modify these parts in the algorithm code for using RLBench, as well as provide a multi-head network for handling different types of input.

Zihan

[cambel]

Thanks for your quick reply!

I will check the code taking into account your tips, thanks!