Bad performance in MuJoCo benchmarks

[jity16]

Hello, I face some problems when running XQL on MuJoCo benchmarks (online).

1. I have tested XQL on MuJoCo benchmarks, and none of the runs would return good performance, for example, Ant-v2: -100
2. Only set the loss function as MSE, it will work well in MuJoCo benchmarks.

I don't really know if there are any sensitive parameters, but it seems that if the MSE loss (aka SAC backbone) mentioned in point 2 can work, the parameters might be reasonable

[jhejna]

Hi!

Thanks for your interest in our work! We have not tried XQL on the Mujoco benchmark, only on DM Control. The reward structure of these environments is very different. If you are interested in getting XQL to work on these environments, I would suggest tuning the value of beta, which our method is extremely sensitive to. I suggest starting with a large beta = 10, and progressively lowering it until good performance is reached.

[jity16]

Thank you very much! We will try to tune beta later on.