Is it possible to build a GT navigator using AI2THOR 2.1.0?

[TopCoder2K]

The latest versions have the 'GetShortestPath' and 'GetShortestPathToPoint' actions, so it is possible to build a trajectory to any desired object. Using this, one can build a GT navigator. But how can this be achieved in the 2.1.0 AI2THOR?

Also, are there any plans to update the simulator or create something like ALFRED 2.0? :)

[MohitShridhar]

@TopCoder2K, checkout ALFWorld's oracle controller. The oracle controller uses an AStar navigator with GT grid maps of rooms.

Also, are there any plans to update the simulator or create something like ALFRED 2.0? :)

Unfortunately, no. The simulator has changed a lot in the past 3 years. Receptacles were moved around, objects look different. All these changes will require us to re-annotate the dataset, which is super expensive. Also see this: #45

[TopCoder2K]

checkout ALFWorld's oracle controller

@MohitShridhar, thank you for pointing this out! I'll look into this. Please do not close the issue, I'll close it by myself if there are no additional questions.

Unfortunately, no.

Well... I've seen #45 but the simulator version here is also old, isn't it? Moreover, we've encountered inconsistent instructions, bugs in AI2THOR, problems with ALFRED deployment, and so on, so creating ALFRED 2.0 may seem reasonable.

[TopCoder2K]

@MohitShridhar, thank you one more time! I've integrated the oracle navigator into our pipeline using your code. The only problem is that it can only navigate to receptacles. I can implement teleportation to objects (e.g. apples, knives that are not hidden) using iteration through all reachable positions (obtained by action='GetReachablePositions' and event.metadata["actionReturn"]) and rotation, but it looks bad. Is there an elegant way of performing teleportation in positions from which one can successfully interact with the desired object?

[MohitShridhar]

@TopCoder2K, if I remember correctly, there is no easy solution to this, at least with ai2thor==2.1.0. The only way is trial and error. This is also why there are a lot of failed trajectories during data generation. The later versions of Ai2Thor might have some solutions, but I have lost touch with the updates made to the simulator in the past 3 years.

[TopCoder2K]

@MohitShridhar, can FloorPlanX-openable.json be used for the all kind of interactions (not just for opening and closing, but also for lifting, putting, turning on/off)? I'm sure you tested the points, but I found at least the following example:


The 'Put' action fails with No valid Receptacle found and I can confirm that both SideTable-s are not visible (distance > 1.5m).

UPD1:
It is the 329th floor plan (used in the 384th episode of valid seen)

UPD2:
I've even found a case when the teleport action to the 'Fridge' fails... It is the 136th episode (the 7th floor plan).

[MohitShridhar]

@TopCoder2K, this script is how the JSONs were generated: https://github.com/askforalfred/alfred/blob/master/gen/layouts/precompute_layout_locations.py#L36

I think we only checked for Closing, Opening, and PickUp. But this should be extendable to other actions.

[TopCoder2K]

@MohitShridhar, thank you for the link. Hmmm, it's strange. I don't fully understand your script, but I can clearly see that obj['visible'] is checked to be True before returning any good_obj_point, even though I'm sure 'SideTable'-s are not visible in my case (and their drawers too, so the 'Open' action must also fail). Maybe, this is again a quirk of AI2THOR...

UPD1:
I also noticed that the generated points do not take into account holding an object: in the 121th episode, I receive Cannot teleport due to hand object collision when trying to teleport to the microwave with a tomato in the hand.

[TopCoder2K]

I've been trying to fix as much problems as possible (with a constraint of spending reasonable time on that) and ended up with a good version of OracleNavigator that can navigate to both receptacles and objects (but objects can't be hidden in closed receptacles). Thank you for the help!