For an ObjectNav task, where is the ObjectGoal computation code?

[ResearcherYan]

Hi, developers for habitat-lab.

I'm currently carrying on an ObjectNav task on the Matterport3D dataset in habitat. However according to my evaluation videos, I found there are so many cases where the agent finds and reaches the goal correctly but the function habitat.Env.get_metrics() returns success=0. Then I started to dive into the habitat-lab source code, trying to figure out how it defines whether an episode succeeds or fails. Then I got to know for a navigation task, when DISTANCE_TO_GOAL.DISTANCE_TO is set to VIEW_POINT, the variable self._episode_view_points will be filled with ObjectGoals from where the object can be seen. The computation process of self._episode_view_points is written in the method reset_metric of class DistanceToGoal in habitat-lab/habitat/tasks/nav/nav.py:

def reset_metric(self, episode, *args: Any, **kwargs: Any):
    self._previous_position = None
    self._metric = None
    if self._config.DISTANCE_TO == "VIEW_POINTS":
        self._episode_view_points = [
            view_point.agent_state.position
            for goal in episode.goals
            for view_point in goal.view_points
        ]
    self.update_metric(episode=episode, *args, **kwargs)  # type: ignore

My question is: Where is the computation code for episode.goals in the above code block? I can't find it in the source code. Please give me some enlightenment! Thanks a lot!

[ResearcherYan]

I think I found where the ObjectGoal is obtained. For MP3D dataset, the ObjectGoal for an episode is not computed. Instead, it is directly loaded from the json file inside the dataset folder. It's implemented in the habitat-lab/habitat/datasets/object_nav/object_nav_dataset.py:

@registry.register_dataset(name="ObjectNav-v1")
class ObjectNavDatasetV1(PointNavDatasetV1):
    ...
    def from_json(
        self, json_str: str, scenes_dir: Optional[str] = None
    ) -> None:
        deserialized = json.loads(json_str)
        ...
        for k, v in deserialized["goals_by_category"].items():
            self.goals_by_category[k] = [self.__deserialize_goal(g) for g in v]
        for i, episode in enumerate(deserialized["episodes"]):
            episode = ObjectGoalNavEpisode(**episode)
            ...
            episode.goals = self.goals_by_category[episode.goals_key]
            ...
            self.episodes.append(episode)  # type: ignore [attr-defined]

So all the view point locations of each goal are predefined in the dataset json file. Now I guess the aforementioned problems (sometimes the agent finds and reaches the goal correctly but the function habitat.Env.get_metrics() returns success=0) maybe caused by the wrong view point locations provided by the dataset itself?