Installation and runtime issues about goat-bench HOT 12 CLOSED

I commented out:

- /habitat_baselines/rl/policy/obs_transforms:
  - resize

in the ver_goat.yaml and it worked. However, I'm not sure if this is the correct solution.

Additionally, I encountered the following error during execution:

FileNotFoundError: [Errno 2] No such file or directory: 'data/datasets/languagenav/hm3d/v5_final/train_embeddings/clip_train_embeddings.pkl

Could I possibly get the relevant data?

from goat-bench.

Hi, I encountered the same issue during inference. In addition, when attempting to load the task dataset alone, I encountered the following error:

Traceback (most recent call last):
  File "NExplore/train/habitat_vis_exploration.py", line 43, in <module>
    dataset = HabitatChallengeDataset(config, device=torch.device("cuda" if torch.cuda.is_available() else "cpu"))
  File "/home/yan/Work/active_mapping/SeRFnav-goat/NExplore/datasets/dataset.py", line 59, in __init__
    self.sim = habitat.Env(cfg)
  File "/home/yan/Work/active_mapping/SeRFnav-goat/habitat-lab/habitat-lab/habitat/core/env.py", line 118, in __init__
    self._task = make_task(
  File "/home/yan/Work/active_mapping/SeRFnav-goat/habitat-lab/habitat-lab/habitat/tasks/registration.py", line 22, in make_task
    return _task(**kwargs)
  File "/home/yan/Work/active_mapping/SeRFnav-goat/goat-bench/goat_bench/task/goat_task.py", line 83, in __init__
    super().__init__(*args, **kwargs)
  File "/home/yan/Work/active_mapping/SeRFnav-goat/habitat-lab/habitat-lab/habitat/tasks/nav/nav.py", line 1320, in __init__
    super().__init__(config=config, sim=sim, dataset=dataset)
  File "/home/yan/Work/active_mapping/SeRFnav-goat/habitat-lab/habitat-lab/habitat/core/embodied_task.py", line 249, in __init__
    self._init_entities(
  File "/home/yan/Work/active_mapping/SeRFnav-goat/habitat-lab/habitat-lab/habitat/core/embodied_task.py", line 274, in _init_entities
    entities[entity_name] = entity_type(
  File "/home/yan/Work/active_mapping/SeRFnav-goat/habitat-lab/habitat-lab/habitat/tasks/nav/object_nav_task.py", line 128, in __init__
    super().__init__(config=config)
  File "/home/yan/Work/active_mapping/SeRFnav-goat/habitat-lab/habitat-lab/habitat/core/simulator.py", line 91, in __init__
    self.observation_space = self._get_observation_space(*args, **kwargs)
  File "/home/yan/Work/active_mapping/SeRFnav-goat/habitat-lab/habitat-lab/habitat/tasks/nav/object_nav_task.py", line 141, in _get_observation_space
    self._dataset.category_to_task_category_id.values()
AttributeError: 'GoatDatasetV1' object has no attribute 'category_to_task_category_id'

It seems that the habitat-lab version isn't correct, or there might be some undeclared modifications to the source code of habitat-lab v0.2.3. Could you provide further guidance on this? @Ram81 @mukulkhanna @gchhablani

from goat-bench.

I installed the packages according to the README, but encountered some issues.

I needed to additionally install the lavis and vc_models packages. Also, even after installing the above packages, I still receive the following error when I try to run the train script.

MissingConfigException: In 'ver_goat.yaml': Could not load 'habitat_baselines/rl/policy/obs_transforms/resize'.

Is v0.2.3 the correct version of habitat-lab?

If you have the correct installation procedure with modifications, could you let me know? I look forward to your response.

I faced the same issues.

    from vc_models.models.vit import model_utils
ModuleNotFoundError: No module named 'vc_models'

How did you install the vc_models package? @miyatai2 @Ram81 @mukulkhanna @gchhablani @Cassie-Lim

from goat-bench.

I used the vc_models from https://github.com/facebookresearch/eai-vc @yusirhhh

from goat-bench.

Thanks for your kind words. I have fixed the package issue with vc_models. However, when I tried to install salesforce-lavis using pip, I encountered the following error:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.                                                                                           
pydantic-core 2.14.6 requires typing-extensions!=4.7.0,>=4.6.0, but you have typing-extensions 4.4.0 which is incompatible.          
openai 1.4.0 requires typing-extensions<5,>=4.5, but you have typing-extensions 4.4.0 which is incompatible.                         
vc-models 0.1 requires timm==0.6.11, but you have timm 0.4.12 which is incompatible.

What's the version of Python and salesforce-lavis you used, @miyatai2?

from goat-bench.

Thanks for your kind words. I have fixed the package issue with vc_models. However, when I tried to install salesforce-lavis using pip, I encountered the following error:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.                                                                                           
pydantic-core 2.14.6 requires typing-extensions!=4.7.0,>=4.6.0, but you have typing-extensions 4.4.0 which is incompatible.          
openai 1.4.0 requires typing-extensions<5,>=4.5, but you have typing-extensions 4.4.0 which is incompatible.                         
vc-models 0.1 requires timm==0.6.11, but you have timm 0.4.12 which is incompatible.

What's the version of Python and salesforce-lavis you used, @miyatai2?

I use salesforce-lavis 1.0.2 and python 3.8.18.

from goat-bench.

I commented out:

- /habitat_baselines/rl/policy/obs_transforms:
  - resize

in the ver_goat.yaml and it worked. However, I'm not sure if this is the correct solution.

Additionally, I encountered the following error during execution:

FileNotFoundError: [Errno 2] No such file or directory: 'data/datasets/languagenav/hm3d/v5_final/train_embeddings/clip_train_embeddings.pkl

Could I possibly get the relevant data?

I can't find 'clip_train_embeddings.pkl' either. Have you solved it? @miyatai2

from goat-bench.

I have not resolved it yet.

from goat-bench.

Hi @miyatai2 @yusirhhh @Cassie-Lim @74284853 , apologies for the delay here. We have fixed a bunch of issues related to configs and installation and pushed to the main branch today. Please let me know if you are facing issues even after you rebase with latest changes from main.

@74284853 the missing cache files are available here.

from goat-bench.

@Ram81 FileNotFoundError: [Errno 2] No such file or directory: 'data/goat-assets/goal_cache/language_nav/val_seen_bert_embedding.pkl'

The val_seen_bert_embedding.pkl is not in https://github.com/Ram81/goat-bench?tab=readme-ov-file#cached-embeddings-and-model-checkpoints

from goat-bench.

@yusirhhh I could download the cache files with the following command:
git lfs clone https://huggingface.co/datasets/axel81/goat-bench data/goat-assets/

from goat-bench.

@Ram81 Thank you for the support! I was able to successfully execute the code with the latest updates/fixes. Since the problem has been resolved, I will be closing this issue now.

from goat-bench.