[Feature] Support Evaluation in VL-LN Bench #193

0309hws · 2025-12-11T15:40:49Z

📊 Extended Evaluation Support

Support IION evaluation
Support Object evaluation

…nd agent created

internnav/agent/dialog_agent.py

internnav/habitat_vlln_extensions/simple_npc/get_description.py

internnav/habitat_extensions/vlln/simple_npc/get_description.py

scripts/eval/bash/srun_eval_dialog.sh

scripts/eval/configs/habitat_dialog_cfg.py

scripts/eval/configs/instance_dialog.yaml

internnav/habitat_vlln_extensions/simple_npc/__init__.py

internnav/habitat_extensions/vlln/utils/dialog_dataset.py

internnav/habitat_extensions/vlln/utils/dialog_episodes.py

update folder structure

internnav/agent/dialog_agent.py

internnav/habitat_extensions/vlln/README.md

internnav/habitat_extensions/vlln/simple_npc/get_description.py

internnav/habitat_extensions/vlln/utils/dialog_dataset.py

internnav/habitat_extensions/vln/measures.py

requirements/core_requirements.txt

scripts/eval/configs/gen_videos.yaml

scripts/eval/configs/habitat_dual_system_cfg.py

scripts/eval/configs/objectnav_hm3d.yaml

kew6688 and others added 21 commits November 3, 2025 08:11

update vln yaml; fix import agent

0112c69

update habitat, using evaluator and config; env and agent is WIP

05ea2a3

add distributed_base evaluator

9902401

Habitat env applied, distributed evaluator applied; clean evaluator a…

0d00014

…nd agent created

fix observation issues

7e25e72

update new register name; tiny fix on style

2b0eb8b

latest tested

b414ba3

delete temp agent; rename default evaluator for habitat

99adf73

update slurm bash

75b38a7

merge to main

dcf7ee5

update readme

08bb9c3

fix init dist print

cde84b3

fix eval config; fix local rank to rank

c89723d

update init distributed mode if condition

7836276

update dist for dlc

dac13e1

fix bug in evaluator

d8734c7

[test] dialog+object

fb21071

merge dev

4ecb613

[Feature] Add testing code for VLLN

cd00d1e

[Feature] Add testing code for VLLN

ee69a31

[Feature] Add testing code for VLLN

1d5a16a

Tai-Wang requested review from kew6688 and yuqiang-yang December 14, 2025 08:34

kew6688 added 5 commits December 16, 2025 06:57

fix bugs; refactor env

8607b2d

Merge branch 'dev' into code_review

7aa020d

update code, merge dev; fix bug

c1e59fb

add raise flag in agent

7058317

update a readme for vlln

f6955a1

yuqiang-yang requested changes Dec 16, 2025

View reviewed changes

kew6688 reviewed Dec 17, 2025

View reviewed changes

scripts/eval/configs/instance_dialog.yaml Outdated Show resolved Hide resolved

Tai-Wang reviewed Dec 22, 2025

View reviewed changes

internnav/habitat_vlln_extensions/simple_npc/__init__.py Outdated Show resolved Hide resolved

Tai-Wang reviewed Dec 22, 2025

View reviewed changes

internnav/habitat_extensions/vlln/utils/dialog_dataset.py Outdated Show resolved Hide resolved

Tai-Wang reviewed Dec 22, 2025

View reviewed changes

internnav/habitat_extensions/vlln/utils/dialog_dataset.py Outdated Show resolved Hide resolved

Tai-Wang reviewed Dec 22, 2025

View reviewed changes

internnav/habitat_extensions/vlln/utils/dialog_episodes.py Outdated Show resolved Hide resolved

kew6688 and others added 3 commits December 22, 2025 06:01

update folder structure

a7b15eb

Merge pull request #2 from kew6688/fix_habitat_extensions

895710d

update folder structure

[FIX] delete files

8445356

kew6688 approved these changes Dec 23, 2025

View reviewed changes

[FIX] docstrings

0c589aa