[Question]: Technical details on the synchronization and conversion process between GLB (Habitat) and USD (Isaac Sim) assets

[QiyuHu0412]

Question

I noticed that InternNav supports both Habitat (using .glb format) and Isaac Sim (using .usd format). In the codebase, these are linked via the same scene_id. It appears that the assets are not converted at runtime but are pre-processed offline.

Questions To better understand the consistency between the two simulation environments, I would like to ask about the offline conversion pipeline:

Conversion Toolchain: Which tools or scripts are used to convert the assets (e.g., from .glb to .usd)? Are these scripts available in this repo, or is the process based on NVIDIA Omniverse's Asset Converter?

Consistency Alignment: How do you ensure that the visual scales, coordinate systems (e.g., Y-up vs. Z-up), and origin points remain identical across Habitat and Isaac Sim?

Physical Properties: Beyond visual meshes, how are the physical properties (collision shapes, mass, friction) synchronized between the two formats to ensure that a policy trained in Habitat behaves predictably in Isaac Sim?

Reason for Inquiry Understanding this process is crucial for users who wish to import their own custom scenes into the InternNav framework while maintaining cross-platform fidelity.

Thank you for your great work on this project!

[CrystalSixone]

Hi! Thanks for your questions. Please find our responses below:

1. Conversion toolchain: We use the official download_mp.py script provided by the Matterport3D dataset to download the matterport_mesh. The downloaded OBJ files are then converted based on the MeshConverter class in Isaac Lab. During direct OBJ conversion, we observed holes in the floor mesh, which could cause the robot to step into empty space. Therefore, we manually filled and repaired the floor planes to ensure safe navigation.

Below is the conversion script used in our earlier experiments. It may depend on an older version of Isaac Lab ( < 1.4 ) and is shared for reference purposes only.
(1) convert_mp3d_obj_to_usd.py

# -*- coding=utf-8 -*-
"""
Convert Matterport3D OBJ files to USD format.

This script converts all .obj files under the Matterport3D/data/v1/scans/xxx/matterport_mesh 
directory to .usd files with proper transformations for scale and rotation.
"""

import time
import argparse
import shutil
import os
import asyncio

import isaacsim
from omni.isaac.kit import SimulationApp

# Argument parser
parser = argparse.ArgumentParser(description="Process all OBJ files in the Matterport3D directory.")
parser.add_argument("--data_dir", default="data/v1/scans", type=str, help="The path to the matterport3D dictionary.")
parser.add_argument(
    "--make-instanceable",
    action="store_true",
    default=False,
    help="Make the asset instanceable for efficient cloning.",
)
parser.add_argument(
    "--collision-approximation",
    type=str,
    default="convexDecomposition",
    choices=["convexDecomposition", "convexHull", "none"],
    help="The method used for approximating collision mesh.",
)
parser.add_argument(
    "--mass",
    type=float,
    default=None,
    help="The mass (in kg) to assign to the converted asset.",
)
parser.add_argument(
    "--scan",
    type=str,
    default=None,
    help="The specific scan in Matterport3D to research",
)
parser.add_argument(
    "--headless",
    action="store_true",
    default=False,
    help="Headless mode (without GUI)"
)

# Initialize the simulation app
args_cli = parser.parse_args()
simulation_app = SimulationApp(vars(args_cli))

# Import Isaac Sim modules
import omni.usd
import omni.isaac.core.utils.prims as prim_utils
import omni.isaac.lab.sim as sim_utils
from omni.isaac.lab.sim import SimulationContext, SimulationCfg
import omni.kit.commands
import omni.isaac.core.utils.stage as stage_utils
import omni.kit.app

from omni.isaac.lab.sim.converters import MeshConverter, MeshConverterCfg
from omni.isaac.lab.sim.schemas import schemas_cfg
from omni.isaac.lab.utils.assets import check_file_path
from omni.isaac.lab.utils.dict import print_dict
from omni.isaac.lab.sim.converters.asset_converter_base import AssetConverterBase
from omni.isaac.core.utils.extensions import enable_extension
from omni.isaac.lab.sim.converters.mesh_converter_cfg import MeshConverterCfg
from omni.isaac.lab.sim.schemas import schemas
from omni.isaac.lab.sim.utils import export_prim_to_file

from pxr import UsdGeom, Usd, Sdf, Gf, UsdPhysics, UsdUtils


def copy_and_rename_obj_file(obj_file_path):
    """Rename OBJ file with prefix to avoid converter issues with files starting with numbers."""
    obj_dir = os.path.dirname(obj_file_path)
    obj_file_name = os.path.basename(obj_file_path)
    
    modified_obj_file_name = "isaacsim_" + obj_file_name
    modified_obj_file_path = os.path.join(obj_dir, modified_obj_file_name)
    
    shutil.copyfile(obj_file_path, modified_obj_file_path)
    
    return modified_obj_file_path


class MP3DMeshConverter(MeshConverter):
    """Custom mesh converter for Matterport3D assets."""
    
    cfg: MeshConverterCfg

    def __init__(self, cfg: MeshConverterCfg):
        """Initialize the converter with configuration."""
        super().__init__(cfg=cfg)

    @staticmethod
    async def _convert_mesh_to_usd(
        in_file: str, out_file: str, prim_path: str = "/World", load_materials: bool = True
    ) -> bool:
        """
        Convert mesh from supported file types to USD.
        
        Args:
            in_file: The file to convert.
            out_file: The path to store the output file.
            prim_path: The prim path of the mesh.
            load_materials: Enable attaching materials from input file.
            
        Returns:
            True if the conversion succeeds.
        """
        enable_extension("omni.kit.asset_converter")
        enable_extension("omni.usd.metrics.assembler")

        import omni.kit.asset_converter
        import omni.usd
        from omni.metrics.assembler.core import get_metrics_assembler_interface

        # Create and configure converter context
        converter_context = omni.kit.asset_converter.AssetConverterContext()
        converter_context.ignore_materials = not load_materials
        converter_context.ignore_animations = True
        converter_context.ignore_camera = True
        converter_context.ignore_light = True
        converter_context.merge_all_meshes = False 
        converter_context.use_meter_as_world_unit = True
        converter_context.baking_scales = True
        converter_context.use_double_precision_to_usd_transform_op = True

        # Create and run converter task
        instance = omni.kit.asset_converter.get_instance()
        out_file_non_metric = out_file.replace(".usd", "_non_metric.usd")
        task = instance.create_converter_task(in_file, out_file_non_metric, None, converter_context)
        
        success = True
        while True:
            success = await task.wait_until_finished()
            if not success:
                await asyncio.sleep(0.1)
            else:
                break

        # Create temporary stage with proper setup
        temp_stage = Usd.Stage.CreateInMemory()
        UsdGeom.SetStageUpAxis(temp_stage, UsdGeom.Tokens.z)
        UsdGeom.SetStageMetersPerUnit(temp_stage, 1.0)
        UsdPhysics.SetStageKilogramsPerUnit(temp_stage, 1.0)

        base_prim = temp_stage.DefinePrim(prim_path, "Xform")
        prim = temp_stage.DefinePrim(f"{prim_path}/geometry", "Xform")
        prim.GetReferences().AddReference(out_file_non_metric)
        
        cache = UsdUtils.StageCache.Get()
        cache.Insert(temp_stage)
        stage_id = cache.GetId(temp_stage).ToLongInt()
        get_metrics_assembler_interface().resolve_stage(stage_id)
        temp_stage.SetDefaultPrim(base_prim)

        # Update prim parameters
        mp3d_prim = prim
        
        rotate_units_resolve_attr = mp3d_prim.GetAttribute('xformOp:rotateX:unitsResolve')
        if not rotate_units_resolve_attr:
            rotate_units_resolve_attr = mp3d_prim.CreateAttribute('xformOp:rotateX:unitsResolve', Sdf.ValueTypeNames.Int)
        rotate_units_resolve_attr.Set(0)

        scale_units_resolve_attr = mp3d_prim.GetAttribute('xformOp:scale:unitsResolve')
        if not scale_units_resolve_attr:
            scale_units_resolve_attr = mp3d_prim.CreateAttribute('xformOp:scale:unitsResolve', Sdf.ValueTypeNames.Float3)
        scale_units_resolve_attr.Set(Gf.Vec3f(1, 1, 1))

        # Configure collision properties
        geometry_prim_path = f"{prim_path}/geometry"
        if not temp_stage.GetPrimAtPath(geometry_prim_path):
            raise ValueError(f"Prim path '{geometry_prim_path}' does not exist.")
        collider_cfg = sim_utils.CollisionPropertiesCfg(collision_enabled=True)
        sim_utils.define_collision_properties(geometry_prim_path, collider_cfg, stage=temp_stage)

        temp_stage.Export(out_file)
        return success


def process_obj_file(obj_file_path):
    """Process a single OBJ file and convert it to USD."""
    if not os.path.isabs(obj_file_path):
        obj_file_path = os.path.abspath(obj_file_path)
    
    if not check_file_path(obj_file_path):
        raise ValueError(f"Invalid mesh file path: {obj_file_path}")

    dest_path = os.path.splitext(obj_file_path)[0] + ".usd"
    usd_dir = os.path.dirname(dest_path)
    usd_file_name = os.path.basename(dest_path)

    # Configure mass properties
    mass_props = schemas_cfg.MassPropertiesCfg(mass=args_cli.mass) if args_cli.mass is not None else None
    rigid_props = schemas_cfg.RigidBodyPropertiesCfg() if args_cli.mass is not None else None

    # Configure collision properties
    collision_props = schemas_cfg.CollisionPropertiesCfg(collision_enabled=args_cli.collision_approximation != "none")

    # Create mesh converter config
    mesh_converter_cfg = MeshConverterCfg(
        mass_props=mass_props,
        rigid_props=rigid_props,
        collision_props=collision_props,
        asset_path=obj_file_path,
        force_usd_conversion=True,
        usd_dir=usd_dir,
        usd_file_name=usd_file_name,
        make_instanceable=args_cli.make_instanceable,
        collision_approximation=args_cli.collision_approximation,
    )

    print("-" * 80)
    print(f"Input Mesh file: {obj_file_path}")
    print("Mesh importer config:")
    print_dict(mesh_converter_cfg.to_dict(), nesting=0)
    print("-" * 80)

    # Create converter and process
    mesh_converter = MP3DMeshConverter(mesh_converter_cfg)
    
    print("Mesh importer output:")
    print(f"Generated USD file: {mesh_converter.usd_path}")
    print("-" * 80)


def main():
    """Main function to process all OBJ files in the specified directory."""
    if args_cli.scan is not None:
        args_cli.data_dir = os.path.join(args_cli.data_dir, args_cli.scan)
                
    for root, dirs, files in os.walk(args_cli.data_dir):
        for file in files:
            # if file.endswith(".obj") and "isaacsim_" not in file:
            if file.endswith(".obj"):
                print(f"Processing file: {file}")
                obj_file_path = os.path.join(root, file)
                modified_obj_file_path = copy_and_rename_obj_file(obj_file_path)
                process_obj_file(modified_obj_file_path)
    
    print("========== Conversion finished! =============")


if __name__ == "__main__":
    main()

(2) update_texture_paths.py

# -*- coding=utf-8 -*-
"""
Update texture paths in USD files to relative paths.

This script processes USD files and converts absolute texture paths to relative paths,
making them portable across different systems.
"""

import os
import shutil
from pxr import Usd, UsdShade, Sdf

def update_usd_texture_path(usd_path):
    """
    Update texture paths in a USD file to use relative paths.
    
    Args:
        usd_path: Absolute path to the USD file.
    """
    stage = Usd.Stage.Open(usd_path)
    all_prims = [prim for prim in stage.Traverse()]
    all_prim_paths = [prim.GetPath() for prim in all_prims]

    for prim_idx, prim in enumerate(all_prims):
        prim_path = all_prim_paths[prim_idx]
        
        # Skip prims that don't contain texture references
        if 'jpg' not in prim_path.pathString:
            continue
        
        if prim.IsA(UsdShade.Shader):
            shader = UsdShade.Shader(prim)
            shader_inputs = shader.GetInputs()

            for shader_input in shader_inputs:
                if shader_input.GetBaseName() == "diffuse_texture":
                    current_value = shader_input.Get()
                    print(f"Shader {prim.GetPath()} current texture path: {current_value}")

                    current_path = current_value.path
                    
                    # Convert to relative path
                    new_path = '/'.join(current_path.split('/')[-2:])
                    new_path = './' + new_path 

                    new_texture_path = Sdf.AssetPath(new_path)
                    shader_input.Set(new_texture_path)

                    print(f"Shader {prim.GetPath()} texture path updated to: {new_texture_path.path}")

    stage.GetRootLayer().Save()
    print(f"USD file texture paths updated: {usd_path}")


def update_usd_texture_paths_batch(src_dir, target_filename="fixed.usd"):
    """
    Batch update texture paths for all USD files in directory tree.
    
    Args:
        src_dir: Root directory to search for USD files.
        target_filename: Name of USD files to process (default: "fixed.usd").
    """
    for root, dirs, files in os.walk(src_dir):
        for file in files:
            if file == target_filename:
                src_file_path = os.path.join(root, file)
                print(f"\nProcessing: {src_file_path}")
                update_usd_texture_path(src_file_path)


if __name__ == '__main__':
    # Example usage:
    # Update texture paths for all USD files in the scans directory
    update_usd_texture_paths_batch(
        src_dir='Matterport3D/data/v1/scans',
        target_filename='isaacsim_*.usd'  # Adjust the filename pattern as needed
    )

2. Consistency alignment: Habitat and Isaac Sim use different coordinate systems. The code for mapping coordinates from Habitat to Isaac Sim can be found here:
   

   InternNav/internnav/env/utils/episode_loader/dataset_utils.py

   Lines 592 to 597 in b654fbb

   	if dataset_type == 'mp3d':
   	x, z, y = item['start_position']
   	new_item['start_position'] = [x, -y, z]
   	r1, r2, r3, r4 = item['start_rotation']
   	new_item['start_rotation'] = transform_rotation_z_90degrees([-r4, r1, r3, -r2])
   	new_item['reference_path'] = [[x, -y, z] for x, z, y in item['reference_path']]

3. Physical properties: We use the default physical settings provided by Isaac Sim. The specific parameters can be found in the demo launch configuration.

Thank you very much for your interest in our work and for your valuable suggestions!