Add tutorial script and documentation for Tiled Camera View in Visualizers

docs/source/how-to/index.rst

@@ -101,6 +101,17 @@ Isaac Lab.
     draw_markers
 
 
+Using Visualizers
+-----------------
+
+This guide demonstrates how to use visualizer-specific views for monitoring and debugging Isaac Lab environments.
+
+.. toctree::
+    :maxdepth: 1
+
+    visualizer_tiled_camera
+
+
 Working with Simulation Data
 ----------------------------
 

docs/source/how-to/visualizer_tiled_camera.rst

@@ -0,0 +1,159 @@
+.. _how-to-visualizer-tiled-camera:
+
+Using Visualizer Tiled Cameras
+==============================
+
+.. currentmodule:: isaaclab
+
+For general visualizer documentation, see :doc:`/source/overview/core-concepts/visualization`.
+
+The visualizer tiled camera view is a live monitoring and debugging tool. It opens a
+non-interactive panel in the Kit or Newton visualizer and streams tiled camera views
+across all selected environments. Note, this is separate from tiled camera observations
+used by policies.
+
+This guide is accompanied by the ``run_tiled_camera_visualizer.py`` script in the
+``IsaacLab/scripts/tutorials/07_visualizers`` directory.
+
+Running this script demonstrates two ways to use tiled cameras:
+
+- configured tiled cameras pointed at and following moving Anymal-D robots shown in the Kit visualizer
+- streaming from existing wrist-mounted robot cameras shown in the Newton visualizer
+
+Note: Visualizer tiled cameras are currently supported only in the Kit and Newton visualizers.
+Either visualizer can be used to run either example.
+
+.. dropdown:: Code for run_tiled_camera_visualizer.py
+   :icon: code
+
+   .. literalinclude:: ../../../scripts/tutorials/07_visualizers/run_tiled_camera_visualizer.py
+      :language: python
+      :emphasize-lines: 74-81,89-97
+      :linenos:
+
+
+Example One: Following Anymal-D Robots
+--------------------------------------
+
+The Kit Visualizer shows the tiled camera view in a separate tab inside the main
+Viewport window. The highlighted tab area in the figures below shows where to
+toggle between the interactive viewport and the visualizer tiled camera view.
+
+.. figure:: ../_static/visualizers/kit_viz_anymal_iteractive_view.png
+   :width: 100%
+   :alt: Kit visualizer interactive viewport for Anymal-D robots
+
+   Kit visualizer showing the default interactive viewport.
+
+.. figure:: ../_static/visualizers/kit_viz_anymal_tiled_view.png
+   :width: 100%
+   :alt: Kit visualizer tiled camera view for Anymal-D robots
+
+   Kit visualizer showing the tiled camera view generated for selected Anymal-D
+   robots.
+
+Note, you can also display the main visualizer camera and the tiled camera view side by
+side for dual monitoring.
+
+To run the tutorial with the args for this example, use:
+
+.. code-block:: bash
+
+   ./isaaclab.sh -p scripts/tutorials/07_visualizers/run_tiled_camera_visualizer.py \
+     --enable_cameras \
+     --task Isaac-Velocity-Rough-Anymal-D-v0 \
+     --num_envs 256 \
+     --viz kit
+
+Within the script, you’ll find the ``KitVisualizerCfg`` configuration used to
+generate this example. You can use this config as a template for your own use
+cases.
+
+In this example, a set of cameras is created to point toward each robot's base
+prim and follow its motion. The camera's position, relative to the prim, is set
+by the ``tiled_cam_eye`` field of ``KitVisualizerCfg``. For this demo, the
+camera is offset by ``(3.0, 3.0, 3.0)`` from each robot base. If you change ``tiled_cam_eye``
+(for example, to ``(0, 0, 5)``), the panel will show a top-down view instead.
+
+In this example, there are 256 total environments, and we randomly sample 36 to stream to the
+tiled camera view.
+
+Also note that the Kit visualizer tiled camera view requires passing
+``--enable_cameras`` CLI arg.
+
+
+Example Two: Streaming from Robot-Mounted Cameras
+-------------------------------------------------
+
+The Newton visualizer provides a tiled camera view in a lightweight OpenGL window.
+Use the highlighted ``Tiled Camera View`` dropdown in the left-hand sidebar to
+show or hide the tiled camera panel.
+
+.. figure:: ../_static/visualizers/newton_viz_galbot_interactive_view.png
+   :width: 100%
+   :alt: Newton visualizer interactive view for the Galbot cube stacking environment
+
+   Newton visualizer showing the default interactive viewport.
+
+.. figure:: ../_static/visualizers/newton_viz_galbot_tiled_view.png
+   :width: 100%
+   :alt: Newton visualizer tiled camera view for Galbot wrist cameras
+
+   Newton visualizer showing the selected Galbot head-camera feeds in the tiled
+   camera panel.
+
+In this example, we use the Galbot cube stacking environment, which comes with
+built-in wrist-mounted cameras. This setup provides an egocentric view of the
+gripper, table, and cubes in each selected environment.
+
+To launch this example, run:
+
+.. code-block:: bash
+
+   ./isaaclab.sh -p scripts/tutorials/07_visualizers/run_tiled_camera_visualizer.py \
+     --task Isaac-Stack-Cube-Galbot-Left-Arm-Gripper-Visuomotor-v0 \
+     --num_envs 25 \
+     --viz newton
+
+Within the script, the ``NewtonVisualizerCfg`` is configured to stream images from the
+existing camera sensor located at
+``/World/envs/env_.*/Robot/head_camera_sim_view_frame/head_camera``. This path
+points to the head camera, but you can edit the ``tiled_cam_prim_path``
+field of ``NewtonVisualizerCfg`` in the script to show a different existing camera if
+needed.
+
+In this demo, 25 environments are simulated, and 12 camera feeds are shown in the tiled panel by default.
+
+
+Configuration notes
+-------------------
+
+To customize tiled camera behavior, edit the highlighted ``VisualizerCfg`` fields in
+``run_tiled_camera_visualizer.py``:
+
+* For generated cameras, ``tiled_cam_target_prim_path`` chooses the followed prim and
+  ``tiled_cam_eye`` sets the camera offset from that prim.
+* For existing scene cameras, ``tiled_cam_prim_path`` must match an Isaac Lab
+  :class:`~isaaclab.sensors.Camera` sensor in the selected task.
+* ``tiled_cam_num`` controls how many environment tiles are shown.
+
+
+Troubleshooting
+---------------
+
+* If a generated view fails with a missing prim error, check that
+  ``tiled_cam_target_prim_path`` resolves in each selected environment. Common template
+  forms include ``/World/envs/*/...`` and ``/World/envs/env_.*/...``.
+* If an existing-camera view reports that no Isaac Lab camera owns the prim, check that
+  ``tiled_cam_prim_path`` matches a :class:`~isaaclab.sensors.Camera` sensor in the task.
+* If ``rerun`` or ``viser`` is selected, use ``--viz kit`` or ``--viz newton`` instead.
+  The tiled camera panel is currently implemented for Kit and Newton.
+* If the view is too expensive, reduce ``tiled_cam_num``, ``--num_envs``, or the camera
+  resolution. The visualizer caps the tiled panel at 100 tiles.
+
+
+See also
+--------
+
+* :doc:`/source/overview/core-concepts/visualization` - visualizer configuration and UI controls.
+* :doc:`/source/how-to/configure_rendering` - selecting rendering presets and quality modes.

docs/source/overview/core-concepts/visualization.rst

@@ -200,6 +200,9 @@ Camera Modes
 To configure camera modes, including launching a tiled camera view, edit the fields described below in the
 ``VisualizerCfg`` config class.
 
+For runnable Kit and Newton examples that use generated and existing tiled cameras,
+see :doc:`/source/how-to/visualizer_tiled_camera`.
+
 The default visualizer camera mode is interactive, with ``eye`` and ``lookat`` specifying the initial pose.
 Kit and Newton visualizers can also run additional tiled camera image panels.
 

scripts/tutorials/07_visualizers/run_tiled_camera_visualizer.py

@@ -0,0 +1,168 @@
+# Copyright (c) 2022-2026, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
+# All rights reserved.
+#
+# SPDX-License-Identifier: BSD-3-Clause
+
+"""
+This script demonstrates the visualizer tiled camera panel.
+
+.. code-block:: bash
+
+    # Kit visualizer tiled camera panel
+    ./isaaclab.sh -p scripts/tutorials/07_visualizers/run_tiled_camera_visualizer.py \
+        --task Isaac-Velocity-Rough-Anymal-D-v0 --num_envs 256 --viz kit
+
+    # Newton visualizer tiled camera panel
+    ./isaaclab.sh -p scripts/tutorials/07_visualizers/run_tiled_camera_visualizer.py \
+        --task Isaac-Stack-Cube-Galbot-Left-Arm-Gripper-Visuomotor-v0 --num_envs 25 --viz newton
+
+"""
+
+from __future__ import annotations
+
+import argparse
+import contextlib
+import sys
+
+import gymnasium as gym
+import torch
+
+import isaaclab_tasks  # noqa: F401
+
+with contextlib.suppress(ImportError):
+    import isaaclab_tasks_experimental  # noqa: F401
+from isaaclab_tasks.utils import (
+    add_launcher_args,
+    fold_preset_tokens,
+    launch_simulation,
+    resolve_task_config,
+    setup_preset_cli,
+)
+
+KIT_DEFAULT_TASK = "Isaac-Velocity-Rough-Anymal-D-v0"
+NEWTON_DEFAULT_TASK = "Isaac-Stack-Cube-Galbot-Left-Arm-Gripper-Visuomotor-v0"
+SUPPORTED_TILED_VISUALIZERS = {"kit", "newton"}
+UNSUPPORTED_TILED_VISUALIZERS = {"rerun", "viser"}
+
+
+def _requested_visualizers(args_cli: argparse.Namespace) -> list[str]:
+    """Return requested visualizers, defaulting to Kit for this tutorial."""
+    visualizers = args_cli.visualizer or ["kit"]
+    visualizers = [str(visualizer).lower() for visualizer in visualizers]
+
+    if "none" in visualizers:
+        raise ValueError("This demo requires a tiled-camera visualizer. Use '--viz kit' or '--viz newton'.")
+    unsupported = sorted(set(visualizers) & UNSUPPORTED_TILED_VISUALIZERS)
+    if unsupported:
+        raise ValueError(
+            "The visualizer tiled camera panel is only implemented for Kit and Newton. "
+            f"Unsupported selection: {unsupported}."
+        )
+    unknown = sorted(set(visualizers) - SUPPORTED_TILED_VISUALIZERS)
+    if unknown:
+        raise ValueError(f"Unknown visualizer selection for this demo: {unknown}.")
+    return visualizers
+
+
+def _make_kit_visualizer_cfg(args_cli: argparse.Namespace):
+    """Create generated tiled cameras that follow each Anymal-D base."""
+    from isaaclab_visualizers.kit import KitVisualizerCfg
+
+    # Generated Kit cameras are not found by the task config scan.
+    args_cli.enable_cameras = True
+
+    visualizer_cfg = KitVisualizerCfg()
+    visualizer_cfg.tiled_cam_view = True
+    visualizer_cfg.tiled_cam_num = 36
+    visualizer_cfg.tiled_cam_prim_path = None
+    # Here is an alternative eye position for a top down view
+    # visualizer_cfg.tiled_cam_eye = (0.0, 0.0, 5.0)
+    visualizer_cfg.tiled_cam_eye = (3.0, 3.0, 3.0)
+    visualizer_cfg.tiled_cam_target_prim_path = "/World/envs/*/Robot/base"
+    return visualizer_cfg
+
+
+def _make_newton_visualizer_cfg():
+    """Stream existing Galbot wrist cameras into the Newton tiled camera panel."""
+    from isaaclab_visualizers.newton import NewtonVisualizerCfg
+
+    visualizer_cfg = NewtonVisualizerCfg()
+    visualizer_cfg.tiled_cam_view = True
+    visualizer_cfg.tiled_cam_num = 12
+    # Here are other robot mounted camera options for this environment
+    # visualizer_cfg.tiled_cam_prim_path = "/World/envs/env_.*/Robot/left_arm_camera_sim_view_frame/left_camera"
+    # visualizer_cfg.tiled_cam_prim_path = "/World/envs/env_.*/Robot/right_arm_camera_sim_view_frame/right_camera"
+    visualizer_cfg.tiled_cam_prim_path = "/World/envs/env_.*/Robot/head_camera_sim_view_frame/head_camera"
+    visualizer_cfg.tiled_cam_eye = None
+    visualizer_cfg.tiled_cam_target_prim_path = None
+    return visualizer_cfg
+
+
+def _configure_visualizers(env_cfg, args_cli: argparse.Namespace) -> None:
+    """Attach tiled camera visualizer configs to the environment simulation config."""
+    visualizers = _requested_visualizers(args_cli)
+    args_cli.visualizer = visualizers
+    env_cfg.sim.visualizer_cfgs = [
+        _make_kit_visualizer_cfg(args_cli) if visualizer == "kit" else _make_newton_visualizer_cfg()
+        for visualizer in visualizers
+    ]
+
+
+def _resolve_task(args_cli: argparse.Namespace) -> str:
+    """Resolve the task for the selected visualizer."""
+    if args_cli.task is not None:
+        return args_cli.task
+    if "newton" in _requested_visualizers(args_cli):
+        return NEWTON_DEFAULT_TASK
+    return KIT_DEFAULT_TASK
+
+
+# add argparse arguments
+parser = argparse.ArgumentParser(description="Showcase the Kit/Newton visualizer tiled camera panel.")
+parser.add_argument("--num_envs", type=int, default=None, help="Number of environments to simulate.")
+parser.add_argument("--task", type=str, default=None, help="Name of the task.")
+# append AppLauncher cli args
+add_launcher_args(parser)
+args_cli, hydra_args = setup_preset_cli(parser)
+args_cli.task = _resolve_task(args_cli)
+sys.argv = [sys.argv[0]] + fold_preset_tokens(hydra_args)
+
+
+def main():
+    """Run a random-action environment with a tiled camera visualizer."""
+    # parse configuration via Hydra (supports preset selection, e.g. presets=newton_mjwarp)
+    env_cfg, _ = resolve_task_config(args_cli.task, "")
+    _configure_visualizers(env_cfg, args_cli)
+
+    with launch_simulation(env_cfg, args_cli):
+        # override with CLI arguments
+        env_cfg.scene.num_envs = args_cli.num_envs if args_cli.num_envs is not None else env_cfg.scene.num_envs
+        env_cfg.sim.device = args_cli.device if args_cli.device is not None else env_cfg.sim.device
+
+        # create environment
+        env = gym.make(args_cli.task, cfg=env_cfg)
+
+        # print info (this is vectorized environment)
+        print(f"[INFO]: Gym observation space: {env.observation_space}")
+        print(f"[INFO]: Gym action space: {env.action_space}")
+        env.reset()
+
+        # keep stepping until all visualizer windows have been closed
+        sim = env.unwrapped.sim
+        if not sim.visualizers:
+            print("[WARN]: No visualizers found. Exiting.")
+            env.close()
+            return
+
+        while True:
+            if sim.visualizers and not any(v.is_running() and not v.is_closed for v in sim.visualizers):
+                break
+            with torch.inference_mode():
+                actions = 2 * torch.rand(env.action_space.shape, device=env.unwrapped.device) - 1
+                env.step(actions)
+
+        env.close()
+
+
+if __name__ == "__main__":
+    main()

source/isaaclab/changelog.d/mtrepte-tiled-camera-visualizer-tutorial.rst

@@ -0,0 +1,5 @@
+Changed
+^^^^^^^
+
+* Updated the visualizer tiled camera tutorial to show generated Kit cameras and
+  existing Newton robot-mounted camera streams with matching documentation figures.

source/isaaclab/isaaclab/envs/utils/camera_view.py

@@ -223,9 +223,18 @@ def prim_world_positions(
 ) -> torch.Tensor:
     """Return world-space translations for concrete prim paths resolved from env ids.
 
-    Uses ``FrameView`` first so PhysX/Fabric-backed transforms are current; falls
-    back to USD only if the backend view cannot be constructed.
+    Uses scene articulation state first when the target is an asset/body path,
+    then falls back to ``FrameView`` and USD for arbitrary prim paths.
     """
+    from pxr import UsdGeom
+
+    from isaaclab.sim.views import FrameView
+
+    if scene is not None:
+        positions_tensor = _scene_articulation_positions(scene, prim_path_template, env_indices)
+        if positions_tensor is not None:
+            return positions_tensor
+
     xform_cache = UsdGeom.XformCache()
     positions = []
     try:
@@ -241,11 +250,6 @@ def prim_world_positions(
     except Exception:
         positions.clear()
 
-    if scene is not None:
-        positions_tensor = _scene_articulation_positions(scene, prim_path_template, env_indices)
-        if positions_tensor is not None:
-            return positions_tensor
-
     for env_id in env_indices:
         prim_path = env_path_from_template(prim_path_template, env_id)
         prim = stage.GetPrimAtPath(prim_path)