o3d_gui_visualizer

class real_robot.utils.visualization.o3d_gui_visualizer.GeometryNode(name: str, id: int = -1, parent: ~real_robot.utils.visualization.o3d_gui_visualizer.GeometryNode | None = None, children: list[GeometryNode] = <factory>, geometry_data: ~open3d.cpu.pybind.geometry.Geometry3D | ~open3d.cpu.pybind.t.geometry.Geometry | ~open3d.cpu.pybind.visualization.rendering.TriangleMeshModel | None = None, cell: ~open3d.cpu.pybind.visualization.gui.Widget | None = None, mat_changed: bool = False, mat_shader_index: int = 1, mat_prefab_text: str = 'Polished ceramic [default]', mat_color: ~open3d.cpu.pybind.visualization.gui.Color = <open3d.cpu.pybind.visualization.gui.Color object>, mat_point_size: float = 3.0)

Bases: object

A geometry node in gui.TreeView() for easy parent-node lookup. NOTE: geometry group node with no children is not allowed

Variables:

• name – full geometry name containing all nested group names starting from root, separated by ‘/’. That is, name is unique based on its path from root node.

• id – item id in gui.TreeView

• parent – parent GeometryNode instance.

• children – list of children GeometryNode instances.

• geometry_data – the visualized geometry data of _o3d_geometry_type

• cell – gui.Widget for changing checkbox

• mat_shader_index – _shader item index

• mat_prefab_text – _material_prefab item text

• mat_color – material color

• mat_point_size – material point size

cell: Widget = None

children: list[GeometryNode]

property display_name: str

Returns the display name in geometries_panel

geometry_data: Geometry3D | Geometry | TriangleMeshModel = None

id: int = -1

mat_changed: bool = False

mat_color: Color = <open3d.cpu.pybind.visualization.gui.Color object>

mat_point_size: float = 3.0

mat_prefab_text: str = 'Polished ceramic [default]'

mat_shader_index: int = 1

name: str

parent: GeometryNode = None

class real_robot.utils.visualization.o3d_gui_visualizer.O3DGUIVisualizer(window_name='Point Clouds', window_size=(1920, 1080), *, run_as_process=False, stream_camera=False, stream_robot=False)

Bases: object

Open3D GUI Application based on http://www.open3d.org/docs/latest/python_api/open3d.visualization.gui.html References: https://github.com/isl-org/Open3D/blob/master/cpp/open3d/visualization/visualizer/O3DVisualizer.cpp https://github.com/isl-org/Open3D/blob/master/examples/python/visualization/all_widgets.py https://github.com/isl-org/Open3D/blob/master/examples/python/visualization/add_geometry.py https://github.com/isl-org/Open3D/blob/master/examples/python/visualization/vis_gui.py https://github.com/isl-org/Open3D/blob/master/examples/python/visualization/mouse_and_point_coord.py https://github.com/isl-org/Open3D/blob/master/examples/python/visualization/

DEFAULT_IBL = 'default'

MATERIAL_NAMES = ['Standard (Lit)', 'Unlit', 'Unlit Line', 'Normal Map', 'Depth']

MATERIAL_SHADERS = ['defaultLit', 'defaultUnlit', 'unlitLine', 'normals', 'depth']

MENU_ABOUT = 21

MENU_EXPORT = 2

MENU_OPEN = 1

MENU_QUIT = 3

MENU_SHOW_SETTINGS = 11

add_camera(camera_name: str, width: int, height: int, K: ndarray, T: ndarray | None = None, fmt: str = 'GL')

Add a camera to view from (OpenGL convention) Camera frame is right(x), up(y), backwards(z) :param camera_name: camera unique name :param width: camera image width :param height: camera image height :param K: camera intrinsic matrix, [3, 3] np.floating np.ndarray :param T: camera pose in world frame, [4, 4] np.floating np.ndarray :param fmt: camera frame conventions, available: [“GL”, “ROS”, “CV”]

add_geometries(geometry_dict: dict[str, o3d.geometry.Geometry3D | o3d.t.geometry.Geometry | rendering.TriangleMeshModel], show: bool | None = None, hide_others: bool = False, reset_camera: bool = False)

Add multiple geometries (allow for computing update fps) :param geometry_dict: dictionary with format {name: Open3D geometry} :param show: whether to show geometry after loading :param hide_others: whether to hide all other geometries (only display new ones) :param reset_camera: whether to reset camera view to fit all geometries

add_geometry(name: str, geometry: Geometry3D | Geometry | TriangleMeshModel, show: bool | None = None, reset_camera: bool = False) → bool

Add a geometry to scene and update the _geometries_tree.

Parameters:

• name – geometry name separated by ‘/’, str. Group names are nested starting from root. Geometry and geometry group with same names can coexist.

• geometry – Open3D geometry

• show – whether to show geometry after loading

Reset_camera:

whether to reset camera view to fit all geometries

Return success:

whether geometry is successfully added

clear_geometries()

Remove all geometries

close()

construct_gui()

Construct the GUI visualizer

export_image(path: str, width: int, height: int)

find_geometry_with_point(point)

Find the name of the geometry that contains the query point (Only support o3d.geometry.Geometry3D for now)

static get_camera_lineset(width: int, height: int, K: ndarray, far=1.0) → LineSet

Create a camera lineset with annotated up-direction :param width: camera image width :param height: camera image height :param K: camera intrinsic matrix, [3, 3] np.floating np.ndarray :param far: camera far clipping plane to draw the lineset, unit in meters

hide_all_geometries()

Hide all geometries by toggling root_geometry_node off

load_geometry(path: str | Path, *, name: str | None = None)

Load and add a geometry from file

Parameters:

• path – path to a geometry file supported by open3d. https://www.open3d.org/docs/release/tutorial/geometry/file_io.html

• name – geometry name separated by ‘/’, str. Group names are nested starting from root. Geometry and geometry group with same names can coexist.

load_urdf(urdf_path: str | Path, *, robot_name: str | None = None, qpos: ndarray | None = None, base_pose: ndarray = array([[1., 0., 0., 0.], [0., 1., 0., 0.], [0., 0., 1., 0.], [0., 0., 0., 1.]]))

Load a robot from a URDF file

Parameters:

• urdf_path – path to a URDF file.

• robot_name – robot name. If not given, use <robot> name attributes.

• qpos – robot joint positions.

• base_pose – T_world_urdfbase pose, [4, 4] np.floating np.ndarray

remove_geometry(name: str)

Remove a geometry from scene and update the _geometries_tree.

Parameters:

name – geometry or geometry group name separated by ‘/’, str. Group names are nested starting from root. Geometry and geometry group with same names can coexist.

render(render_step_fn=None)

Update GUI and respond to mouse and keyboard events for one tick :param render_step_fn: additional render step function to call.

run_as_process()

Run O3DGUIVisualizer as a separate process

set_focused_camera(name: str)

Set the camera view to look at :param name: existing camera name in self._camera_list

toggle_pause(paused: bool)

update_camera_pose(camera_name, T: ndarray, fmt: str = 'GL')

Update viewing camera pose (OpenGL convention) Camera frame is right(x), up(y), backwards(z) :param camera_name: camera unique name :param T: camera pose in world frame, [4, 4] np.floating np.ndarray :param fmt: camera frame conventions, available: [“GL”, “ROS”, “CV”]

update_render_info(text: str, color=[1.0, 1.0, 1.0])

update_urdf(robot_name: str, *, qpos: ndarray | None = None, base_pose: ndarray | None = None)

Update a robot URDF with given qpos and base_pose

Parameters:

• robot_name – robot name.

• qpos – robot joint positions.

• base_pose – T_world_urdfbase pose, [4, 4] np.floating np.ndarray

class real_robot.utils.visualization.o3d_gui_visualizer.O3DGeometryDefaultDict

Bases: dict

This defaultdict helps to store open3d geometries by name (only known at runtime) so we don’t need to frequently create them. Used in O3DGUIVisualizer.run_as_process()

class real_robot.utils.visualization.o3d_gui_visualizer.Settings

Bases: object

CUSTOM_PROFILE_NAME = 'Custom'

DEFAULT_MATERIAL_NAME = 'Polished ceramic [default]'

DEFAULT_PROFILE_NAME = 'Bright day with sun at +Y [default]'

DEPTH = 'depth'

GROUND_PLANE = {'XY': <GroundPlane.XY: 1>, 'XZ': <GroundPlane.XZ: 0>, 'YZ': <GroundPlane.YZ: 2>}

LIGHTING_PROFILES = {'Bright day with sun at +Y [default]': {'ibl_intensity': 45000, 'sun_dir': [0.577, -0.577, -0.577], 'sun_intensity': 45000, 'use_ibl': True, 'use_sun': True}, 'Bright day with sun at +Z': {'ibl_intensity': 45000, 'sun_dir': [0.577, 0.577, -0.577], 'sun_intensity': 45000, 'use_ibl': True, 'use_sun': True}, 'Bright day with sun at -Y': {'ibl_intensity': 45000, 'sun_dir': [0.577, 0.577, 0.577], 'sun_intensity': 45000, 'use_ibl': True, 'use_sun': True}, 'Cloudy day (no direct sun)': {'ibl_intensity': 60000, 'sun_intensity': 50000, 'use_ibl': True, 'use_sun': False}, 'Less Bright day with sun at +Y': {'ibl_intensity': 35000, 'sun_dir': [0.577, -0.577, -0.577], 'sun_intensity': 50000, 'use_ibl': True, 'use_sun': True}, 'Less Bright day with sun at +Z': {'ibl_intensity': 35000, 'sun_dir': [0.577, 0.577, -0.577], 'sun_intensity': 50000, 'use_ibl': True, 'use_sun': True}, 'Less Bright day with sun at -Y': {'ibl_intensity': 35000, 'sun_dir': [0.577, 0.577, 0.577], 'sun_intensity': 50000, 'use_ibl': True, 'use_sun': True}}

LIT = 'defaultLit'

NORMALS = 'normals'

POINT_CLOUD_PROFILE_NAME = 'Cloudy day (no direct sun)'

PREFAB = {'Clay': {'anisotropy': 0.0, 'clearcoat': 0.1, 'clearcoat_roughness': 0.287, 'metallic': 0.0, 'reflectance': 0.5, 'roughness': 1.0}, 'Glazed ceramic': {'anisotropy': 0.0, 'clearcoat': 1.0, 'clearcoat_roughness': 0.1, 'metallic': 0.0, 'reflectance': 0.9, 'roughness': 0.5}, 'Metal (rougher)': {'anisotropy': 0.0, 'clearcoat': 0.0, 'clearcoat_roughness': 0.0, 'metallic': 1.0, 'reflectance': 0.9, 'roughness': 0.5}, 'Metal (smoother)': {'anisotropy': 0.0, 'clearcoat': 0.0, 'clearcoat_roughness': 0.0, 'metallic': 1.0, 'reflectance': 0.9, 'roughness': 0.3}, 'Plastic': {'anisotropy': 0.0, 'clearcoat': 0.5, 'clearcoat_roughness': 0.2, 'metallic': 0.0, 'reflectance': 0.5, 'roughness': 0.5}, 'Polished ceramic [default]': {'anisotropy': 0.0, 'clearcoat': 0.2, 'clearcoat_roughness': 0.2, 'metallic': 0.0, 'reflectance': 0.5, 'roughness': 0.7}}

UNLIT = 'defaultUnlit'

UNLIT_LINE = 'unlitLine'

apply_lighting_profile(name: str)

apply_material_prefab(name: str)