Tutorial Index

Kaolin provides tutorials as ipython notebooks, docs pages and simple scripts. Note that the links point to the main branch and are not tied to the version. Here is a brief introduction to our tutorials and repository structure:

Detailed Tutorials

  • Representation-Agnostic Physics Simulation with Simplicits:
  • Differentiable Rendering Docs Tutorials:
  • Camera and Rasterization: Rasterize ShapeNet mesh with nvdiffrast and camera:

    • Load ShapeNet mesh

    • Preprocess mesh and materials

    • Create a camera with from_args() general constructor

    • Render a mesh with multiple materials with nvdiffrast

    • Move camera and see the resulting rendering

  • Optimizing Diffuse Lighting: Optimize lighting parameters with spherical gaussians and spherical harmonics:

    • Load an obj mesh with normals and materials

    • Rasterize the diffuse and specular albedo

    • Render and optimize diffuse lighting: * Spherical harmonics * Spherical gaussian with inner product implementation * Spherical gaussian with fitted approximation

  • Optimize Diffuse and Specular Lighting with Spherical Gaussians:

    • Load an obj mesh with normals and materials

    • Generate view rays from camera

    • Rasterize the diffuse and specular albedo

    • Render and optimize diffuse and specular lighting with spherical gaussians

  • Working with Surface Meshes:

    • loading and constructing kaolin.rep.SurfaceMesh objects

    • batching of meshes

    • auto-computing common attributes (like face_normals)

  • Understanding Structured Point Clouds (SPCs): walks through SPC features, covering:

    • under-the-hood explanation of SPC, why it’s useful and key ops

    • loading a mesh

    • sampling a point cloud

    • converting a point cloud to SPC

    • setting up camera

    • rendering SPC with ray tracing

    • storing features in an SPC

  • Differentiable Rendering: optimizes a triangular mesh from images using DIB-R renderer, covering:

    • generating data with Omniverse Kaolin App, and loading this synthetic data

    • loading a mesh

    • computing mesh laplacian

    • DIB-R rasterization

    • differentiable texture mapping

    • computing mask intersection-over-union loss (IOU)

    • using Timelapse API for 3D checkpoints

    • visualizing 3D results of training

  • Fitting a 3D Bounding Box: fits a 3D bounding box around an object in images using DIB-R renderer, covering:

    • generating data with Omniverse Kaolin App, and loading this synthetic data

    • loading a mesh

    • DIB-R rasterization

    • computing mask intersection-over-union loss (IOU)

  • 3D Checkpoint Visualization: explains saving 3D checkpoints and visualizing them, covering:

    • using Timelapse API for writing 3D checkpoints

    • understanding output file format

    • visualizing 3D checkpoints using Omniverse Kaolin App

    • visualizing 3D checkpoints using bundled kaolin-dash3d commandline utility

  • Reconstructing Point Cloud with DMTet: Trains an SDF estimator to reconstruct a mesh from a point cloud covering:

    • using point clouds data generated with Omniverse Kaolin App

    • loading point clouds from an USD file.

    • defining losses and regularizer for a mesh with point cloud ground truth

    • applying marching tetrahedra

    • using Timelapse API for 3D checkpoints

    • visualizing 3D checkpoints using kaolin-dash3d

Simple Recipes