Recipes# Recipes are bite-sized, actionable examples of how to use specific PyTorch features, different from our full-length tutorials. All Defining a Neural Network Learn how to use PyTorch's torch.nn package to create and define a neural network for the MNIST dataset. Basics What is a state_dict in PyTorch Learn how state_dict objects and Python dictionaries are used in saving or loading models from PyTorch. Basics Warmstarting model using parameters from a different model in PyTorch Learn how warmstarting the training process by partially loading a model or loading a partial model can help your model converge much faster than training from scratch. Basics Zeroing out gradients in PyTorch Learn when you should zero out gradients and how doing so can help increase the accuracy of your model. Basics PyTorch Benchmark Learn how to use PyTorch's benchmark module to measure and compare the performance of your code Basics PyTorch Benchmark (quick start) Learn how to measure snippet run times and collect instructions. Basics PyTorch Profiler Learn how to use PyTorch's profiler to measure operators time and memory consumption Basics PyTorch Profiler with Instrumentation and Tracing Technology API (ITT API) support Learn how to use PyTorch's profiler with Instrumentation and Tracing Technology API (ITT API) to visualize operators labeling in Intel® VTune™ Profiler GUI Basics Dynamic Compilation Control with ``torch.compiler.set_stance`` Learn how to use torch.compiler.set_stance Compiler Reasoning about Shapes in PyTorch Learn how to use the meta device to reason about shapes in your model. Basics Tips for Loading an nn.Module from a Checkpoint Learn tips for loading an nn.Module from a checkpoint. Basics (beta) Using TORCH_LOGS to observe torch.compile Learn how to use the torch logging APIs to observe the compilation process. Basics Extension points in nn.Module for loading state_dict and tensor subclasses New extension points in nn.Module. Basics torch.export AOTInductor Tutorial for Python runtime Learn an end-to-end example of how to use AOTInductor for python runtime. Basics Demonstration of torch.export flow, common challenges and the solutions to address them Learn how to export models for popular usecases Compiler,TorchCompile Model Interpretability using Captum Learn how to use Captum attribute the predictions of an image classifier to their corresponding image features and visualize the attribution results. Interpretability,Captum How to use TensorBoard with PyTorch Learn basic usage of TensorBoard with PyTorch, and how to visualize data in TensorBoard UI Visualization,TensorBoard DebugMode: Recording Dispatched Operations and Numerical Debugging Inspect dispatched ops, tensor hashes, and module boundaries to debug eager and ``torch.compile`` runs. Interpretability,Compiler Automatic Mixed Precision Use torch.cuda.amp to reduce runtime and save memory on NVIDIA GPUs. Model-Optimization Performance Tuning Guide Tips for achieving optimal performance. Model-Optimization Optimizing CPU Performance on Intel® Xeon® with run_cpu Script How to use run_cpu script for optimal runtime configurations on Intel® Xeon CPUs. Model-Optimization (beta) Utilizing Torch Function modes with torch.compile Override torch operators with Torch Function modes and torch.compile Model-Optimization (beta) Compiling the Optimizer with torch.compile Speed up the optimizer using torch.compile Model-Optimization (beta) Running the compiled optimizer with an LR Scheduler Speed up training with LRScheduler and torch.compiled optimizer Model-Optimization (beta) Explicit horizontal fusion with foreach_map and torch.compile Horizontally fuse pointwise ops with torch.compile Model-Optimization Using User-Defined Triton Kernels with ``torch.compile`` Learn how to use user-defined kernels with ``torch.compile`` Model-Optimization Compile Time Caching in ``torch.compile`` Learn how to use compile time caching in ``torch.compile`` Model-Optimization Compile Time Caching Configurations Learn how to configure compile time caching in ``torch.compile`` Model-Optimization Reducing torch.compile cold start compilation time with regional compilation Learn how to use regional compilation to control cold start compile time Model-Optimization Intel® Neural Compressor for PyTorch Ease-of-use quantization for PyTorch with Intel® Neural Compressor. Quantization,Model-Optimization Getting Started with DeviceMesh Learn how to use DeviceMesh Distributed-Training Shard Optimizer States with ZeroRedundancyOptimizer How to use ZeroRedundancyOptimizer to reduce memory consumption. Distributed-Training Direct Device-to-Device Communication with TensorPipe RPC How to use RPC with direct GPU-to-GPU communication. Distributed-Training Getting Started with Distributed Checkpoint (DCP) Learn how to checkpoint distributed models with Distributed Checkpoint package. Distributed-Training Asynchronous Checkpointing (DCP) Learn how to checkpoint distributed models with Distributed Checkpoint package. Distributed-Training Getting Started with CommDebugMode Learn how to use CommDebugMode for DTensors Distributed-Training Reducing AoT cold start compilation time with regional compilation Learn how to use regional compilation to control AoT cold start compile time Model-Optimization
