Some example stages found in a pipeline might be data preprocessing, feature extraction, and normalization. For more details, see Pipelines for inference. PipelineParallel (PP) Parallelism technique in which the model is split up vertically (layer-level) across multiple GPUs, so that only one or several layers of the model are placed on a single GPU. Each GPU processes in parallel different stages of the pipeline and working on a small chunk of the batch.