Hugging Face's logo

Diffusers documentation

Pipeline

You are viewing main version, which requires installation from source. If you'd like regular pip install, checkout the latest stable version (v0.34.0).
Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

Pipeline

ModularPipeline

class diffusers.ModularPipeline

< >

( blocks: typing.Optional[diffusers.modular_pipelines.modular_pipeline.ModularPipelineBlocks] = None pretrained_model_name_or_path: typing.Union[str, os.PathLike, NoneType] = None components_manager: typing.Optional[diffusers.modular_pipelines.components_manager.ComponentsManager] = None collection: typing.Optional[str] = None **kwargs )

Parameters

  • blocks — ModularPipelineBlocks, the blocks to be used in the pipeline

Base class for all Modular pipelines.

This is an experimental feature and is likely to change in the future.

from_pretrained

< >

( pretrained_model_name_or_path: typing.Union[str, os.PathLike, NoneType] trust_remote_code: typing.Optional[bool] = None components_manager: typing.Optional[diffusers.modular_pipelines.components_manager.ComponentsManager] = None collection: typing.Optional[str] = None **kwargs )

Parameters

  • pretrained_model_name_or_path (str or os.PathLike, optional) — Path to a pretrained pipeline configuration. If provided, will load component specs (only for from_pretrained components) and config values from the modular_model_index.json file.
  • trust_remote_code (bool, optional) — Whether to trust remote code when loading the pipeline, need to be set to True if you want to create pipeline blocks based on the custom code in pretrained_model_name_or_path
  • components_manager (ComponentsManager, optional) — ComponentsManager instance for managing multiple component cross different pipelines and apply offloading strategies.
  • collection (str, optional) —` Collection name for organizing components in the ComponentsManager.

Load a ModularPipeline from a huggingface hub repo.

get_component_spec

< >

( name: str )

load_components

< >

( names: typing.Union[typing.List[str], str] **kwargs )

Parameters

  • names — List of component names to load; by default will not load any components
  • **kwargs — additional kwargs to be passed to from_pretrained().Can be:
    • a single value to be applied to all components to be loaded, e.g. torch_dtype=torch.bfloat16
    • a dict, e.g. torch_dtype={“unet”: torch.bfloat16, “default”: torch.float32}
    • if potentially override ComponentSpec if passed a different loading field in kwargs, e.g. repo, variant, revision, etc.

Load selected components from specs.

load_default_components

< >

( **kwargs )

Parameters

  • **kwargs — Additional arguments passed to from_pretrained method, e.g. torch_dtype, cache_dir, etc.

Load from_pretrained components using the loading specs in the config dict.

register_components

< >

( **kwargs )

Parameters

  • **kwargs — Keyword arguments where keys are component names and values are component objects. E.g., register_components(unet=unet_model, text_encoder=encoder_model)

Register components with their corresponding specifications.

This method is responsible for:

  1. Sets component objects as attributes on the loader (e.g., self.unet = unet)
  2. Updates the config dict, which will be saved as modular_model_index.json during save_pretrained (only for from_pretrained components)
  3. Adds components to the component manager if one is attached (only for from_pretrained components)

This method is called when:

  • Components are first initialized in init:
    • from_pretrained components not loaded during init so they are registered as None;
    • non from_pretrained components are created during init and registered as the object itself
  • Components are updated with the update_components() method: e.g. loader.update_components(unet=unet) or loader.update_components(guider=guider_spec)
  • (from_pretrained) Components are loaded with the load_default_components() method: e.g. loader.load_default_components(names=[“unet”])

Notes:

  • When registering None for a component, it sets attribute to None but still syncs specs with the config dict, which will be saved as modular_model_index.json during save_pretrained
  • component_specs are updated to match the new component outside of this method, e.g. in update_components() method

save_pretrained

< >

( save_directory: typing.Union[str, os.PathLike] push_to_hub: bool = False **kwargs )

Parameters

  • save_directory (str or os.PathLike) — Path to the directory where the pipeline will be saved.
  • push_to_hub (bool, optional) — Whether to push the pipeline to the huggingface hub.
  • **kwargs — Additional arguments passed to save_config() method

Save the pipeline to a directory. It does not save components, you need to save them separately.

to

< >

( *args **kwargs ) → DiffusionPipeline

Parameters

  • dtype (torch.dtype, optional) — Returns a pipeline with the specified dtype
  • device (torch.Device, optional) — Returns a pipeline with the specified device
  • silence_dtype_warnings (str, optional, defaults to False) — Whether to omit warnings if the target dtype is not compatible with the target device.

Returns

DiffusionPipeline

The pipeline converted to specified dtype and/or dtype.

Performs Pipeline dtype and/or device conversion. A torch.dtype and torch.device are inferred from the arguments of self.to(*args, **kwargs).

If the pipeline already has the correct torch.dtype and torch.device, then it is returned as is. Otherwise, the returned pipeline is a copy of self with the desired torch.dtype and torch.device.

Here are the ways to call to:

  • to(dtype, silence_dtype_warnings=False) → DiffusionPipeline to return a pipeline with the specified dtype
  • to(device, silence_dtype_warnings=False) → DiffusionPipeline to return a pipeline with the specified device
  • to(device=None, dtype=None, silence_dtype_warnings=False) → DiffusionPipeline to return a pipeline with the specified device and dtype

update_components

< >

( **kwargs )

Parameters

  • **kwargs — Component objects, ComponentSpec objects, or configuration values to update:
    • Component objects: Only supports components we can extract specs using ComponentSpec.from_component() method i.e. components created with ComponentSpec.load() or ConfigMixin subclasses that aren’t nn.Modules (e.g., unet=new_unet, text_encoder=new_encoder)
    • ComponentSpec objects: Only supports default_creation_method == “from_config”, will call create() method to create a new component (e.g., guider=ComponentSpec(name="guider", type_hint=ClassifierFreeGuidance, config={...}, default_creation_method="from_config"))
    • Configuration values: Simple values to update configuration settings (e.g., requires_safety_checker=False)

Raises

ValueError

  • ValueError — If a component object is not supported in ComponentSpec.from_component() method:
    • nn.Module components without a valid _diffusers_load_id attribute
    • Non-ConfigMixin components without a valid _diffusers_load_id attribute

Update components and configuration values and specs after the pipeline has been instantiated.

This method allows you to:

  1. Replace existing components with new ones (e.g., updating self.unet or self.text_encoder)
  2. Update configuration values (e.g., changing self.requires_safety_checker flag)

In addition to updating the components and configuration values as pipeline attributes, the method also updates:

  • the corresponding specs in _component_specs and _config_specs
  • the config dict, which will be saved as modular_model_index.json during save_pretrained

Examples:

# Update multiple components at once
pipeline.update_components(unet=new_unet_model, text_encoder=new_text_encoder)

# Update configuration values
pipeline.update_components(requires_safety_checker=False)

# Update both components and configs together
pipeline.update_components(unet=new_unet_model, requires_safety_checker=False)

# Update with ComponentSpec objects (from_config only)
pipeline.update_components(
    guider=ComponentSpec(
        name="guider",
        type_hint=ClassifierFreeGuidance,
        config={"guidance_scale": 5.0},
        default_creation_method="from_config",
    )
)

Notes:

  • Components with trained weights must be created using ComponentSpec.load(). If the component has not been shared in huggingface hub and you don’t have loading specs, you can upload it using push_to_hub()
  • ConfigMixin objects without weights (e.g., schedulers, guiders) can be passed directly
  • ComponentSpec objects with default_creation_method=“from_pretrained” are not supported in update_components()
< > Update on GitHub

←Quantization Blocks→