Mitchins commited on
Commit
c87b146
·
verified ·
1 Parent(s): 9045aae

Add flux1-schnell-bf16-blackwell engines

Browse files
Files changed (3) hide show
  1. README.md +52 -0
  2. config.json +20 -0
  3. engines/ENGINES_NEEDED.txt +15 -0
README.md ADDED
@@ -0,0 +1,52 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # FLUX1-SCHNELL-BF16-BLACKWELL
2
+
3
+ TensorRT-RTX engines for FLUX1-SCHNELL optimized for Blackwell architecture with BF16 quantization.
4
+
5
+ ## Specifications
6
+
7
+ - **Model**: FLUX1-SCHNELL
8
+ - **Base Model**: black-forest-labs/FLUX.1-schnell
9
+ - **Architecture**: Blackwell
10
+ - **Quantization**: BF16
11
+ - **Batch Size**: 1 (optimized)
12
+ - **Resolution**: 1024x1024 (optimized)
13
+ - **Estimated Size**: 12.0 GB
14
+
15
+ ## Performance Estimates
16
+
17
+ Based on architecture and quantization:
18
+ - **Memory Usage**: ~12.0GB VRAM
19
+ - **Speed**: ~2.5s (H200)
20
+
21
+ ## Files
22
+
23
+ - `engines/`: TensorRT engine files (.plan)
24
+ - `config.json`: Configuration metadata
25
+ - `README.md`: This file
26
+
27
+ ## Usage
28
+
29
+ ```python
30
+ from imageai_server.tensorrt.nvidia_sdxl_pipeline import NVIDIASDXLPipeline
31
+
32
+ pipeline = NVIDIASDXLPipeline()
33
+ pipeline.load_engines(
34
+ engine_dir="./engines",
35
+ framework_model_dir="./framework",
36
+ onnx_dir="./onnx"
37
+ )
38
+ pipeline.activate_engines()
39
+
40
+ images, time_ms = pipeline.infer(
41
+ prompt="a beautiful landscape",
42
+ height=1024,
43
+ width=1024
44
+ )
45
+ ```
46
+
47
+ ## Requirements
48
+
49
+ - **GPU**: Blackwell architecture (Compute Capability 8.0+)
50
+ - **VRAM**: 12.0GB+
51
+ - **TensorRT-RTX**: 1.0.0.21+
52
+ - **CUDA**: 12.0+
config.json ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_type": "flux1-schnell",
3
+ "architecture": "blackwell",
4
+ "quantization": "bf16",
5
+ "batch_size": 1,
6
+ "height": 1024,
7
+ "width": 1024,
8
+ "base_model": "black-forest-labs/FLUX.1-schnell",
9
+ "expected_engines": [
10
+ "clip.plan",
11
+ "t5.plan",
12
+ "transformer.plan",
13
+ "vae.plan"
14
+ ],
15
+ "created_at": 1754966251.0015864,
16
+ "tensorrt_version": "10.13.2.6",
17
+ "tensorrt_rtx_version": "1.0.0.21",
18
+ "size_estimate_gb": 12.0,
19
+ "build_system": "comprehensive_offline_builder"
20
+ }
engines/ENGINES_NEEDED.txt ADDED
@@ -0,0 +1,15 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Engines for flux1-schnell-bf16-blackwell need to be built.
2
+
3
+ This is a placeholder structure. To complete:
4
+ 1. Build actual flux1-schnell engines with bf16 quantization
5
+ 2. Copy .plan files to this directory
6
+ 3. Remove this placeholder file
7
+
8
+ Expected files:
9
+ - clip.plan
10
+ - t5.plan
11
+ - transformer.plan
12
+ - vae.plan
13
+
14
+ Build command:
15
+ python build_tensorrt_engines.py --single flux1-schnell blackwell bf16