Fast Neural Style Transfer
Use-cases
This artistic style transfer model mixes the content of an image with the style of another image. Examples of the styles can be seen in this PyTorch example.
Description
The model uses the method described in Perceptual Losses for Real-Time Style Transfer and Super-Resolution along with Instance Normalization.
Model
| Model | Download | Download (with sample test data) | ONNX version | Opset version |
|---|---|---|---|---|
| Mosaic | 6.6 MB | 7.2 MB | 1.4 | 9 |
| Candy | 6.6 MB | 7.2 MB | 1.4 | 9 |
| Rain Princess | 6.6 MB | 7.2 MB | 1.4 | 9 |
| Udnie | 6.6 MB | 7.2 MB | 1.4 | 9 |
| Pointilism | 6.6 MB | 7.2 MB | 1.4 | 9 |
| Mosaic | 6.6 MB | 7.2 MB | 1.4 | 8 |
| Candy | 6.6 MB | 7.2 MB | 1.4 | 8 |
| Rain Princess | 6.6 MB | 7.2 MB | 1.4 | 8 |
| Udnie | 6.6 MB | 7.2 MB | 1.4 | 8 |
| Pointilism | 6.6 MB | 7.2 MB | 1.4 | 8 |
Inference
Refer to style-transfer-ort.ipynb for detailed preprocessing and postprocessing.
Input to model
The input to the model are 3-channel RGB images. The images have to be loaded in a range of [0, 255]. If running into memory issues, try resizing the image by increasing the scale number.
Preprocessing steps
from PIL import Image
import numpy as np
# loading input and resize if needed
image = Image.open("PATH TO IMAGE")
size_reduction_factor = 1
image = image.resize((int(image.size[0] / size_reduction_factor), int(image.size[1] / size_reduction_factor)), Image.ANTIALIAS)
# Preprocess image
x = np.array(image).astype('float32')
x = np.transpose(x, [2, 0, 1])
x = np.expand_dims(x, axis=0)
Output of model
The converted ONNX model outputs a NumPy float32 array of shape [1, 3, ‘height’, ‘width’]. The height and width of the output image are the same as the height and width of the input image.
Postprocessing steps
result = np.clip(result, 0, 255)
result = result.transpose(1,2,0).astype("uint8")
img = Image.fromarray(result)
Dataset (Train and validation)
The original fast neural style model is from pytorch/examples/fast_neural_style. All models are trained using the COCO 2014 Training images dataset [80K/13GB].
Training
Refer to pytorch/examples/fast_neural_style for training details in PyTorch. Refer to conversion.ipynb to learn how the PyTorch models are converted to ONNX format.
References
Original style transfer model in PyTorch: https://github.com/pytorch/examples/tree/master/fast_neural_style
Contributors
License
BSD-3-Clause