metadata

tags:
  - kernel

Activation

Activation is a python package that contains custom CUDA-based activation kernels, primarily targeting AMD GPUs.

Currently implemented
- PolyNorm

Usage

import torch
from kernels import get_kernel

activation = get_kernel("motif-technologies/activation")

torch.set_default_device("cuda")
poly_norm = activation.layers.PolyNorm(eps=1e-6)
x = torch.randn(10, 10)

print(poly_norm(x))