metadata
tags:
- kernel
Activation
Activation is a python package that contains custom CUDA-based activation kernels, primarily targeting AMD GPUs.
Usage
import torch
from kernels import get_kernel
activation = get_kernel("motif-technologies/activation")
torch.set_default_device("cuda")
poly_norm = activation.layers.PolyNorm(eps=1e-6)
x = torch.randn(10, 10)
print(poly_norm(x))
Performance
PolyNorm
- Test cases are from the Motif LLM
- You can reproduce the results with:
cd tests
pytest --run-perf --do-plot
