loss alignment Attention-UNet
loss based ONNX implementation for quantized attention.
- Input
- 619-dim embedding
- Encoder
- 53 x Attention-UNet with 22 heads
- Output
- perplexity projection
Training config
optimizer=SGD, lr=0.894, scheduler=exponential, warmup=689