pretrained reward CNN
pretrained based DeepSpeed implementation for decision-tree transformer.
- Input
- 1024-dim embedding
- Encoder
- 69 x CNN with 6 heads
- Output
- rouge-l projection
Training config
optimizer=NAdam, lr=0.378, scheduler=polynomial, warmup=150