training meteor Attention-UNet
training based DeepSpeed implementation for agent decoder.
- Input
- 1927-dim embedding
- Encoder
- 64 x Attention-UNet with 58 heads
- Output
- precision projection
Training config
optimizer=NAdam, lr=0.684, scheduler=polynomial, warmup=1094