pszemraj
/

pegasus-x-large-book-summary

text2text-generation

Model card Files Files and versions

pszemraj commited on Sep 18, 2022

Commit

2f6950d

·

1 Parent(s): 9745b8e

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -241,7 +241,7 @@ The following hyperparameters were used during training:
 - distributed_type: multi-GPU
 - gradient_accumulation_steps: 16
 - total_train_batch_size: 64
-- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
 - lr_scheduler_warmup_ratio: 0.03
 - num_epochs: 2

 - distributed_type: multi-GPU
 - gradient_accumulation_steps: 16
 - total_train_batch_size: 64
+- optimizer: _ADAN_ using lucidrains' `adan-pytorch` with default betas
 - lr_scheduler_type: cosine
 - lr_scheduler_warmup_ratio: 0.03
 - num_epochs: 2