ailab-bio/PROTAC-Splitter-EncoderDecoder-lr_cosine_restarts-opt25

This model is a fine-tuned version of seyonec/ChemBERTa-zinc-base-v1 on the ailab-bio/PROTAC-Splitter-Dataset dataset. It achieves the following results on the evaluation set:

• Loss: 0.3086
• Poi Heavy Atoms Difference: 2.1208
• E3 Valid: 0.9896
• Poi Valid: 0.9272
• Poi Has Attachment Point(s): 0.9272
• All Ligands Equal: 0.5462
• Valid: 0.9157
• Reassembly: 0.5544
• Poi Tanimoto Similarity: 0.0
• Linker Tanimoto Similarity: 0.0
• Poi Graph Edit Distance: inf
• Linker Heavy Atoms Difference: 0.3144
• Linker Graph Edit Distance Norm: inf
• E3 Graph Edit Distance Norm: inf
• Num Fragments: 2.9998
• E3 Heavy Atoms Difference Norm: 0.0131
• Linker Valid: 0.9961
• E3 Heavy Atoms Difference: 0.5553
• E3 Tanimoto Similarity: 0.0
• Poi Heavy Atoms Difference Norm: 0.0719
• Reassembly Nostereo: 0.5796
• Linker Equal: 0.7666
• Linker Has Attachment Point(s): 0.9961
• Has All Attachment Points: 0.9836
• Poi Equal: 0.7680
• E3 Graph Edit Distance: inf
• Tanimoto Similarity: 0.0
• E3 Has Attachment Point(s): 0.9896
• Poi Graph Edit Distance Norm: inf
• E3 Equal: 0.8045
• Heavy Atoms Difference Norm: 0.0939
• Has Three Substructures: 0.9991
• Linker Graph Edit Distance: inf
• Heavy Atoms Difference: 7.0102
• Linker Heavy Atoms Difference Norm: 0.0033

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 128
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine_with_restarts
• lr_scheduler_warmup_steps: 100
• training_steps: 10000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Poi Heavy Atoms Difference	E3 Valid	Poi Valid	Poi Has Attachment Point(s)	All Ligands Equal	Valid	Reassembly	Poi Tanimoto Similarity	Linker Tanimoto Similarity	Poi Graph Edit Distance	Linker Heavy Atoms Difference	Linker Graph Edit Distance Norm	E3 Graph Edit Distance Norm	Num Fragments	E3 Heavy Atoms Difference Norm	Linker Valid	E3 Heavy Atoms Difference	E3 Tanimoto Similarity	Poi Heavy Atoms Difference Norm	Reassembly Nostereo	Linker Equal	Linker Has Attachment Point(s)	Has All Attachment Points	Poi Equal	E3 Graph Edit Distance	Tanimoto Similarity	E3 Has Attachment Point(s)	Poi Graph Edit Distance Norm	E3 Equal	Heavy Atoms Difference Norm	Has Three Substructures	Linker Graph Edit Distance	Heavy Atoms Difference	Linker Heavy Atoms Difference Norm
0.0156	0.4932	5000	0.2891	2.3129	0.9916	0.9252	0.9252	0.4596	0.9169	0.4665	0.0	0.0	inf	0.3805	inf	inf	3.0006	0.0061	0.9965	0.3913	0.0	0.0739	0.4911	0.6623	0.9965	0.9863	0.7265	inf	0.0	0.9916	inf	0.7815	0.0918	0.9988	35410764872521246890440289523238443113605795566260484204134400.0000	7.0030	0.0005
0.0077	0.7398	7500	0.3073	2.5750	0.9857	0.9161	0.9161	0.4898	0.8987	0.4975	0.0	0.0	inf	0.4483	inf	inf	3.0002	0.0088	0.9943	0.4845	0.0	0.0832	0.5220	0.7095	0.9943	0.9812	0.7363	inf	0.0	0.9857	inf	0.7908	0.1093	0.9995	56657223796033995024704463237181508981769272906016774726615040.0000	8.2839	0.0090
0.0046	0.9864	10000	0.3086	2.1208	0.9896	0.9272	0.9272	0.5462	0.9157	0.5544	0.0	0.0	inf	0.3144	inf	inf	2.9998	0.0131	0.9961	0.5553	0.0	0.0719	0.5796	0.7666	0.9961	0.9836	0.7680	inf	0.0	0.9896	inf	0.8045	0.0939	0.9991	inf	7.0102	0.0033

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.1+cu121
• Datasets 3.0.0
• Tokenizers 0.19.1