caduceus-ps_seqlen-131k_d_model-256_n_layer-16_ft_BioS73_1kbpHG19_DHSs_H3K27AC

This model is a fine-tuned version of kuleshov-group/caduceus-ps_seqlen-131k_d_model-256_n_layer-16 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 144515.0938
• F1 Score: 0.2783
• Precision: 0.2883
• Recall: 0.2691
• Accuracy: 0.2504
• Auc: 0.7677
• Prc: 0.7884

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Precision	Recall	Accuracy	Auc	Prc
589539311.616	1.0	1342	486516.875	0.3847	0.3674	0.4036	0.3063	0.7774	0.7982
196098.672	2.0	2684	142830.5469	0.7345	0.7482	0.7212	0.7198	0.7781	0.7987
162470.704	3.0	4026	104812.0781	0.7474	0.7459	0.7490	0.7280	0.7785	0.7989
191921.936	4.0	5368	5804.9341	0.7193	0.7455	0.6949	0.7086	0.7790	0.7993
163291.136	5.0	6710	36408.6133	0.2866	0.3047	0.2705	0.2765	0.7793	0.7994
330819.552	6.0	8052	235902.1406	0.1125	0.2140	0.0763	0.3532	0.7854	0.8030
210969.488	7.0	9394	85169.4531	0.7538	0.6947	0.8239	0.7109	0.7830	0.8009
232610.912	8.0	10736	81787.5625	0.7168	0.5595	0.9972	0.5768	0.7735	0.7956
489027.84	9.0	12078	170635.1562	0.5811	0.4753	0.7476	0.4210	0.7724	0.8006
128496.72	10.0	13420	391980.6562	0.5288	0.4461	0.6491	0.3785	0.7735	0.8006
260038.32	11.0	14762	261249.3906	0.3126	0.3117	0.3135	0.2593	0.7688	0.7948
174084.704	12.0	16104	144515.0938	0.2783	0.2883	0.2691	0.2504	0.7677	0.7884

Framework versions

• Transformers 4.41.2
• Pytorch 2.2.0
• Datasets 2.15.0
• Tokenizers 0.19.1