Hugging Face's logo

SidSinha16
/
Test_Training_v3

PEFT
Safetensors
Generated from Trainer
Model card Files
xet
Community
SidSinha16 commited on
Commit
67472d3
·
verified ·
1 Parent(s): bbd9891

End of training

Browse files
Files changed (3) hide show
  1. README.md +37 -26
  2. adapter_model.safetensors +1 -1
  3. training_args.bin +1 -1
README.md CHANGED
@@ -1,22 +1,13 @@
1
- ---
2
- license: bigcode-openrail-m
3
- base_model: bigcode/starcoderbase-1b
4
- tags:
5
- - generated_from_trainer
6
- - windchill
7
- - starcoder
8
- - peft
9
- - lora
10
- - code-generation
11
- - transformers
12
- - huggingface
13
- library_name: peft
14
- model-index:
15
- - name: Test_Training_v3
16
- results: []
17
- datasets:
18
- - SidSinha16/windchill_code
19
- ---
20
 
21
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
22
  should probably proofread and complete it, then remove this comment. -->
@@ -25,7 +16,7 @@ should probably proofread and complete it, then remove this comment. -->
25
 
26
  This model is a fine-tuned version of [bigcode/starcoderbase-1b](https://huggingface.co/bigcode/starcoderbase-1b) on an unknown dataset.
27
  It achieves the following results on the evaluation set:
28
- - Loss: 0.8814
29
 
30
  ## Model description
31
 
@@ -45,20 +36,40 @@ More information needed
45
 
46
  The following hyperparameters were used during training:
47
  - learning_rate: 0.0005
48
- - train_batch_size: 4
49
- - eval_batch_size: 4
50
  - seed: 42
 
 
51
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
52
  - lr_scheduler_type: cosine
53
- - lr_scheduler_warmup_steps: 5
54
- - training_steps: 200
55
 
56
  ### Training results
57
 
58
  | Training Loss | Epoch | Step | Validation Loss |
59
  |:-------------:|:-----:|:----:|:---------------:|
60
- | 0.2578 | 0.5 | 100 | 0.8038 |
61
- | 0.1981 | 1.0 | 200 | 0.8814 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
62
 
63
 
64
  ### Framework versions
 
1
+ ---
2
+ license: bigcode-openrail-m
3
+ base_model: bigcode/starcoderbase-1b
4
+ tags:
5
+ - generated_from_trainer
6
+ library_name: peft
7
+ model-index:
8
+ - name: Test_Training_v3
9
+ results: []
10
+ ---
 
 
 
 
 
 
 
 
 
11
 
12
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
13
  should probably proofread and complete it, then remove this comment. -->
 
16
 
17
  This model is a fine-tuned version of [bigcode/starcoderbase-1b](https://huggingface.co/bigcode/starcoderbase-1b) on an unknown dataset.
18
  It achieves the following results on the evaluation set:
19
+ - Loss: 1.5622
20
 
21
  ## Model description
22
 
 
36
 
37
  The following hyperparameters were used during training:
38
  - learning_rate: 0.0005
39
+ - train_batch_size: 2
40
+ - eval_batch_size: 2
41
  - seed: 42
42
+ - gradient_accumulation_steps: 4
43
+ - total_train_batch_size: 8
44
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
45
  - lr_scheduler_type: cosine
46
+ - lr_scheduler_warmup_steps: 30
47
+ - training_steps: 2000
48
 
49
  ### Training results
50
 
51
  | Training Loss | Epoch | Step | Validation Loss |
52
  |:-------------:|:-----:|:----:|:---------------:|
53
+ | 0.4169 | 0.05 | 100 | 0.9127 |
54
+ | 0.2774 | 0.1 | 200 | 0.9903 |
55
+ | 0.1967 | 0.15 | 300 | 1.1247 |
56
+ | 0.1572 | 0.2 | 400 | 1.1690 |
57
+ | 0.1649 | 0.25 | 500 | 1.2579 |
58
+ | 0.123 | 0.3 | 600 | 1.2884 |
59
+ | 0.1023 | 0.35 | 700 | 1.3079 |
60
+ | 0.0777 | 0.4 | 800 | 1.3562 |
61
+ | 0.0674 | 0.45 | 900 | 1.4247 |
62
+ | 0.0704 | 0.5 | 1000 | 1.4536 |
63
+ | 0.0715 | 0.55 | 1100 | 1.4713 |
64
+ | 0.0502 | 0.6 | 1200 | 1.4968 |
65
+ | 0.0553 | 0.65 | 1300 | 1.5002 |
66
+ | 0.0505 | 0.7 | 1400 | 1.5015 |
67
+ | 0.043 | 0.75 | 1500 | 1.5359 |
68
+ | 0.0458 | 0.8 | 1600 | 1.5474 |
69
+ | 0.0355 | 0.85 | 1700 | 1.5581 |
70
+ | 0.0438 | 0.9 | 1800 | 1.5616 |
71
+ | 0.0507 | 0.95 | 1900 | 1.5633 |
72
+ | 0.0378 | 1.0 | 2000 | 1.5622 |
73
 
74
 
75
  ### Framework versions
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0a2f349e949dd739c1d8694c4e76fcb4e4f64b96212f9d0d55b98fc3ce8c3152
3
  size 22241240
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bcf91cef60b329e70540935be4ad43fcf8287e064ce0871429d0a8550d73fb8e
3
  size 22241240
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:9bcaad46073b52f47eec804de095136078ee7586f4b90c4d9acc277c9187e7d7
3
  size 4920
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9ba77d6a721ba4e3463b0b4b25fcb8cf5f8f89584be13254435fd935f72d4e54
3
  size 4920