Update README.md

README.md

@@ -3,3 +3,302 @@ license: other
 license_name: license.md
 license_link: LICENSE
 ---
+# Model Card for CodeFuse-CodeGeeX2-6B
+<p align="center">
+    <img src="https://modelscope.cn/api/v1/models/codefuse-ai/CodeFuse-CodeGeeX2-6B/repo?Revision=master&FilePath=LOGO.jpg&View=true" width="800"/>
+<p>
+
+[[中文]](#chinese)    [[English]](#english)
+
+
+<a id="english"></a>
+
+## Model Description
+
+CodeFuse-CodeGeeX2-6B is a 6B Code-LLM finetuned by LoRA of multiple code tasks on the base model CodeGeeX2. 
+
+<br>
+
+## News and Updates
+
+  🔥🔥 2023-11-10 CodeFuse-CodeGeeX2-6B has been released, achieving a pass@1 (greedy decoding) score of 45.12% on HumanEval, which is a 9.22% increase compared to CodeGeeX2 35.9%.
+
+  🔥🔥 2023-10-20 CodeFuse-QWen-14B technical documentation has been released. For those interested, please refer to the CodeFuse article on our WeChat official account via the provided link.(https://mp.weixin.qq.com/s/PCQPkvbvfxSPzsqjOILCDw)
+
+  🔥🔥 2023-10-16 CodeFuse-QWen-14B has been released, achieving a pass@1 (greedy decoding) score of 48.78% on HumanEval, which is a 16% increase compared to Qwen-14b's 32.3%.
+
+  🔥🔥 2023-09-27 CodeFuse-StarCoder-15B has been released, achieving a pass@1 (greedy decoding) score of 54.9% on HumanEval, which is a 21% increase compared to StarCoder's 33.6%.
+
+🔥🔥🔥 2023-09-26 We are pleased to announce the release of the [4-bit quantized version](https://modelscope.cn/models/codefuse-ai/CodeFuse-CodeLlama-34B-4bits/summary) of [CodeFuse-CodeLlama-34B](https://modelscope.cn/models/codefuse-ai/CodeFuse-CodeLlama-34B/summary). Despite the quantization process, the model still achieves a remarkable 73.8% accuracy (greedy decoding) on the HumanEval pass@1 metric.
+
+🔥🔥🔥 2023-09-11 [CodeFuse-CodeLlama34B](https://modelscope.cn/models/codefuse-ai/CodeFuse-CodeLlama-34B/summary) has achived 74.4% of pass@1 (greedy decoding) on HumanEval, which is SOTA results for openspurced LLMs at present.
+
+<br>
+
+## Code Community
+
+**Homepage**: 🏡 https://github.com/codefuse-ai (**Please give us your support with a Star🌟 + Fork🚀 + Watch👀**)
+
++ If you wish to fine-tune the model yourself, you can visit ✨[MFTCoder](https://github.com/codefuse-ai/MFTCoder)✨✨
+
++ If you wish to deploy the model yourself, you can visit ✨[FasterTransformer4CodeFuse](https://github.com/codefuse-ai/FasterTransformer4CodeFuse)✨✨
+
++ If you wish to see a demo of the model, you can visit ✨[CodeFuse Demo](https://github.com/codefuse-ai/codefuse)✨✨
+
+<br>
+
+## Performance
+
+
+| Model                       | HumanEval(pass@1) |  Date   |
+|:----------------------------|:-----------------:|:-------:|
+| **CodeFuse-CodeLlama-34B**  |     **74.4%**      | 2023.9  |
+|**CodeFuse-CodeLlama-34B-4bits** |     **73.8%**  |  2023.9 |
+| WizardCoder-Python-34B-V1.0 |       73.2%       | 2023.8  |
+| GPT-4(zero-shot)            |       67.0%       | 2023.3  |
+| PanGu-Coder2 15B            |       61.6%       | 2023.8  |
+| CodeLlama-34b-Python        |       53.7%       | 2023.8  |
+| CodeLlama-34b               |       48.8%       | 2023.8  |
+| GPT-3.5(zero-shot)          |       48.1%       | 2022.11 |
+| OctoCoder                   |       46.2%       | 2023.8  |
+| StarCoder-15B               |       33.6%       | 2023.5  |
+| Qwen-14b               |       32.3%       | 2023.10  |
+| **CodeFuse-StarCoder-15B**  |     **54.9%**     | 2023.9  |
+| **CodeFuse-QWen-14B**       |     **48.78%**    | 2023.10 |
+| **CodeFuse-CodeGeeX2-6B**       |     **45.12%**    | 2023.11 |
+
+
+<br>
+
+## Requirements
+
+* python>=3.8 
+* pytorch>=2.0.0
+* transformers==4.33.2
+* Sentencepiece
+* CUDA 11.4
+  <br>
+
+##  Inference String Format
+
+The inference string is a concatenated string formed by combining conversation data(system, human and bot contents) in the training data format.  It is used as input during the inference process.
+Here is an example format of the concatenated string:
+
+```python
+"""
+<s>system
+System instruction
+<s>human
+Human 1st round input
+<s>bot
+Bot 1st round output<|endoftext|>
+<s>human
+Human 2nd round input
+<s>bot
+Bot 2nd round output<|endoftext|>
+...
+...
+...
+<s>human
+Human nth round input
+<s>bot
+{Bot output to be genreated}<|endoftext|>
+"""
+```
+
+When applying inference, you always make your input string end with "\<s\>bot" to ask the model generating answers.
+
+
+## Quickstart
+
+
+```bash
+pip install transformers modelscope cpm_kernels -U
+pip install -r requirements.txt
+```
+
+```python
+import torch
+from modelscope import (
+    AutoTokenizer, 
+    AutoModel,
+    snapshot_download
+)
+model_dir = snapshot_download('codefuse-ai/CodeFuse-CodeGeeX2-6B',revision = 'v1.0.0')
+tokenizer = AutoTokenizer.from_pretrained(model_dir, trust_remote_code=True)
+tokenizer.padding_side = "left"
+# try 4bit loading if cuda memory not enough
+model = AutoModel.from_pretrained(model_dir,
+                                             trust_remote_code=True,
+                                             load_in_4bit=False,
+                                             device_map="auto",
+                                             torch_dtype=torch.bfloat16)
+model.eval()
+
+HUMAN_ROLE_START_TAG = "<s>human\n"
+BOT_ROLE_START_TAG = "<s>bot\n"
+
+text = f"{HUMAN_ROLE_START_TAG}write a python function of quick sort.\n{BOT_ROLE_START_TAG}" 
+inputs = tokenizer(text, return_tensors='pt', padding=True, add_special_tokens=False).to("cuda")
+outputs = model.generate(
+        inputs=inputs["input_ids"],
+        attention_mask=inputs["attention_mask"],
+        max_new_tokens=512,
+        top_p=0.95,
+        temperature=0.1,
+        do_sample=True,
+        eos_token_id=tokenizer.eos_token_id,
+        pad_token_id=tokenizer.pad_token_id
+    )
+
+gen_text = tokenizer.batch_decode(outputs[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)
+print(gen_text[0])
+```
+
+
+
+
+
+
+
+
+<a id="chinese"></a>
+
+## 模型简介
+
+CodeFuse-CodeGeeX2-6B 是一个通过LoRA对基座模型CodeGeeeX2进行多代码任务微调的代码大模型。
+<br>
+
+## 新闻
+
+  🔥🔥 2023-11-10 开源了CodeFuse-CodeGeeX2-6B模型，在HumanEval pass@1(greedy decoding)上可以达到48.12%, 比CodeGeeX2提高了9.22%的代码能力（HumanEval）
+
+  🔥🔥 2023-10-20 公布了CodeFuse-QWen-14B技术文档，感兴趣详见微信公众号CodeFuse文章：https://mp.weixin.qq.com/s/PCQPkvbvfxSPzsqjOILCDw
+
+  🔥🔥 2023-10-16开源了CodeFuse-QWen-14B模型，在HumanEval pass@1(greedy decoding)上可以达到48.78%, 比Qwen-14b提高了16%的代码能力（HumanEval）
+
+  🔥🔥 2023-09-27开源了CodeFuse-StarCoder-15B模型，在HumanEval pass@1(greedy decoding)上可以达到54.9%, 比StarCoder提高了21%的代码能力（HumanEval）
+
+🔥🔥🔥 2023-09-26 [CodeFuse-CodeLlama-34B 4bits](https://modelscope.cn/models/codefuse-ai/CodeFuse-CodeLlama-34B-4bits/summary)量化版本发布，量化后模型在HumanEval pass@1指标为73.8% (贪婪解码)。
+
+🔥🔥🔥 2023-09-11 [CodeFuse-CodeLlama-34B](https://modelscope.cn/models/codefuse-ai/CodeFuse-CodeLlama-34B/summary)发布，HumanEval pass@1指标达到74.4% (贪婪解码), 为当前开源SOTA。
+
+<br>
+
+## 代码社区
+**大本营**： 🏡 https://github.com/codefuse-ai （**请支持我们的项目Star🌟 + Fork🚀 + Watch👀**）
+
++ 如果您想自己微调该模型，可以访问 ✨[MFTCoder](https://github.com/codefuse-ai/MFTCoder)✨✨
+
++ 如果您想自己部署该模型，可以访问 ✨[FasterTransformer4CodeFuse](https://github.com/codefuse-ai/FasterTransformer4CodeFuse)✨✨
+
++ 如果您想观看该模型示例，可以访问 ✨[CodeFuse Demo](https://github.com/codefuse-ai/codefuse)✨✨
+
+<br>
+
+
+## 评测表现
+
+### 代码
+
+
+| 模型                          | HumanEval(pass@1) |   日期    |
+|:----------------------------|:-----------------:|:-------:|
+| **CodeFuse-CodeLlama-34B**  |     **74.4%**      | 2023.9  |
+|**CodeFuse-CodeLlama-34B-4bits** |     **73.8%**  |  2023.9 |
+| WizardCoder-Python-34B-V1.0 |       73.2%       | 2023.8  |
+| GPT-4(zero-shot)            |       67.0%       | 2023.3  |
+| PanGu-Coder2 15B            |       61.6%       | 2023.8  |
+| CodeLlama-34b-Python        |       53.7%       | 2023.8  |
+| CodeLlama-34b               |       48.8%       | 2023.8  |
+| GPT-3.5(zero-shot)          |       48.1%       | 2022.11 |
+| OctoCoder                   |       46.2%       | 2023.8  |
+| StarCoder-15B               |       33.6%       | 2023.5  |
+| Qwen-14b               |       32.3%       | 2023.10  |
+| **CodeFuse-StarCoder-15B**  |     **54.9%**     | 2023.9  |
+| **CodeFuse-QWen-14B**       |     **48.78%**     | 2023.8 |
+| **CodeFuse-CodeGeeX2-6B**       |     **45.12%**    | 2023.11 |
+
+
+## Requirements
+
+* python>=3.8 
+* pytorch>=2.0.0
+* transformers==4.33.2
+* Sentencepiece
+* CUDA 11.4
+<br>
+
+## 推理数据格式
+
+推理数据为模型在训练数据格式下拼接的字符串形式，它也是推理时输入prompt拼接的方式：
+
+```python
+"""
+<s>system
+这是System指令
+<s>human
+这是第1轮用户输入的问题
+<s>bot
+这是第1轮模型生成的内容<|endoftext|>
+<s>human
+这是第2轮用户输入的问题
+<s>bot
+这是第2轮模型生成的内容<|endoftext|>
+...
+...
+...
+<s>human
+这是第n轮用户输入的问题
+<s>bot
+{模型现在要生成的内容}<|endoftext|>
+"""
+```
+
+推理时，请确保拼接的prompt字符串以"\<s\>bot\n"结尾，引导模型生成回答。
+
+## 快速使用
+
+
+```bash
+pip install transformers modelscope cpm_kernels -U
+pip install -r requirements.txt
+```
+
+```python
+import torch
+from modelscope import (
+    AutoTokenizer, 
+    AutoModel,
+    snapshot_download
+)
+model_dir = snapshot_download('codefuse-ai/CodeFuse-CodeGeeX2-6B',revision = 'v1.0.0')
+tokenizer = AutoTokenizer.from_pretrained(model_dir, trust_remote_code=True)
+tokenizer.padding_side = "left"
+# try 4bit loading if cuda memory not enough
+model = AutoModel.from_pretrained(model_dir,
+                                             trust_remote_code=True,
+                                             load_in_4bit=False,
+                                             device_map="auto",
+                                             torch_dtype=torch.bfloat16)
+model.eval()
+
+HUMAN_ROLE_START_TAG = "<s>human\n"
+BOT_ROLE_START_TAG = "<s>bot\n"
+
+text = f"{HUMAN_ROLE_START_TAG}write a python function of quick sort.\n{BOT_ROLE_START_TAG}" 
+inputs = tokenizer(text, return_tensors='pt', padding=True, add_special_tokens=False).to("cuda")
+outputs = model.generate(
+        inputs=inputs["input_ids"],
+        attention_mask=inputs["attention_mask"],
+        max_new_tokens=512,
+        top_p=0.95,
+        temperature=0.1,
+        do_sample=True,
+        eos_token_id=tokenizer.eos_token_id,
+        pad_token_id=tokenizer.pad_token_id
+    )
+
+gen_text = tokenizer.batch_decode(outputs[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)
+print(gen_text[0])
+```