---
license: mit
language:
- en
tags:
- mental-health
- text-generation
- conversational-ai
- gpt2
pipeline_tag: text-generation
base_model: openai-community/gpt2
---

# GPT-2 (Fine-Tuned for MindPadi)

This is a fine-tuned version of GPT-2 for the **MindPadi** mental health chatbot. It has been adapted to generate empathetic, therapeutic, and contextually relevant responses for mental health support. It is the primary generative model used in long-form conversation and therapy-related dialogue management in MindPadi.



## 🧠 Model Summary

- **Model Type:** GPT-2 (12-layer transformer)
- **Parameters:** ~124M
- **Fine-Tuned For:** Empathetic and supportive text generation
- **Used In:** `app/chatbot/fusion_bot.py`
- **Architecture:** Decoder-only transformer (causal LM)
- **Framework:** Hugging Face Transformers + PyTorch



## 🧾 Intended Use

### ✔️ Primary Use Cases
- Generating thoughtful, compassionate responses in mental health conversations
- Completing sentences in a therapy dialogue setting
- Supporting GPTFusion workflows in MindPadi backend

### 🚫 Not Recommended For
- Clinical diagnoses or medical decisions
- Domains outside mental health (e.g., finance, legal)
- Multilingual generation (trained on English only)



## 🏋️‍♀️ Training Details

- **Base Model:** `gpt2` from Hugging Face
- **Fine-Tuning Script:** `training/finetune_gpt2_pipeline.py`
- **Datasets:** Mental health dialogue datasets (e.g., therapy transcripts, Reddit mental health subreddits)
  - Location: `training/datasets/finetuned/`
- **Preprocessing:**
  - Cleaned for profanity, PII, and formatting noise
  - Converted to conversation format: `User: ... Assistant: ...`

### Hyperparameters
- Epochs: 3–5
- Batch Size: 4–8
- Learning Rate: 5e-5
- Warmup Steps: 200
- Optimizer: AdamW

### Hardware
- NVIDIA RTX 2080 / A100 / equivalent (local or cloud)



## 📈 Evaluation

- **Evaluation Script:** `training/evaluate_model.py`
- **Metrics:**
  - Perplexity: Reduced ~20% compared to base GPT-2
  - BLEU Score: Improved ~12% in test responses
  - Human Evaluation: Higher relevance and emotional alignment in blind tests
- **Examples:** Available in `logs/training.log` and test dialogues



## 📂 Files

| File | Purpose |
|------|---------|
| `config.json` | Model architecture |
| `pytorch_model.bin` or `model.safetensors` | Fine-tuned weights |
| `tokenizer.json`, `vocab.json`, `merges.txt` | Tokenizer and BPE merge rules |
| `checkpoint-*/` | Intermediate training checkpoints (optional) |



## 💬 Example Usage

```python
from transformers import GPT2LMHeadModel, GPT2Tokenizer

model = GPT2LMHeadModel.from_pretrained("mindpadi/gpt2")
tokenizer = GPT2Tokenizer.from_pretrained("mindpadi/gpt2")

prompt = "User: I feel hopeless and tired.\nAssistant:"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_length=100, do_sample=True)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))
````



## 🔧 Integration in MindPadi

This model is integrated into:

* `app/chatbot/fusion_bot.py`: Primary text generator
* `app/chatbot/gpt_router.py`: Fusion routing between GPT-2 and DistilGPT2
* `app/chatbot/core.py`: Chat interface logic
* LangGraph workflows: Via GPTFusionRouter nodes



## ⚠️ Limitations

* **Bias:** May carry biases from internet pretraining or mental health corpora
* **Language:** English-only
* **Token Limit:** \~1024 tokens context window
* **Sensitivity:** May generate inappropriate responses if inputs are adversarial or out-of-distribution
* **Non-Determinism:** Sampling may produce varied outputs even for the same input



## 🔐 Ethical Considerations

* Not intended for crisis response or as a substitute for professional help
* Includes guardrails to detect emergency keywords (e.g., "suicide", "abuse")
* Should always inform users they are interacting with AI (see `chat_interface.py`)
* Responses are not clinically validated; user discretion advised



## 🧪 Deployment

You can deploy this model via Hugging Face Inference Endpoints for GPU-powered responses.

```python
import requests

api_url = "https://<your-endpoint>.hf.space"
headers = {"Authorization": f"Bearer <your-token>", "Content-Type": "application/json"}

payload = {"inputs": "User: I feel anxious.\nAssistant:"}
response = requests.post(api_url, headers=headers, json=payload)
print(response.json())
```



## 📜 License

MIT License – Free for commercial and non-commercial use with attribution.



## 📬 Contact

* **Project:** [MindPadi AI Chatbot](https://huggingface.co/mindpadi)
* **Maintainer:** MindPadi Team
* **Email:** \[[you@example.com](mailto:you@example.com)]
* **GitHub:** \[github.com/mindpadi/mindpadi]

*Last updated: May 2025*