Upload 7 files

README.md

@@ -1,3 +1,158 @@
 ---
 license: mit
+tags:
+- emotion-classification
+- mental-health
+- multi-label
+- transformers
+- distilbert
+- goemotions
+language:
+- en
+metrics:
+- f1
+- precision
+- recall
+pipeline_tag: text-classification
+base_model: distilbert-base-uncased
 ---
+
+# Mental Health Emotion Detection - Enhanced DistilBERT
+
+This model is a fine-tuned DistilBERT for multi-label emotion classification in mental health applications, detecting 28 different emotions from text input with enhanced architecture and advanced training techniques.
+
+## Model Description
+
+- **Model Type:** Enhanced DistilBERT (Fine-tuned)
+- **Base Model:** distilbert-base-uncased
+- **Task:** Multi-label emotion classification
+- **Dataset:** GoEmotions (balanced and enhanced)
+- **Languages:** English
+- **Architecture:** Enhanced with additional layers, focal loss, and class balancing
+
+## Performance
+
+| Metric | Score |
+|--------|-------|
+| F1-Score | 0.298 |
+| Precision | 0.459 |
+| Recall | 0.260 |
+| Accuracy | 89.5% |
+| Improvement | 7.6x over baseline |
+
+## Emotions Detected
+
+The model can detect 28 emotions: admiration, amusement, anger, annoyance, approval, caring, confusion, curiosity, desire, disappointment, disapproval, disgust, embarrassment, excitement, fear, gratitude, grief, joy, love, nervousness, optimism, pride, realization, relief, remorse, sadness, surprise, neutral.
+
+## Usage
+
+```python
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+import torch
+
+# Load model and tokenizer
+tokenizer = AutoTokenizer.from_pretrained("YOUR_USERNAME/mental-health-enhanced-distilbert")
+model = AutoModelForSequenceClassification.from_pretrained("YOUR_USERNAME/mental-health-enhanced-distilbert")
+
+# Example usage
+text = "I'm feeling really anxious about tomorrow"
+inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True, max_length=128)
+
+with torch.no_grad():
+    outputs = model(**inputs)
+    predictions = torch.sigmoid(outputs.logits)
+
+# Get emotion labels
+emotions = []
+for i, score in enumerate(predictions[0]):
+    if score > 0.4:  # Threshold
+        emotion = model.config.id2label[i]
+        emotions.append((emotion, score.item()))
+
+print(emotions)
+```
+
+## Training Details
+
+### Enhanced Architecture
+- **Base:** DistilBERT with additional hidden layers
+- **Enhancements:** 
+  - Layer normalization
+  - Dropout regularization
+  - Enhanced forward pass with ReLU activations
+  - Multi-layer classification head (768 → 512 → 256 → 128 → 28)
+
+### Advanced Training Techniques
+- **Loss Function:** Focal Loss for class imbalance handling
+- **Class Weighting:** Advanced weighting for rare emotions
+- **Data Balancing:** Oversampling rare emotions, undersampling common ones
+- **Optimization:** AdamW with cosine scheduling
+- **Early Stopping:** Patience-based with best model saving
+
+### Training Data
+- **Dataset:** GoEmotions (balanced subset)
+- **Training Samples:** ~12,750
+- **Validation Samples:** ~2,250
+- **Preprocessing:** Contraction expansion, lowercase normalization
+- **Balancing:** Advanced sampling for 28 emotion categories
+
+## Model Architecture
+
+```
+Input Text → DistilBERT Encoder → Enhanced Classification Head
+                                        ↓
+                                   Hidden Layer 1 (768→512)
+                                        ↓
+                                   Hidden Layer 2 (512→256)  
+                                        ↓
+                                   Hidden Layer 3 (256→128)
+                                        ↓
+                                   Output Layer (128→28)
+```
+
+## Intended Use
+
+This model is designed for:
+- Mental health chatbots and companions
+- Emotion-aware dialogue systems
+- Mental health screening tools
+- Research in computational psychology
+- Empathetic AI applications
+
+## Limitations
+
+- Trained primarily on English text
+- Performance may vary with very informal language
+- Should not be used as sole diagnostic tool for mental health
+- Requires context for optimal performance
+
+## Training Metrics by Epoch
+
+| Epoch | F1-Score | Precision | Recall |
+|-------|----------|-----------|--------|
+| 1 | 0.0145 | 0.0419 | 0.0089 |
+| 2 | 0.1430 | 0.2797 | 0.1211 |
+| 3 | 0.2141 | 0.4751 | 0.1804 |
+| 4 | 0.2749 | 0.4317 | 0.2340 |
+| 5 | 0.2897 | 0.4524 | 0.2533 |
+| 6 | 0.2981 | 0.4592 | 0.2597 |
+
+## Citation
+
+If you use this model, please cite:
+
+```
+@misc{mental-health-emotion-distilbert,
+  title={Mental Health Emotion Detection - Enhanced DistilBERT},
+  author={Your Name},
+  year={2024},
+  publisher={Hugging Face},
+  url={https://huggingface.co/YOUR_USERNAME/mental-health-enhanced-distilbert}
+}
+```
+
+## Acknowledgments
+
+- Built on DistilBERT by Hugging Face
+- Trained on GoEmotions dataset
+- Enhanced with advanced ML techniques for mental health applications

config.json

@@ -0,0 +1,79 @@
+{
+  "activation": "gelu",
+  "attention_dropout": 0.1,
+  "dim": 768,
+  "dropout": 0.1,
+  "hidden_dim": 3072,
+  "id2label": {
+    "0": "admiration",
+    "1": "amusement",
+    "2": "anger",
+    "3": "annoyance",
+    "4": "approval",
+    "5": "caring",
+    "6": "confusion",
+    "7": "curiosity",
+    "8": "desire",
+    "9": "disappointment",
+    "10": "disapproval",
+    "11": "disgust",
+    "12": "embarrassment",
+    "13": "excitement",
+    "14": "fear",
+    "15": "gratitude",
+    "16": "grief",
+    "17": "joy",
+    "18": "love",
+    "19": "nervousness",
+    "20": "optimism",
+    "21": "pride",
+    "22": "realization",
+    "23": "relief",
+    "24": "remorse",
+    "25": "sadness",
+    "26": "surprise",
+    "27": "neutral"
+  },
+  "initializer_range": 0.02,
+  "label2id": {
+    "admiration": 0,
+    "amusement": 1,
+    "anger": 2,
+    "annoyance": 3,
+    "approval": 4,
+    "caring": 5,
+    "confusion": 6,
+    "curiosity": 7,
+    "desire": 8,
+    "disappointment": 9,
+    "disapproval": 10,
+    "disgust": 11,
+    "embarrassment": 12,
+    "excitement": 13,
+    "fear": 14,
+    "gratitude": 15,
+    "grief": 16,
+    "joy": 17,
+    "love": 18,
+    "nervousness": 19,
+    "neutral": 27,
+    "optimism": 20,
+    "pride": 21,
+    "realization": 22,
+    "relief": 23,
+    "remorse": 24,
+    "sadness": 25,
+    "surprise": 26
+  },
+  "max_position_embeddings": 512,
+  "model_type": "distilbert",
+  "n_heads": 12,
+  "n_layers": 6,
+  "pad_token_id": 0,
+  "problem_type": "multi_label_classification",
+  "qa_dropout": 0.1,
+  "seq_classif_dropout": 0.2,
+  "sinusoidal_pos_embds": false,
+  "transformers_version": "4.56.0",
+  "vocab_size": 30522
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b844b4253f4b7502efc99fa889584c27425d58bbfc82550d70f9259d60165c10
+size 267749227

requirements.txt

@@ -0,0 +1,3 @@
+transformers>=4.21.0
+torch>=1.12.0
+numpy>=1.21.0

special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

tokenizer_config.json

@@ -0,0 +1,58 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "never_split": null,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "DistilBertTokenizer",
+  "unk_token": "[UNK]"
+}