Create README.md

README.md

@@ -0,0 +1,148 @@
+---
+license: mit
+language:
+- en
+base_model:
+- google-bert/bert-base-uncased
+---
+# RustBusters BERT Relevance Assessment Model Card
+
+## Model Description
+
+**Model Name:** RustBusters-BERT-Relevance-Classifier  
+**Base Model:** bert-base-uncased  
+**Architecture:** BERT (Bidirectional Encoder Representations from Transformers)  
+**Task:** Binary Text Classification  
+**Version:** 1.0  
+**Last Updated:** March 2025  
+
+## Intended Use
+
+This model is designed to classify incoming customer queries as either relevant or not relevant to laser cleaning services. The model serves as a first-line filter to:
+
+- Identify queries related to laser cleaning that should be routed to RustBusters' customer service
+- Filter out unrelated queries to improve response efficiency
+- Help automate initial query triage in customer service workflows
+- Support chatbots and digital assistants in determining when to engage with laser cleaning queries
+
+## Training Details
+
+- **Base Model:** bert-base-uncased (110M parameters)
+- **Training Data:** 714 examples (571 training, 143 testing)
+  - Positive examples (relevant to laser cleaning): 557 (78%)
+  - Negative examples (not relevant to laser cleaning): 157 (22%)
+- **Training Method:** Fine-tuning with AdamW optimizer
+- **Training Parameters:**
+  - Learning rate: 2e-5
+  - Batch size: 16
+  - Epochs: 3
+  - Sequence length: 128 tokens
+- **Performance:**
+  - Final accuracy: 95.8%
+  - Precision for relevant class: 0.97
+  - Recall for relevant class: 0.97
+  - F1-score for relevant class: 0.97
+  - Precision for non-relevant class: 0.90
+  - Recall for non-relevant class: 0.90
+  - F1-score for non-relevant class: 0.90
+
+## Performance and Limitations
+
+- **Strengths:**
+  - High accuracy (95.8%) on test set
+  - Well-balanced precision and recall for both classes
+  - Effective at identifying laser cleaning related queries
+  - Small model size, efficient for deployment
+  - Fast inference times
+
+- **Limitations:**
+  - Limited to binary classification (relevant vs. not relevant)
+  - May struggle with highly ambiguous queries
+  - Cannot categorize queries by type, urgency, or complexity
+  - Limited exposure to industry-specific terminology beyond training data
+  - Performance dependent on queries being similar to training examples
+
+## Implementation Guidelines
+
+The model assigns label 1 for relevant queries and label 0 for non-relevant queries. Implementation should account for this labeling scheme:
+
+```python
+def classify_query(text):
+    # Tokenize input
+    encoding = tokenizer(
+        text,
+        add_special_tokens=True,
+        max_length=128,
+        padding='max_length',
+        truncation=True,
+        return_attention_mask=True,
+        return_tensors='pt'
+    )
+    
+    # Get prediction
+    model.eval()
+    with torch.no_grad():
+        outputs = model(
+            input_ids=encoding['input_ids'].to(device),
+            attention_mask=encoding['attention_mask'].to(device)
+        )
+        
+        # Apply softmax to get probabilities
+        probs = torch.nn.functional.softmax(outputs.logits, dim=1)[0]
+        class_0_prob = probs[0].item()  # Not relevant probability
+        class_1_prob = probs[1].item()  # Relevant probability
+        
+        # Simple threshold-based classification
+        predicted_class = 1 if class_1_prob > 0.5 else 0
+        
+        # Optional: Enhanced classification with keyword verification
+        laser_keywords = ["laser", "clean", "rust", "metal", "surface"]
+        contains_keywords = any(keyword in text.lower() for keyword in laser_keywords)
+        
+        # Return classification result
+        if predicted_class == 1 or contains_keywords:
+            return "Relevant to laser cleaning"
+        else:
+            return "Not relevant to laser cleaning"
+```
+
+## Data Characteristics
+
+The model was trained on a rich dataset containing:
+- Queries about laser cleaning services, pricing, processes, and applications
+- Questions about materials that can be laser cleaned (metals, industrial equipment, automotive parts)
+- Service area inquiries related to Huntsville and Alabama
+- Edge cases like general rust removal without mentioning laser
+- Negative examples including:
+  - General information requests unrelated to laser cleaning
+  - Other cleaning-related queries that aren't laser-specific
+  - Questions about completely different services and products
+
+The dataset was systematically expanded through:
+- Template-based generation with material/problem variations
+- Compound questions combining multiple aspects of laser cleaning
+- Paraphrasing of base examples
+- Inclusion of carefully labeled ambiguous examples
+
+## Ethical Considerations
+
+- **False Negatives:** Important customer inquiries might be misclassified as irrelevant
+- **Transparency:** Users should be informed if their queries are being automatically filtered
+- **Human Oversight:** Regular auditing of model classifications is recommended
+- **Bias:** Monitor for potential bias against certain query formulations or terminology
+
+## Maintenance Recommendations
+
+We recommend:
+- Periodically retraining with new customer queries to capture evolving language patterns
+- Monitoring performance metrics, especially on edge cases
+- Adding any consistently misclassified queries to the training dataset
+- Considering expansion to multi-class classification for more nuanced routing
+
+## Contact Information
+
+For issues, improvements, or questions about this model, please contact the RustBusters AI team.
+
+---
+
+*This model card follows best practices for AI documentation and transparency.*