Neurazum/Vbai-DPA-2.4

Vbai-DPA 2.4 Sürümü (TR)

Model	Boyut	Parametre	mAPᵛᵃᴵ	APᵛᵃᴵ	CPU b1	V100 b1	V100 b32
Vbai-DPA 2.4f	224	38.68M	%56.66	%53.80	28.8ms	14.4ms	3.64ms
Vbai-DPA 2.4c	224	77.52M	%62.87	%60.54	40.33ms	20.16ms	4.03ms
Vbai-DPA 2.4q	224	131.30M	%82.93	%87.45	85.48ms	42.74ms	8.55ms

Tanım

Vbai-DPA 2.4 (Dementia, Parkinson, Alzheimer) modeli, MRI veya fMRI görüntüsü üzerinden beyin hastalıklarını teşhis etmek amacıyla eğitilmiş ve geliştirilmiştir. Hastanın parkinson olup olmadığını, demans durumunu ve alzheimer riskini yüksek doğruluk oranı ile göstermektedir. Vbai-DPA 2.3'e göre görüntüde işaretleme yapma özelliği eklenip, ince ayar ve daha fazla veri ile eğitilmiş versiyonlarıdır.

Kitle / Hedef

Vbai modelleri tamamen öncelik olarak hastaneler, sağlık merkezleri ve bilim merkezleri için geliştirilmiştir.

Sınıflar

• Alzheimer Hastası: Hasta kişi, kesinlikle alzheimer hastasıdır.
• Ortalama Alzheimer Riski: Hasta kişi, yakın bir zamanda alzheimer olabilir.
• Hafif Alzheimer Riski: Hasta kişinin, alzheimer olması için biraz daha zamanı vardır.
• Çok Hafif Alzheimer Riski: Hasta kişinin, alzheimer seviyesine gelmesine zaman vardır.
• Risk Yok: Kişinin herhangi bir riski bulunmamaktadır.
• Parkinson Hastası: Kişi, parkinson hastasıdır.

----------------------------------------

Vbai-DPA 2.4 Versions (EN)

Model	Test Size	Params	mAPᵛᵃᴵ	APᵛᵃᴵ	CPU b1	V100 b1	V100 b32
Vbai-DPA 2.4f	224	38.68M	56.66%	53.80%	28.8ms	14.4ms	3.64ms
Vbai-DPA 2.4c	224	77.52M	62.87%	60.54%	40.33ms	20.16ms	4.03ms
Vbai-DPA 2.4q	224	131.30M	82.93%	87.45%	85.48ms	42.74ms	8.55ms

Description

The Vbai-DPA 2.4 (Dementia, Parkinson's, Alzheimer's) model has been trained and developed to diagnose brain diseases using MRI or fMRI images. It indicates whether the patient has Parkinson's disease, dementia, and Alzheimer's risk with a high accuracy rate. Compared to Vbai-DPA 2.3, it features image annotation capabilities and has been trained with fine-tuning and additional data.

Audience / Target

Vbai models are developed exclusively for hospitals, health centres and science centres.

Classes

• Alzheimer's disease: The sick person definitely has Alzheimer's disease.
• Average Risk of Alzheimer's Disease: The sick person may develop Alzheimer's disease in the near future.
• Mild Alzheimer's Risk: The sick person has a little more time to develop Alzheimer's disease.
• Very Mild Alzheimer's Risk: The sick person has time to reach the level of Alzheimer's disease.
• No Risk: The person does not have any risk.
• Parkinson's Disease: The person has Parkinson's disease.

Model

import torch
import torch.nn as nn
import torch.nn.functional as F


class CBAMAttentionCheckpoint(nn.Module):
    def __init__(self, channels, reduction=8):
        super(CBAMAttentionCheckpoint, self).__init__()

        # Channel attention (using Conv2d 1x1 as in checkpoint, NO BIAS)
        reduced_channels = max(channels // reduction, 1)
        self.channel_attention = nn.Sequential(
            nn.Conv2d(channels, reduced_channels, 1, bias=False),
            nn.ReLU(),
            nn.Conv2d(reduced_channels, channels, 1, bias=False),
        )

        # Spatial attention (7x7 conv as in checkpoint, NO BIAS)
        self.spatial_attention = nn.Sequential(
            nn.Conv2d(2, 1, 7, padding=3, bias=False),
        )

    def forward(self, x):
        avg_pool = F.adaptive_avg_pool2d(x, 1)
        max_pool = F.adaptive_max_pool2d(x, 1)

        avg_out = self.channel_attention(avg_pool)
        max_out = self.channel_attention(max_pool)

        channel_att = torch.sigmoid(avg_out + max_out)
        x = x * channel_att

        avg_out = torch.mean(x, dim=1, keepdim=True)
        max_out, _ = torch.max(x, dim=1, keepdim=True)
        spatial_in = torch.cat([avg_out, max_out], dim=1)

        spatial_att = torch.sigmoid(self.spatial_attention(spatial_in))
        x = x * spatial_att

        return x


class CheckpointVbaiDPA24(nn.Module):
    def __init__(self, model_type='f', num_classes=6):
        super(CheckpointVbaiDPA24, self).__init__()

        self.model_type = model_type
        self.num_classes = num_classes

        # Conv layers: Sequential with Conv+BN+ReLU+Pool pattern
        if model_type == 'f':
            self.conv_layers = nn.Sequential(
                nn.Conv2d(3, 16, 3, padding=1),
                nn.BatchNorm2d(16),
                nn.ReLU(),
                nn.MaxPool2d(2, 2),
                nn.Conv2d(16, 32, 3, padding=1),
                nn.BatchNorm2d(32),
                nn.ReLU(),
                nn.MaxPool2d(2, 2),
                nn.Conv2d(32, 64, 3, padding=1),
                nn.BatchNorm2d(64),
                nn.ReLU(),
                nn.MaxPool2d(2, 2),
            )
            self.attention_modules = nn.ModuleList([
                CBAMAttentionCheckpoint(16, reduction=8),
                CBAMAttentionCheckpoint(32, reduction=8),
                CBAMAttentionCheckpoint(64, reduction=8),
            ])
            fc_input = 64 * 28 * 28
            fc_sizes = [256, 128, 6]

        elif model_type == 'c':
            self.conv_layers = nn.Sequential(
                nn.Conv2d(3, 32, 3, padding=1),
                nn.BatchNorm2d(32),
                nn.ReLU(),
                nn.MaxPool2d(2, 2),
                nn.Conv2d(32, 64, 3, padding=1),
                nn.BatchNorm2d(64),
                nn.ReLU(),
                nn.MaxPool2d(2, 2),
                nn.Conv2d(64, 128, 3, padding=1),
                nn.BatchNorm2d(128),
                nn.ReLU(),
                nn.MaxPool2d(2, 2),
            )
            self.attention_modules = nn.ModuleList([
                CBAMAttentionCheckpoint(32, reduction=8),
                CBAMAttentionCheckpoint(64, reduction=8),
                CBAMAttentionCheckpoint(128, reduction=8),
            ])
            fc_input = 128 * 28 * 28
            fc_sizes = [512, 256, 6]

        elif model_type == 'q':
            self.conv_layers = nn.Sequential(
                nn.Conv2d(3, 64, 3, padding=1),
                nn.BatchNorm2d(64),
                nn.ReLU(),
                nn.MaxPool2d(2, 2),
                nn.Conv2d(64, 128, 3, padding=1),
                nn.BatchNorm2d(128),
                nn.ReLU(),
                nn.MaxPool2d(2, 2),
                nn.Conv2d(128, 256, 3, padding=1),
                nn.BatchNorm2d(256),
                nn.ReLU(),
                nn.MaxPool2d(2, 2),
                nn.Conv2d(256, 512, 3, padding=1),
                nn.BatchNorm2d(512),
                nn.ReLU(),
                nn.MaxPool2d(2, 2),
            )
            self.attention_modules = nn.ModuleList([
                CBAMAttentionCheckpoint(64, reduction=8),
                CBAMAttentionCheckpoint(128, reduction=8),
                CBAMAttentionCheckpoint(256, reduction=8),
                CBAMAttentionCheckpoint(512, reduction=8),
            ])
            fc_input = 512 * 14 * 14
            fc_sizes = [1024, 512, 6]

        # Edge detection branch
        self.edge_conv1 = nn.Conv2d(1, 32, 3, padding=1)
        self.edge_conv2 = nn.Conv2d(32, 64, 3, padding=1)
        # Edge fc input size depends on pooling
        self.edge_fc = nn.Linear(64 * 56 * 56, 128)

        # Main classifier
        self.classifier = nn.Sequential(
            nn.Linear(fc_input, fc_sizes[0]),
            nn.ReLU(),
            nn.Dropout(0.5),
            nn.Linear(fc_sizes[0], fc_sizes[1]),
            nn.ReLU(),
            nn.Dropout(0.5),
            nn.Linear(fc_sizes[1], fc_sizes[2]),
        )

        # Combined classifier (with edge features)
        self.combined_classifier = nn.Sequential(
            nn.Linear(fc_sizes[0] + 128, fc_sizes[0]),
            nn.ReLU(),
            nn.Dropout(0.5),
            nn.Linear(fc_sizes[0], fc_sizes[2]),
        )

    def forward(self, x, edge_x=None):
        if self.model_type == 'f' or self.model_type == 'c':
            x = self.conv_layers[0:4](x)
            x = self.attention_modules[0](x)

            x = self.conv_layers[4:8](x)
            x = self.attention_modules[1](x)

            x = self.conv_layers[8:12](x)
            x = self.attention_modules[2](x)
            attention_map = x

        elif self.model_type == 'q':
            x = self.conv_layers[0:4](x)
            x = self.attention_modules[0](x)

            x = self.conv_layers[4:8](x)
            x = self.attention_modules[1](x)

            x = self.conv_layers[8:12](x)
            x = self.attention_modules[2](x)

            x = self.conv_layers[12:16](x)
            x = self.attention_modules[3](x)
            attention_map = x

        x = x.view(x.size(0), -1)

        x = self.classifier[0](x)
        x = self.classifier[1](x)
        features = self.classifier[2](x)

        if edge_x is not None:
            try:
                edge_x = F.relu(self.edge_conv1(edge_x))
                edge_x = F.max_pool2d(edge_x, 2, 2)

                edge_x = F.relu(self.edge_conv2(edge_x))
                edge_x = F.max_pool2d(edge_x, 2, 2)

                edge_features = edge_x.view(edge_x.size(0), -1)
                edge_features = self.edge_fc(edge_features)

                combined = torch.cat([features, edge_features], dim=1)
                output = self.combined_classifier(combined)
            except Exception as e:
                output = self.classifier[3:](features)
        else:
            output = self.classifier[3:](features)

        return output, attention_map

Kullanım / Usage

import os
import torch
import torch.nn.functional as F
from torchvision import transforms
from PIL import Image
import matplotlib.pyplot as plt
import numpy as np
import cv2
from pytorch_grad_cam import GradCAM
from pytorch_grad_cam.utils.model_targets import ClassifierOutputTarget
from checkpoint_model import CheckpointVbaiDPA24

TEST_IMAGE = "test/image/path"

MODEL = 'c' # Model type | f => fast, c => classic, q => quality

MODEL_PATHS = {
    'f': 'Vbai-DPA 2.4f.pt/model/path',
    'c': 'Vbai-DPA 2.4c.pt/model/path',
    'q': 'Vbai-DPA 2.4q.pt/model/path',
}

CLASS_NAMES = [
    'AD Alzheimer Diseases',
    'AD Mild Demented',
    'AD Moderate Demented',
    'AD Very Mild Demented',
    'CN Non Demented',
    'PD Parkinson Diseases'
]

transform = transforms.Compose([
    transforms.Resize((224, 224)),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])

edge_transform = transforms.Compose([
    transforms.Resize((224, 224)),
    transforms.Grayscale(num_output_channels=1),
    transforms.ToTensor(),
])


def detect_brain_edges(edge_tensor):
    edge_np = edge_tensor.squeeze().cpu().numpy()
    edge_np = ((edge_np - edge_np.min()) / (edge_np.max() - edge_np.min() + 1e-8) * 255).astype(np.uint8)
    edges = cv2.Canny(edge_np, 50, 150)

    kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5, 5))
    edges_closed = cv2.morphologyEx(edges, cv2.MORPH_CLOSE, kernel)
    contours, _ = cv2.findContours(edges_closed, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
    contours = sorted(contours, key=cv2.contourArea, reverse=True)[:3]

    return edges, contours


def main():
    print("\n" + "="*70)
    print("Vbai-DPA 2.4")
    print("="*70)

    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
    print(f"✓ Device: {device}")

    print(f"✓ Loading model: Vbai-DPA 2.4{MODEL.upper()}")
    model = CheckpointVbaiDPA24(model_type=MODEL, num_classes=6).to(device)
    checkpoint = torch.load(MODEL_PATHS[MODEL], map_location=device, weights_only=False)
    model.load_state_dict(checkpoint.get('model_state_dict', checkpoint))
    model.eval()

    target_layer = model.conv_layers[12] if MODEL == 'q' else model.conv_layers[8]
    cam = GradCAM(model=model, target_layers=[target_layer])

    print(f"✓ Loading image: {os.path.basename(TEST_IMAGE)}")
    img = Image.open(TEST_IMAGE).convert('RGB')
    inp = transform(img).unsqueeze(0).to(device)
    edge_tensor = edge_transform(img).unsqueeze(0).to(device)

    print(f"✓ Prediction...")
    with torch.no_grad():
        output, attention_map = model(inp, edge_tensor)
        probs = F.softmax(output, dim=1)
        pred = output.argmax(1).item()
        conf = probs[0, pred].item() * 100
        top3_probs, top3_indices = torch.topk(probs[0], 3)

    print("\n" + "="*70)
    print("RESULTS:")
    print("="*70)
    print(f"✓ Main Pred: {CLASS_NAMES[pred]}")
    print(f"✓ Confidence Score: {conf:.2f}%")
    print(f"\nTop-3 Predictions:")
    for i, (idx, prob) in enumerate(zip(top3_indices, top3_probs), 1):
        print(f"  {i}. {CLASS_NAMES[idx.item()]}: {prob.item()*100:.2f}%")
    print("="*70)

    print(f"\n✓ Analysing brain structure...")
    edges, contours = detect_brain_edges(edge_tensor)

    print(f"✓ Calculating Grad-CAM...")
    cam_mask = cam(input_tensor=inp, targets=[ClassifierOutputTarget(pred)])[0]
    cam_mask = np.squeeze(cam_mask)

    att_np = attention_map[0].mean(0).cpu().numpy()
    att_np = (att_np - att_np.min()) / (att_np.max() - att_np.min() + 1e-8)
    att_np = cv2.resize(att_np, (224, 224))

    print(f"✓ Creating visualization...")
    fig, axes = plt.subplots(2, 3, figsize=(18, 12))

    axes[0, 0].imshow(img)
    axes[0, 0].set_title('Original MRI', fontsize=14, fontweight='bold')
    axes[0, 0].axis('off')

    axes[0, 1].imshow(edges, cmap='gray')
    axes[0, 1].set_title('Brain Edges', fontsize=14, fontweight='bold')
    axes[0, 1].axis('off')

    img_contours = np.array(img.resize((224, 224)))
    img_contours = cv2.cvtColor(img_contours, cv2.COLOR_RGB2BGR)
    if contours:
        cv2.drawContours(img_contours, contours, -1, (0, 255, 0), 2)
        for i, cnt in enumerate(contours):
            M = cv2.moments(cnt)
            if M['m00'] != 0:
                cx, cy = int(M['m10']/M['m00']), int(M['m01']/M['m00'])
                cv2.circle(img_contours, (cx, cy), 5, (255, 0, 0), -1)
                cv2.putText(img_contours, f"{i+1}", (cx+10, cy),
                          cv2.FONT_HERSHEY_SIMPLEX, 0.6, (255, 0, 0), 2)
    img_contours = cv2.cvtColor(img_contours, cv2.COLOR_BGR2RGB)
    axes[0, 2].imshow(img_contours)
    axes[0, 2].set_title('Brain Structure Signs', fontsize=14, fontweight='bold')
    axes[0, 2].axis('off')

    im = axes[1, 0].imshow(att_np, cmap='hot')
    axes[1, 0].set_title('Attention Map', fontsize=14, fontweight='bold')
    axes[1, 0].axis('off')
    plt.colorbar(im, ax=axes[1, 0], fraction=0.046)

    im2 = axes[1, 1].imshow(cam_mask, cmap='jet')
    axes[1, 1].set_title('Grad-CAM', fontsize=14, fontweight='bold')
    axes[1, 1].axis('off')
    plt.colorbar(im2, ax=axes[1, 1], fraction=0.046)

    rgb_np = inp[0].permute(1, 2, 0).cpu().numpy()
    rgb_np = (rgb_np - rgb_np.min()) / (rgb_np.max() - rgb_np.min())
    cam_colored = cv2.applyColorMap(np.uint8(cam_mask * 255), cv2.COLORMAP_JET)
    cam_colored = cv2.cvtColor(cam_colored, cv2.COLOR_BGR2RGB) / 255.0
    overlay = cam_colored * 0.5 + rgb_np * 0.5
    overlay = np.clip(overlay, 0, 1)
    axes[1, 2].imshow(overlay)
    axes[1, 2].set_title(f'Prediction: {CLASS_NAMES[pred]}\nConfidence: {conf:.1f}%',
                        fontsize=14, fontweight='bold')
    axes[1, 2].axis('off')

    fig.suptitle(f'Vbai-DPA 2.4{MODEL.upper()} - {os.path.basename(TEST_IMAGE)}',
                fontsize=16, fontweight='bold', y=0.98)

    plt.tight_layout()

    output_file = f'quick_test_result_{MODEL}.png'
    plt.savefig(output_file, dpi=150, bbox_inches='tight')
    print(f"✓ Results saved: {output_file}")

    plt.show()

    print("\n✓ Test Completed!")
    print("="*70 + "\n")


if __name__ == '__main__':
    try:
        main()
    except Exception as e:
        print(f"\n✗ ERROR: {str(e)}")
        import traceback
        traceback.print_exc()