{ "cells": [ { "cell_type": "markdown", "source": [ "# Cours 6: La toxicité" ], "metadata": { "id": "ND9o13T1WTVV" } }, { "cell_type": "markdown", "source": [ "Dans ce notebook, nous allons explorer comment nous pouvons utiliser le Machine Learning, et plus précisément les modèles de Natural Language Inference (NLI) et de Large Language Model (LLM), pour détecter la toxicité dans le texte et les menaces potentielles contenues dans les queries.\n", "\n", "Les modèles NLI sont des modèles d'apprentissage automatique qui peuvent déterminer la relation entre deux phrases, ce qui peut être utile pour identifier le contexte et la nuance dans le language. D'autre part, les LLM sont capables d'apprendre la structure et les subtilités d'une langue, ce qui peut être utile pour comprendre le texte à un niveau plus profond.\n", "\n", "Mais avant ça amusons-nous un peu 😊" ], "metadata": { "id": "LVOMzP1dZHLg" } }, { "cell_type": "markdown", "source": [ "## Comment prévenir la toxicité en IA\n", "\n", "Allez sur https://gandalf.lakera.ai/. Amusez vous à cracker le mot de passe retenu par le LLM. Essayez d'aller le plus loin possible. Quels sont les mécanismes pour hacker un LLM ?" ], "metadata": { "id": "5-hjrE22WTVZ" } }, { "cell_type": "markdown", "source": [ "## Comment détecter la toxicité en IA?\n", "Dans cette partie, nous allons nous interesser à comment détecter le langage toxique et les menaces potentielles. Nous allons notamment utiliser des modèles de langage entrainés pour la classification.\n", "Mais avant ça, importez les différentes bibliothèques et les différentes fonctions qui nous seront utiles pour la suite." ], "metadata": { "id": "KHVRWHMsWTVZ" } }, { "cell_type": "code", "execution_count": 3, "source": [ "!pip install transformers\n", "!pip install torch\n", "!pip install detoxify\n", "!pip install datasets\n", "!pip install scikit-learn\n", "!pip install evaluate" ], "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Requirement already satisfied: transformers in /usr/local/lib/python3.10/dist-packages (4.41.1)\n", "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers) (3.14.0)\n", "Requirement already satisfied: huggingface-hub<1.0,>=0.23.0 in /usr/local/lib/python3.10/dist-packages (from transformers) (0.23.2)\n", "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from transformers) (1.25.2)\n", "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from transformers) (23.2)\n", "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from transformers) (6.0.1)\n", "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers) (2024.5.15)\n", "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers) (2.32.3)\n", "Requirement already satisfied: tokenizers<0.20,>=0.19 in /usr/local/lib/python3.10/dist-packages (from transformers) (0.19.1)\n", "Requirement already satisfied: safetensors>=0.4.1 in /usr/local/lib/python3.10/dist-packages (from transformers) (0.4.3)\n", "Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/dist-packages (from transformers) (4.66.4)\n", "Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.23.0->transformers) (2023.6.0)\n", "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.23.0->transformers) (4.12.0)\n", "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->transformers) (3.3.2)\n", "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->transformers) (3.7)\n", "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->transformers) (2.0.7)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->transformers) (2024.2.2)\n", "Requirement already satisfied: torch in /usr/local/lib/python3.10/dist-packages (2.3.0+cu121)\n", "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch) (3.14.0)\n", "Requirement already satisfied: typing-extensions>=4.8.0 in /usr/local/lib/python3.10/dist-packages (from torch) (4.12.0)\n", "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch) (1.12.1)\n", "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch) (3.3)\n", "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch) (3.1.4)\n", "Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from torch) (2023.6.0)\n", "Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch) (12.1.105)\n", "Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch) (12.1.105)\n", "Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch) (12.1.105)\n", "Requirement already satisfied: nvidia-cudnn-cu12==8.9.2.26 in /usr/local/lib/python3.10/dist-packages (from torch) (8.9.2.26)\n", "Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /usr/local/lib/python3.10/dist-packages (from torch) (12.1.3.1)\n", "Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /usr/local/lib/python3.10/dist-packages (from torch) (11.0.2.54)\n", "Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /usr/local/lib/python3.10/dist-packages (from torch) (10.3.2.106)\n", "Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in /usr/local/lib/python3.10/dist-packages (from torch) (11.4.5.107)\n", "Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in /usr/local/lib/python3.10/dist-packages (from torch) (12.1.0.106)\n", "Requirement already satisfied: nvidia-nccl-cu12==2.20.5 in /usr/local/lib/python3.10/dist-packages (from torch) (2.20.5)\n", "Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch) (12.1.105)\n", "Requirement already satisfied: triton==2.3.0 in /usr/local/lib/python3.10/dist-packages (from torch) (2.3.0)\n", "Requirement already satisfied: nvidia-nvjitlink-cu12 in /usr/local/lib/python3.10/dist-packages (from nvidia-cusolver-cu12==11.4.5.107->torch) (12.5.40)\n", "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch) (2.1.5)\n", "Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from sympy->torch) (1.3.0)\n", "Requirement already satisfied: detoxify in /usr/local/lib/python3.10/dist-packages (0.5.2)\n", "Requirement already satisfied: transformers in /usr/local/lib/python3.10/dist-packages (from detoxify) (4.41.1)\n", "Requirement already satisfied: torch>=1.7.0 in /usr/local/lib/python3.10/dist-packages (from detoxify) (2.3.0+cu121)\n", "Requirement already satisfied: sentencepiece>=0.1.94 in /usr/local/lib/python3.10/dist-packages (from detoxify) (0.1.99)\n", "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (3.14.0)\n", "Requirement already satisfied: typing-extensions>=4.8.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (4.12.0)\n", "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (1.12.1)\n", "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (3.3)\n", "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (3.1.4)\n", "Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (2023.6.0)\n", "Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (12.1.105)\n", "Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (12.1.105)\n", "Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (12.1.105)\n", "Requirement already satisfied: nvidia-cudnn-cu12==8.9.2.26 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (8.9.2.26)\n", "Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (12.1.3.1)\n", "Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (11.0.2.54)\n", "Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (10.3.2.106)\n", "Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (11.4.5.107)\n", "Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (12.1.0.106)\n", "Requirement already satisfied: nvidia-nccl-cu12==2.20.5 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (2.20.5)\n", "Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (12.1.105)\n", "Requirement already satisfied: triton==2.3.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (2.3.0)\n", "Requirement already satisfied: nvidia-nvjitlink-cu12 in /usr/local/lib/python3.10/dist-packages (from nvidia-cusolver-cu12==11.4.5.107->torch>=1.7.0->detoxify) (12.5.40)\n", "Requirement already satisfied: huggingface-hub<1.0,>=0.23.0 in /usr/local/lib/python3.10/dist-packages (from transformers->detoxify) (0.23.2)\n", "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from transformers->detoxify) (1.25.2)\n", "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from transformers->detoxify) (23.2)\n", "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from transformers->detoxify) (6.0.1)\n", "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers->detoxify) (2024.5.15)\n", "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers->detoxify) (2.32.3)\n", "Requirement already satisfied: tokenizers<0.20,>=0.19 in /usr/local/lib/python3.10/dist-packages (from transformers->detoxify) (0.19.1)\n", "Requirement already satisfied: safetensors>=0.4.1 in /usr/local/lib/python3.10/dist-packages (from transformers->detoxify) (0.4.3)\n", "Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/dist-packages (from transformers->detoxify) (4.66.4)\n", "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch>=1.7.0->detoxify) (2.1.5)\n", "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->transformers->detoxify) (3.3.2)\n", "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->transformers->detoxify) (3.7)\n", "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->transformers->detoxify) (2.0.7)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->transformers->detoxify) (2024.2.2)\n", "Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=1.7.0->detoxify) (1.3.0)\n", "Requirement already satisfied: datasets in /usr/local/lib/python3.10/dist-packages (2.19.2)\n", "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from datasets) (3.14.0)\n", "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from datasets) (1.25.2)\n", "Requirement already satisfied: pyarrow>=12.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (14.0.2)\n", "Requirement already satisfied: pyarrow-hotfix in /usr/local/lib/python3.10/dist-packages (from datasets) (0.6)\n", "Requirement already satisfied: dill<0.3.9,>=0.3.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (0.3.8)\n", "Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from datasets) (2.0.3)\n", "Requirement already satisfied: requests>=2.32.1 in /usr/local/lib/python3.10/dist-packages (from datasets) (2.32.3)\n", "Requirement already satisfied: tqdm>=4.62.1 in /usr/local/lib/python3.10/dist-packages (from datasets) (4.66.4)\n", "Requirement already satisfied: xxhash in /usr/local/lib/python3.10/dist-packages (from datasets) (3.4.1)\n", "Requirement already satisfied: multiprocess in /usr/local/lib/python3.10/dist-packages (from datasets) (0.70.16)\n", "Requirement already satisfied: fsspec[http]<=2024.3.1,>=2023.1.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (2023.6.0)\n", "Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from datasets) (3.9.5)\n", "Requirement already satisfied: huggingface-hub>=0.21.2 in /usr/local/lib/python3.10/dist-packages (from datasets) (0.23.2)\n", "Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from datasets) (23.2)\n", "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from datasets) (6.0.1)\n", "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.3.1)\n", "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (23.2.0)\n", "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.4.1)\n", "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (6.0.5)\n", "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.9.4)\n", "Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (4.0.3)\n", "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.21.2->datasets) (4.12.0)\n", "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.32.1->datasets) (3.3.2)\n", "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.32.1->datasets) (3.7)\n", "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.32.1->datasets) (2.0.7)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.32.1->datasets) (2024.2.2)\n", "Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2.8.2)\n", "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2023.4)\n", "Requirement already satisfied: tzdata>=2022.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2024.1)\n", "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.2->pandas->datasets) (1.16.0)\n", "Requirement already satisfied: scikit-learn in /usr/local/lib/python3.10/dist-packages (1.2.2)\n", "Requirement already satisfied: numpy>=1.17.3 in /usr/local/lib/python3.10/dist-packages (from scikit-learn) (1.25.2)\n", "Requirement already satisfied: scipy>=1.3.2 in /usr/local/lib/python3.10/dist-packages (from scikit-learn) (1.11.4)\n", "Requirement already satisfied: joblib>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from scikit-learn) (1.4.2)\n", "Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from scikit-learn) (3.5.0)\n", "Requirement already satisfied: evaluate in /usr/local/lib/python3.10/dist-packages (0.4.2)\n", "Requirement already satisfied: datasets>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from evaluate) (2.19.2)\n", "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from evaluate) (1.25.2)\n", "Requirement already satisfied: dill in /usr/local/lib/python3.10/dist-packages (from evaluate) (0.3.8)\n", "Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from evaluate) (2.0.3)\n", "Requirement already satisfied: requests>=2.19.0 in /usr/local/lib/python3.10/dist-packages (from evaluate) (2.32.3)\n", "Requirement already satisfied: tqdm>=4.62.1 in /usr/local/lib/python3.10/dist-packages (from evaluate) (4.66.4)\n", "Requirement already satisfied: xxhash in /usr/local/lib/python3.10/dist-packages (from evaluate) (3.4.1)\n", "Requirement already satisfied: multiprocess in /usr/local/lib/python3.10/dist-packages (from evaluate) (0.70.16)\n", "Requirement already satisfied: fsspec[http]>=2021.05.0 in /usr/local/lib/python3.10/dist-packages (from evaluate) (2023.6.0)\n", "Requirement already satisfied: huggingface-hub>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from evaluate) (0.23.2)\n", "Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from evaluate) (23.2)\n", "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from datasets>=2.0.0->evaluate) (3.14.0)\n", "Requirement already satisfied: pyarrow>=12.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets>=2.0.0->evaluate) (14.0.2)\n", "Requirement already satisfied: pyarrow-hotfix in /usr/local/lib/python3.10/dist-packages (from datasets>=2.0.0->evaluate) (0.6)\n", "Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from datasets>=2.0.0->evaluate) (3.9.5)\n", "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from datasets>=2.0.0->evaluate) (6.0.1)\n", "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.7.0->evaluate) (4.12.0)\n", "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->evaluate) (3.3.2)\n", "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->evaluate) (3.7)\n", "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->evaluate) (2.0.7)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->evaluate) (2024.2.2)\n", "Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from pandas->evaluate) (2.8.2)\n", "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->evaluate) (2023.4)\n", "Requirement already satisfied: tzdata>=2022.1 in /usr/local/lib/python3.10/dist-packages (from pandas->evaluate) (2024.1)\n", "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets>=2.0.0->evaluate) (1.3.1)\n", "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets>=2.0.0->evaluate) (23.2.0)\n", "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets>=2.0.0->evaluate) (1.4.1)\n", "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets>=2.0.0->evaluate) (6.0.5)\n", "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets>=2.0.0->evaluate) (1.9.4)\n", "Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets>=2.0.0->evaluate) (4.0.3)\n", "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.2->pandas->evaluate) (1.16.0)\n" ] } ], "metadata": { "id": "KVxXyw8LWTVa", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "3f7ddefa-4324-4e19-91b7-0ba30d5460bb" } }, { "cell_type": "code", "execution_count": 4, "source": [ "import matplotlib.pyplot as plt\n", "import pandas as pd\n", "import numpy as np\n", "import torch\n", "import evaluate\n", "\n", "from transformers import GPT2LMHeadModel, GPT2Tokenizer\n", "from detoxify import Detoxify\n" ], "outputs": [], "metadata": { "id": "B-J9XfgGWTVb" } }, { "cell_type": "markdown", "source": [ "Importer le dataset JIGSAW disponible sur le gitlab.\n", "\n", "Le dataset Jigsaw est un ensemble de données créé par Jigsaw et l'équipe Counter-Abuse Technology de Google, qui contient une vaste collection de commentaires en ligne provenant de divers sites d'actualités. Ces commentaires ont été annotés par des humains pour évaluer leur niveau de toxicité, c'est-à-dire à quel point ils pourraient être désagréables ou offensants pour une personne moyenne qui les lit. Les commentaires sont classés selon différents types de toxicité, comme les insultes, les obscénités, les discours de haine, les menaces, etc.\n", "\n", "Tout au long de ce TP, nous allons l'utiliser pour évaluer les algorithmes de classification. Ce dataset est donc composé de 50 commentaires. Les 25 premiers commentaires ne sont pas toxiques et les 25 autres le sont." ], "metadata": { "id": "_ticzaL1WTVd" } }, { "cell_type": "code", "execution_count": 5, "source": [ "dataset = pd.read_csv('./JIGSAW.csv')" ], "outputs": [], "metadata": { "id": "I6ivHkaWs-gB" } }, { "cell_type": "markdown", "source": [ "Mainenant, nous allons essayer de prédire la toxicitité d'une phrase du dataset JIGSAW. Pour cela, nous allons utiliser la bibliothèque Detoxify.\n", "Detoxify est une bibliothèque Python qui fournie des modèles pré-entraînés pour la détection de la toxicité dans les textes. Ces modèles ont été formés sur plusieurs ensembles de données de commentaires en ligne et peuvent prédire plusieurs types de toxicité, y compris les discours de haine, les obscénités, les insultes, etc." ], "metadata": { "id": "516aAOqUuHj4" } }, { "cell_type": "code", "execution_count": 6, "source": [ "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n", "def check_toxicity(text):\n", " results = Detoxify('original',device=device).predict(text)\n", " return results\n", "\n", "toxicity_results = check_toxicity(dataset['comment_text'][25])\n", "print('Toxicity results:\\n', toxicity_results)" ], "outputs": [ { "output_type": "stream", "name": "stderr", "text": [ "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.\n", " warnings.warn(\n", "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_token.py:89: UserWarning: \n", "The secret `HF_TOKEN` does not exist in your Colab secrets.\n", "To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.\n", "You will be able to reuse this secret in all of your notebooks.\n", "Please note that authentication is recommended but still optional to access public models or datasets.\n", " warnings.warn(\n" ] }, { "output_type": "stream", "name": "stdout", "text": [ "Toxicity results:\n", " {'toxicity': 0.8165003, 'severe_toxicity': 0.0019058662, 'obscene': 0.06713744, 'threat': 0.003179473, 'insult': 0.2952709, 'identity_attack': 0.008143629}\n" ] } ], "metadata": { "id": "P_sYDfW4WTVd", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "91651a11-7a57-404e-c0cb-55c7c6bc017f" } }, { "cell_type": "markdown", "source": [ "Créez une fonction qui prend en entrée le dictionnaire issu de la fonction check_toxicity qui contient des labels et les scores de toxicité , et qui permet de visualiser ces scores sous forme de diagramme à barres horizontales.\n", "\n" ], "metadata": { "id": "qQHFbqjZwZ89" } }, { "cell_type": "code", "execution_count": 9, "source": [ "def visualize_toxicity(results):\n", " import matplotlib.pyplot as plt\n", " plt.clf()\n", " plt.barh(list(results.keys()),[results[key] for key in results])\n", " plt.show()\n", "visualize_toxicity(toxicity_results)" ], "outputs": [ { "output_type": "display_data", "data": { "text/plain": [ "
" ], "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnYAAAGdCAYAAACfPz4lAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAz7ElEQVR4nO3deXQV9f3/8dfNdpOQ3LAYIEAkCIRFCIsoTZBFoQ2CFFsXUIqyBKSIFC0uFJVVQAQFrbSKmFiLUgVqFdk0GiiLgEiQJYCERFCCEYUsWBNIPr8/PNyf+RIgNyv55Pk4Z87JnfnMzPt9r3BffmYmOIwxRgAAAKj2vKq6AAAAAJQPgh0AAIAlCHYAAACWINgBAABYgmAHAABgCYIdAACAJQh2AAAAliDYAQAAWMKnqgtA5SksLNTx48cVHBwsh8NR1eUAAIASMMYoJydHjRo1kpfXpefkCHY1yPHjxxUeHl7VZQAAgFI4duyYmjRpcskxBLsaJDg4WNLP/2G4XK4qrgYAAJREdna2wsPD3d/jl0Kwq0HOX351uVwEOwAAqpmS3EbFwxMAAACWINgBAABYgmAHAABgCYIdAACAJQh2AAAAliDYAQAAWIJgBwAAYAmCHQAAgCUIdgAAAJYg2AEAAFiCYAcAAGAJgh0AAIAlCHYAAACW8KnqAlD52k1ZJy9noPt1+pz+VVgNAAAoL8zYAQAAWIJgBwAAYAmCHQAAgCUIdgAAAJYg2AEAAFiCYAcAAGAJgh0AAIAlCHYAAACWINgBAABYgmAHAABgCYIdAACAJQh2AAAAliDYAQAAWIJgBwAAYAmCHQAAgCUIdgAAAJYg2AEAAFiCYAcAAGAJj4Ndr169NGHChItuj4iI0IIFC8pQUskkJSXJ4XDo9OnTFX6uK8Xl3nsAAFCzeRzsVq5cqRkzZlRELRdVXKCJiYlRRkaGQkJCJEkJCQmqXbt2pdYlSQ6HQ++++26RdVOnTlXHjh0rvRYAAFCz+Xi6Q926dSuiDo/5+fmpYcOGVV0GAADAFaNMl2IzMzM1YMAABQQEqFmzZlq6dOkF40+fPq24uDiFhobK5XLp5ptv1u7du93bz89uvfHGG4qIiFBISIgGDx6snJwcSdKwYcO0YcMGLVy4UA6HQw6HQ+np6UUuxSYlJWn48OHKyspyj5k6daqmT5+udu3aXVBTx44d9eSTT1621x07dujXv/61rrrqKoWEhKhnz576/PPP3dsjIiIkSb/73e/kcDgUERGhhIQETZs2Tbt373bXkpCQIEl67rnn1L59e9WqVUvh4eEaO3ascnNzi5xz8+bN6tWrlwIDA1WnTh3Fxsbq1KlTxdb3wQcfKCQkpNj3HQAA1Dxlenhi2LBhOnbsmD755BMtX75cixYtUmZmZpExd955pzIzM7VmzRrt3LlTnTt3Vu/evfXDDz+4x6Smpurdd9/VqlWrtGrVKm3YsEFz5syRJC1cuFDR0dEaNWqUMjIylJGRofDw8CLniImJ0YIFC+RyudxjJk6cqBEjRiglJUU7duxwj921a5e++OILDR8+/LL95eTk6L777tOmTZv06aefqmXLlurXr587dJ4/bnx8vDIyMrRjxw4NGjRIf/7zn3Xttde6axk0aJAkycvLSy+88IL27dun119/XR9//LEeffRR9/mSk5PVu3dvtW3bVlu3btWmTZs0YMAAFRQUXFDbm2++qbvvvltLly7VkCFDiq0/Ly9P2dnZRRYAAGAvjy/Fnnfo0CGtWbNG27dv1/XXXy9JWrJkidq0aeMes2nTJm3fvl2ZmZlyOp2SpHnz5undd9/V8uXLNXr0aElSYWGhEhISFBwcLEkaOnSoEhMT9fTTTyskJER+fn4KDAy86KVXPz8/hYSEyOFwFBkTFBSk2NhYxcfHu2uMj49Xz549dc0111y2x5tvvrnI61deeUW1a9fWhg0bdOuttyo0NFSSVLt27QvO6+Pjc0G9v7xPMCIiQjNnztSYMWO0aNEiSdLcuXPVpUsX92tJuvbaay+o66WXXtLkyZP1/vvvq2fPnhetf/bs2Zo2bdpl+wQAAHYo9YxdSkqKfHx8dN1117nXtW7dusgDDLt371Zubq7q1aunoKAg95KWlqbU1FT3uIiICHeok6SwsLALZv5Ka9SoUXrrrbf0008/KT8/X2+++aZGjBhRon2//fZbjRo1Si1btlRISIhcLpdyc3N19OjRUtXy0UcfqXfv3mrcuLGCg4M1dOhQff/99/rxxx8l/f8Zu0tZvny5HnroIX344YeXDHWSNGnSJGVlZbmXY8eOlapuAABQPZR6xq4kcnNzFRYWpqSkpAu2/TIA+vr6FtnmcDhUWFhYLjUMGDBATqdT//73v+Xn56ezZ8/qjjvuKNG+9913n77//nstXLhQTZs2ldPpVHR0tPLz8z2uIz09Xbfeeqv++Mc/6umnn1bdunW1adMmjRw5Uvn5+QoMDFRAQMBlj9OpUyd9/vnneu2119SlSxc5HI6LjnU6ne6ZUgAAYL9SB7vWrVvr3Llz2rlzp/sy58GDB4v8XrnOnTvrxIkT8vHxcT9oUBp+fn7F3mdWkjE+Pj667777FB8fLz8/Pw0ePLhEAUr6+UGGRYsWqV+/fpKkY8eO6eTJk0XG+Pr6XnDe4mrZuXOnCgsLNX/+fHl5/TxR+vbbbxcZExUVpcTExEtePm3evLnmz5+vXr16ydvbW3/9619L1AsAALBfqS/FtmrVSn379tX999+vbdu2aefOnYqLiysSmvr06aPo6GjddtttWr9+vdLT07VlyxZNnjxZn332WYnPFRERoW3btik9PV0nT54sdjYvIiJCubm5SkxM1MmTJ92XNyUpLi5OH3/8sdauXVviy7CS1LJlS73xxhtKSUnRtm3bNGTIkAtCYUREhBITE3XixAn306sRERFKS0tTcnKyTp48qby8PLVo0UJnz57Viy++qCNHjuiNN97Q3//+9yLHmjRpknbs2KGxY8fqiy++0IEDB/S3v/3tgjAZGRmpTz75RCtWrOAXFgMAALcyPRUbHx+vRo0aqWfPnvr973+v0aNHq379+u7tDodDq1evVo8ePTR8+HBFRkZq8ODB+uqrr9SgQYMSn2fixIny9vZW27ZtFRoaWuw9bjExMRozZowGDRqk0NBQzZ07172tZcuWiomJUevWrdW1a9cSn3fJkiU6deqUOnfurKFDh2r8+PFF+pOk+fPn68MPP1R4eLg6deokSbr99tvVt29f3XTTTQoNDdVbb72lDh066LnnntMzzzyjdu3aaenSpZo9e3aRY0VGRmr9+vXavXu3brjhBkVHR+s///mPfHwunFht1aqVPv74Y7311lv685//XOKeAACAvRzGGFPVRVQ0Y4xatmypsWPH6uGHH67qcqpMdna2QkJCFD7hbXk5A93r0+f0r8KqAADApZz//s7KypLL5brk2Ap9eOJK8N1332nZsmU6ceJEiX53HQAAQHVlfbCrX7++rrrqKr3yyiuqU6dOkW1BQUEX3W/NmjXq3r17RZcHAABQbqwPdpe60pycnHzRbY0bN66AagAAACqO9cHuUlq0aFHVJQAAAJSbMj0VCwAAgCsHwQ4AAMASBDsAAABLEOwAAAAsQbADAACwBMEOAADAEgQ7AAAASxDsAAAALEGwAwAAsATBDgAAwBIEOwAAAEsQ7AAAACzhU9UFoPLtnRYrl8tV1WUAAIByxowdAACAJQh2AAAAliDYAQAAWIJgBwAAYAmCHQAAgCUIdgAAAJYg2AEAAFiCYAcAAGAJgh0AAIAlCHYAAACW4J8Uq4HaTVknL2dgVZcBXFL6nP5VXQIAVDvM2AEAAFiCYAcAAGAJgh0AAIAlCHYAAACWINgBAABYgmAHAABgCYIdAACAJQh2AAAAliDYAQAAWIJgBwAAYAmCHQAAgCUIdgAAAJYg2AEAAFiCYAcAAGAJgh0AAIAlCHYAAACWINgBAABYgmBXSr169dKECRMq5VxJSUlyOBw6ffp0pZwPAABUTz5VXUB1tXLlSvn6+lbJuRMSEjRhwgSCHgAAKIJgV0p169at6hIAAACK4FJsKf3yUmxERIRmzZqlESNGKDg4WFdffbVeeeUV99j8/HyNGzdOYWFh8vf3V9OmTTV79mxJUnp6uhwOh5KTk93jT58+LYfDoaSkpAvOm5SUpOHDhysrK0sOh0MOh0NTp06twE4BAEB1QbArJ/Pnz1eXLl20a9cujR07Vn/84x918OBBSdILL7yg9957T2+//bYOHjyopUuXKiIiolTniYmJ0YIFC+RyuZSRkaGMjAxNnDixHDsBAADVFZdiy0m/fv00duxYSdJjjz2m559/Xp988olatWqlo0ePqmXLlrrxxhvlcDjUtGnTUp/Hz89PISEhcjgcatiw4SXH5uXlKS8vz/06Ozu71OcFAABXPmbsyklUVJT75/OhKzMzU5I0bNgwJScnq1WrVho/frzWr19fKTXNnj1bISEh7iU8PLxSzgsAAKoGwa6c/N8nZB0OhwoLCyVJnTt3VlpammbMmKH//e9/uuuuu3THHXdIkry8fv4IjDHufc+ePVsuNU2aNElZWVnu5dixY+VyXAAAcGXiUmwlcblcGjRokAYNGqQ77rhDffv21Q8//KDQ0FBJUkZGhjp16iRJRR6kKI6fn58KCgoue06n0ymn01nm2gEAQPVAsKsEzz33nMLCwtSpUyd5eXnpnXfeUcOGDVW7dm15eXnpV7/6lebMmaNmzZopMzNTTzzxxCWPFxERodzcXCUmJqpDhw4KDAxUYGBgJXUDAACuVFyKrQTBwcGaO3euunTpouuvv17p6elavXq1+zLsa6+9pnPnzum6667ThAkTNHPmzEseLyYmRmPGjNGgQYMUGhqquXPnVkYbAADgCucwv7y5C1bLzs7++SGKCW/Ly8kMH65s6XP6V3UJAHBFOP/9nZWVJZfLdcmxzNgBAABYgmAHAABgCYIdAACAJQh2AAAAliDYAQAAWIJgBwAAYAmCHQAAgCUIdgAAAJYg2AEAAFiCYAcAAGAJgh0AAIAlCHYAAACWINgBAABYgmAHAABgCYIdAACAJQh2AAAAliDYAQAAWMKnqgtA5ds7LVYul6uqywAAAOWMGTsAAABLEOwAAAAsQbADAACwBMEOAADAEgQ7AAAASxDsAAAALEGwAwAAsATBDgAAwBIEOwAAAEsQ7AAAACxBsKuB2k1ZV9UlAACACkCwAwAAsATBDgAAwBIEOwAAAEsQ7AAAACxBsAMAALAEwQ4AAMASBDsAAABLEOwAAAAsQbADAACwBMEOAADAEgQ7AAAASxDsAAAALEGwAwAAsATBDgAAwBIEOwAAAEsQ7AAAACxBsAMAALAEwa4UkpKS5HA4dPr06aouBQAAwI1gVwK9evXShAkTqroMt4iICC1YsKCqywAAAFcYgl0lyc/Pr+oSAACA5Qh2lzFs2DBt2LBBCxculMPhkMPhUHp6uiRp586d6tKliwIDAxUTE6ODBw+695s6dao6duyoV199Vc2aNZO/v78k6fTp04qLi1NoaKhcLpduvvlm7d69271famqqBg4cqAYNGigoKEjXX3+9PvroI/f2Xr166auvvtJDDz3krgcAAEAi2F3WwoULFR0drVGjRikjI0MZGRkKDw+XJE2ePFnz58/XZ599Jh8fH40YMaLIvocPH9aKFSu0cuVKJScnS5LuvPNOZWZmas2aNdq5c6c6d+6s3r1764cffpAk5ebmql+/fkpMTNSuXbvUt29fDRgwQEePHpUkrVy5Uk2aNNH06dPd9QAAAEiST1UXcKULCQmRn5+fAgMD1bBhQ0nSgQMHJElPP/20evbsKUl6/PHH1b9/f/3000/u2bn8/Hz94x//UGhoqCRp06ZN2r59uzIzM+V0OiVJ8+bN07vvvqvly5dr9OjR6tChgzp06OA+/4wZM/Tvf/9b7733nsaNG6e6devK29tbwcHB7nouJi8vT3l5ee7X2dnZ5fSuAACAKxEzdmUQFRXl/jksLEySlJmZ6V7XtGlTd6iTpN27dys3N1f16tVTUFCQe0lLS1Nqaqqkn2fsJk6cqDZt2qh27doKCgpSSkqKe8bOE7Nnz1ZISIh7OT/TCAAA7MSMXRn4+vq6fz5/r1thYaF7Xa1atYqMz83NVVhYmJKSki44Vu3atSVJEydO1Icffqh58+apRYsWCggI0B133FGqhy8mTZqkhx9+2P06OzubcAcAgMUIdiXg5+engoKCMh+nc+fOOnHihHx8fBQREVHsmM2bN2vYsGH63e9+J+nnMHj+YQ1P63E6ne5LvgAAwH5cii2BiIgIbdu2Tenp6Tp58mSRWTlP9OnTR9HR0brtttu0fv16paena8uWLZo8ebI+++wzSVLLli3dD1vs3r1b99xzzwXni4iI0MaNG/XNN9/o5MmTZe4PAADYgWBXAhMnTpS3t7fatm2r0NDQUt3vJv18uXb16tXq0aOHhg8frsjISA0ePFhfffWVGjRoIEl67rnnVKdOHcXExGjAgAGKjY1V586dixxn+vTpSk9PV/PmzYvcwwcAAGo2hzHGVHURqBzZ2dk/P0Qx4W0dff7Oqi4HAACUwPnv76ysLLlcrkuOZcYOAADAEgQ7AAAASxDsAAAALEGwAwAAsATBDgAAwBIEOwAAAEsQ7AAAACxBsAMAALAEwQ4AAMASBDsAAABLEOwAAAAsQbADAACwBMEOAADAEgQ7AAAASxDsAAAALEGwAwAAsATBDgAAwBIEuxpo77TYqi4BAABUAIIdAACAJQh2AAAAliDYAQAAWIJgBwAAYAmCHQAAgCUIdgAAAJYg2AEAAFiCYAcAAGAJgh0AAIAlCHYAAACW8KnqAlD52k1ZJy9nYFWXcYH0Of2rugQAAKo1ZuwAAAAsQbADAACwBMEOAADAEgQ7AAAASxDsAAAALEGwAwAAsATBDgAAwBIEOwAAAEsQ7AAAACxBsAMAALAEwQ4AAMASBDsAAABLEOwAAAAsQbADAACwBMEOAADAEgQ7AAAASxDsAAAALEGwk5SUlCSHw6HTp09XdSkAAAClRrADAACwBMEOAADAEjUm2OXl5Wn8+PGqX7++/P39deONN2rHjh1FxmzevFlRUVHy9/fXr371K+3du9e97auvvtKAAQNUp04d1apVS9dee61Wr17t3r5v3z7deuutcrlcCg4OVvfu3ZWamure/uqrr6pNmzby9/dX69attWjRIve29PR0ORwOrVy5UjfddJMCAwPVoUMHbd26tUh9mzZtUvfu3RUQEKDw8HCNHz9eZ86cKe+3CgAAVFM1Jtg9+uijWrFihV5//XV9/vnnatGihWJjY/XDDz+4xzzyyCOaP3++duzYodDQUA0YMEBnz56VJD3wwAPKy8vTxo0btWfPHj3zzDMKCgqSJH3zzTfq0aOHnE6nPv74Y+3cuVMjRozQuXPnJElLly7VU089paefflopKSmaNWuWnnzySb3++utFapw8ebImTpyo5ORkRUZG6u6773YfIzU1VX379tXtt9+uL774Qv/617+0adMmjRs37qI95+XlKTs7u8gCAADs5TDGmKouoqKdOXNGderUUUJCgu655x5J0tmzZxUREaEJEybo+uuv10033aRly5Zp0KBBkqQffvhBTZo0UUJCgu666y5FRUXp9ttv15QpUy44/l/+8hctW7ZMBw8elK+v7wXbW7RooRkzZujuu+92r5s5c6ZWr16tLVu2KD09Xc2aNdOrr76qkSNHSpL279+va6+9VikpKWrdurXi4uLk7e2tl19+2X2MTZs2qWfPnjpz5oz8/f0vOO/UqVM1bdq0C9aHT3hbXs5AD9/Fipc+p39VlwAAwBUnOztbISEhysrKksvluuTYGjFjl5qaqrNnz6pbt27udb6+vrrhhhuUkpLiXhcdHe3+uW7dumrVqpV7+/jx4zVz5kx169ZNU6ZM0RdffOEem5ycrO7duxcb6s6cOaPU1FSNHDlSQUFB7mXmzJlFLtVKUlRUlPvnsLAwSVJmZqYkaffu3UpISChyjNjYWBUWFiotLa3YvidNmqSsrCz3cuzYsRK/ZwAAoPrxqeoCqou4uDjFxsbqgw8+0Pr16zV79mzNnz9fDz74oAICAi66X25uriRp8eLF6tq1a5Ft3t7eRV7/Mhg6HA5JUmFhofs4999/v8aPH3/BOa6++upiz+10OuV0OkvQHQAAsEGNmLFr3ry5/Pz8tHnzZve6s2fPaseOHWrbtq173aeffur++dSpUzp06JDatGnjXhceHq4xY8Zo5cqV+vOf/6zFixdL+nmm7b///a/7frxfatCggRo1aqQjR46oRYsWRZZmzZqVuIfOnTtr//79FxyjRYsW8vPz8+j9AAAAdqoRwa5WrVr64x//qEceeURr167V/v37NWrUKP3444/ue9okafr06UpMTNTevXs1bNgwXXXVVbrtttskSRMmTNC6deuUlpamzz//XJ988ok79I0bN07Z2dkaPHiwPvvsM3355Zd64403dPDgQUnStGnTNHv2bL3wwgs6dOiQ9uzZo/j4eD333HMl7uGxxx7Tli1bNG7cOCUnJ+vLL7/Uf/7zn0s+PAEAAGqWGnMpds6cOSosLNTQoUOVk5OjLl26aN26dapTp06RMX/605/05ZdfqmPHjnr//ffds2EFBQV64IEH9PXXX8vlcqlv3756/vnnJUn16tXTxx9/rEceeUQ9e/aUt7e3Onbs6L6nLy4uToGBgXr22Wf1yCOPqFatWmrfvr0mTJhQ4vqjoqK0YcMGTZ48Wd27d5cxRs2bN3c/7AEAAFAjnorFz84/VcNTsQAAVB88FQsAAFADEewAAAAsQbADAACwBMEOAADAEgQ7AAAASxDsAAAALEGwAwAAsATBDgAAwBIEOwAAAEsQ7AAAACxBsAMAALAEwQ4AAMASBDsAAABLEOwAAAAsQbADAACwBMEOAADAEgQ7AAAAS/hUdQGofHunxcrlclV1GQAAoJwxYwcAAGAJgh0AAIAlCHYAAACWINgBAABYgmAHAABgCYIdAACAJQh2AAAAliDYAQAAWIJgBwAAYAmCHQAAgCUIdgAAAJYg2AEAAFiCYAcAAGAJgh0AAIAlCHYAAACWINgBAABYgmAHAABgCYIdAACAJQh2AAAAliDYAQAAWIJgBwAAYAmCHQAAgCUIdgAAAJYg2AEAAFiCYAcAAGAJgh0AAIAlCHYAAACWINgBAABYgmBXQaZOnaqOHTuWeLzD4dC7775bYfUAAAD71fhgl5CQoNq1a5f7cSdOnKjExMQSj8/IyNAtt9wiSUpPT5fD4VBycnK51wUAAOzlU9UFlEVBQYEcDoe8vK68fBoUFKSgoKASj2/YsGEFVgMAAGoCjxPR8uXL1b59ewUEBKhevXrq06ePzpw5I0l69dVX1aZNG/n7+6t169ZatGiRe7+YmBg99thjRY713XffydfXVxs3bpQk5eXlaeLEiWrcuLFq1aqlrl27KikpyT3+/Ozae++9p7Zt28rpdOro0aOX3e9ikpKSNHz4cGVlZcnhcMjhcGjq1KmSpFOnTunee+9VnTp1FBgYqFtuuUVffvmlu+6GDRtq1qxZ7mNt2bJFfn5+7lm64i7Fvvbaa7r22mvldDoVFhamcePGubf98lJss2bNJEmdOnWSw+FQr169tHHjRvn6+urEiRNFjjlhwgR17979sr0CAIAawHjg+PHjxsfHxzz33HMmLS3NfPHFF+all14yOTk55p///KcJCwszK1asMEeOHDErVqwwdevWNQkJCcYYY/7617+aq6++2hQWFrqP9+KLLxZZFxcXZ2JiYszGjRvN4cOHzbPPPmucTqc5dOiQMcaY+Ph44+vra2JiYszmzZvNgQMHzJkzZy6738Xk5eWZBQsWGJfLZTIyMkxGRobJyckxxhjz29/+1rRp08Zs3LjRJCcnm9jYWNOiRQuTn59vjDHmgw8+ML6+vmbHjh0mOzvbXHPNNeahhx5yH3vKlCmmQ4cO7teLFi0y/v7+ZsGCBebgwYNm+/bt5vnnn3dvl2T+/e9/G2OM2b59u5FkPvroI5ORkWG+//57Y4wxkZGRZu7cue598vPzzVVXXWVee+21Yvv76aefTFZWlns5duyYkWSysrIu+b4AAIArR1ZWVom/vz0Kdjt37jSSTHp6+gXbmjdvbt58880i62bMmGGio6ONMcZkZmYaHx8fs3HjRvf26Oho89hjjxljjPnqq6+Mt7e3+eabb4oco3fv3mbSpEnGmJ+DnSSTnJzs3l6S/S4lPj7ehISEFFl36NAhI8ls3rzZve7kyZMmICDAvP322+51Y8eONZGRkeaee+4x7du3Nz/99JN72/8Ndo0aNTKTJ0++aB2/DHZpaWlGktm1a1eRMc8884xp06aN+/WKFStMUFCQyc3NLfaYU6ZMMZIuWAh2AABUH54EO48uxXbo0EG9e/dW+/btdeedd2rx4sU6deqUzpw5o9TUVI0cOdJ9b1lQUJBmzpyp1NRUSVJoaKh+85vfaOnSpZKktLQ0bd26VUOGDJEk7dmzRwUFBYqMjCxyjA0bNriPIUl+fn6Kiopyvy7pfp5ISUmRj4+Punbt6l5Xr149tWrVSikpKe518+bN07lz5/TOO+9o6dKlcjqdxR4vMzNTx48fV+/evUtVz3nDhg3T4cOH9emnn0r6+dL0XXfdpVq1ahU7ftKkScrKynIvx44dK9P5AQDAlc2jhye8vb314YcfasuWLVq/fr1efPFFTZ48We+//74kafHixUXC0Pl9zhsyZIjGjx+vF198UW+++abat2+v9u3bS5Jyc3Pl7e2tnTt3FtlHUpGHEAICAuRwONyvS7pfRUhNTdXx48dVWFio9PR0dy//V0BAQLmcr379+howYIDi4+PVrFkzrVmz5pL3EjqdzouGTQAAYB+Pn4p1OBzq1q2bunXrpqeeekpNmzbV5s2b1ahRIx05csQ9A1ecgQMHavTo0Vq7dq3efPNN3Xvvve5tnTp1UkFBgTIzMz16GKC0+53n5+engoKCIuvatGmjc+fOadu2bYqJiZEkff/99zp48KDatm0rScrPz9cf/vAHDRo0SK1atVJcXJz27Nmj+vXrX3CO4OBgRUREKDExUTfddFOJapJ0QV2SFBcXp7vvvltNmjRR8+bN1a1bN497BgAAdvIo2G3btk2JiYn6zW9+o/r162vbtm367rvv1KZNG02bNk3jx49XSEiI+vbtq7y8PH322Wc6deqUHn74YUlSrVq1dNttt+nJJ59USkqK7r77bvexIyMjNWTIEN17772aP3++OnXqpO+++06JiYmKiopS//79i62ptPudFxERodzcXCUmJqpDhw4KDAxUy5YtNXDgQI0aNUovv/yygoOD9fjjj6tx48YaOHCgJGny5MnKysrSCy+8oKCgIK1evVojRozQqlWrij3P1KlTNWbMGNWvX1+33HKLcnJytHnzZj344IMXjK1fv74CAgK0du1aNWnSRP7+/goJCZEkxcbGyuVyaebMmZo+ffrlPzQAAFBzeHLz3v79+01sbKwJDQ01TqfTREZGmhdffNG9fenSpaZjx47Gz8/P1KlTx/To0cOsXLmyyDFWr15tJJkePXpccPz8/Hzz1FNPmYiICOPr62vCwsLM7373O/PFF18YY4p/0KEk+13OmDFjTL169YwkM2XKFGOMMT/88IMZOnSoCQkJMQEBASY2Ntb9lO0nn3xifHx8zH//+1/3MdLS0ozL5TKLFi0yxlz48IQxxvz97383rVq1ctf44IMPurfpFw9PGGPM4sWLTXh4uPHy8jI9e/Yscpwnn3zSeHt7m+PHj5eov/M8ufkSAABcGTz5/nYYY0yVJkt4bOTIkfruu+/03nvvebRfdna2QkJClJWVJZfLVUHVAQCA8uTJ93e1/pcnapqsrCzt2bNHb775psehDgAA2O/K+7e4ytktt9xS5Neg/HL55b8cUR0MHDhQv/nNbzRmzBj9+te/rupyAADAFcb6S7HffPON/ve//xW7rW7duqpbt24lV1R1uBQLAED1w6XYX2jcuHFVlwAAAFAprL8UCwAAUFMQ7AAAACxBsAMAALAEwQ4AAMASBDsAAABLEOwAAAAsQbADAACwBMEOAADAEgQ7AAAASxDsAAAALEGwAwAAsATBDgAAwBIEOwAAAEsQ7AAAACxBsAMAALAEwQ4AAMASBDsAAABLEOwAAAAsQbADAACwBMEOAADAEgQ7AAAASxDsAAAALEGwAwAAsATBDgAAwBI+VV0AKl+7Kevk5Qys6jIAALBG+pz+VV2CJGbsAAAArEGwAwAAsATBDgAAwBIEOwAAAEsQ7AAAACxBsAMAALAEwQ4AAMASBDsAAABLEOwAAAAsQbADAACwBMEOAADAEgQ7AAAASxDsAAAALEGwAwAAsATBDgAAwBIEOwAAAEsQ7AAAACxBsKsgERERWrBgQYnGJiQkqHbt2hVaDwAAsB/BTlKvXr00YcKEcj3mjh07NHr06BKNHTRokA4dOuR+PXXqVHXs2LFc6wEAAPbzqeoCbBUaGlrisQEBAQoICKjAagAAQE1Q42fshg0bpg0bNmjhwoVyOBxyOBxKT0/Xhg0bdMMNN8jpdCosLEyPP/64zp07J0n6xz/+oaCgIH355Zfu44wdO1atW7fWjz/+KOnCS7GnT5/W/fffrwYNGsjf31/t2rXTqlWrJBW9FJuQkKBp06Zp9+7d7noSEhI0YsQI3XrrrUVqP3v2rOrXr68lS5ZU4DsEAACqixo/Y7dw4UIdOnRI7dq10/Tp0yVJBQUF6tevn4YNG6Z//OMfOnDggEaNGiV/f39NnTpV9957r1atWqUhQ4Zoy5YtWrdunV599VVt3bpVgYGBF5yjsLBQt9xyi3JycvTPf/5TzZs31/79++Xt7X3B2EGDBmnv3r1au3atPvroI0lSSEiIIiMj1aNHD2VkZCgsLEyStGrVKv34448aNGhQsb3l5eUpLy/P/To7O7vM7xcAALhy1fhgFxISIj8/PwUGBqphw4aSpMmTJys8PFx//etf5XA41Lp1ax0/flyPPfaYnnrqKXl5eenll19WVFSUxo8fr5UrV2rq1Km67rrrij3HRx99pO3btyslJUWRkZGSpGuuuabYsQEBAQoKCpKPj4+7HkmKiYlRq1at9MYbb+jRRx+VJMXHx+vOO+9UUFBQsceaPXu2pk2bVur3BgAAVC81/lJscVJSUhQdHS2Hw+Fe161bN+Xm5urrr7+WJNWpU0dLlizR3/72NzVv3lyPP/74RY+XnJysJk2auENdacXFxSk+Pl6S9O2332rNmjUaMWLERcdPmjRJWVlZ7uXYsWNlOj8AALiyEezKYOPGjfL29lZGRobOnDlz0XHl9WDEvffeqyNHjmjr1q365z//qWbNmql79+4XHe90OuVyuYosAADAXgQ7SX5+fiooKHC/btOmjbZu3SpjjHvd5s2bFRwcrCZNmkiStmzZomeeeUbvv/++goKCNG7cuIsePyoqSl9//XWRX2niST3n1atXT7fddpvi4+OVkJCg4cOHl7RFAABQAxDs9PMTrNu2bVN6erpOnjypsWPH6tixY3rwwQd14MAB/ec//9GUKVP08MMPy8vLSzk5ORo6dKjGjx+vW265RUuXLtW//vUvLV++vNjj9+zZUz169NDtt9+uDz/8UGlpaVqzZo3Wrl170XrS0tKUnJyskydPFnkAIi4uTq+//rpSUlJ03333Vcj7AQAAqieCnaSJEyfK29tbbdu2VWhoqM6ePavVq1dr+/bt6tChg8aMGaORI0fqiSeekCT96U9/Uq1atTRr1ixJUvv27TVr1izdf//9+uabb4o9x4oVK3T99dfr7rvvVtu2bfXoo48WOysnSbfffrv69u2rm266SaGhoXrrrbfc2/r06aOwsDDFxsaqUaNG5fxOAACA6sxhfnm9EVe83NxcNW7cWPHx8fr973/v0b7Z2dkKCQlR+IS35eW88NeyAACA0kmf07/Cjn3++zsrK+uy98vX+F93Ul0UFhbq5MmTmj9/vmrXrq3f/va3VV0SAAC4whDsqomjR4+qWbNmatKkiRISEuTjw0cHAACKIh1UExEREeKqOQAAuBQengAAALAEwQ4AAMASBDsAAABLEOwAAAAsQbADAACwBMEOAADAEgQ7AAAASxDsAAAALEGwAwAAsATBDgAAwBIEOwAAAEsQ7AAAACxBsAMAALCET1UXgMq3d1qsXC5XVZcBAADKGTN2AAAAliDYAQAAWIJgBwAAYAmCHQAAgCUIdgAAAJYg2AEAAFiCYAcAAGAJgh0AAIAlCHYAAACWINgBAABYgmAHAABgCYIdAACAJQh2AAAAliDYAQAAWIJgBwAAYAmfqi4AlccYI0nKzs6u4koAAEBJnf/ePv89fikEuxrk+++/lySFh4dXcSUAAMBTOTk5CgkJueQYgl0NUrduXUnS0aNHL/sfRnWWnZ2t8PBwHTt2TC6Xq6rLqTD0aZea0GdN6FGiT9tcCX0aY5STk6NGjRpddizBrgbx8vr5lsqQkBCr/xCe53K56NMi9GmPmtCjRJ+2qeo+Szohw8MTAAAAliDYAQAAWIJgV4M4nU5NmTJFTqezqkupUPRpF/q0R03oUaJP21S3Ph2mJM/OAgAA4IrHjB0AAIAlCHYAAACWINgBAABYgmAHAABgCYKdZV566SVFRETI399fXbt21fbt2y85/p133lHr1q3l7++v9u3ba/Xq1ZVUadl40ue+fft0++23KyIiQg6HQwsWLKi8QsvIkz4XL16s7t27q06dOqpTp4769Olz2c//SuFJnytXrlSXLl1Uu3Zt1apVSx07dtQbb7xRidWWjqd/Ns9btmyZHA6HbrvttootsJx40mdCQoIcDkeRxd/fvxKrLT1PP8/Tp0/rgQceUFhYmJxOpyIjI6vF37ee9NmrV68LPk+Hw6H+/ftXYsWl4+nnuWDBArVq1UoBAQEKDw/XQw89pJ9++qmSqr0MA2ssW7bM+Pn5mddee83s27fPjBo1ytSuXdt8++23xY7fvHmz8fb2NnPnzjX79+83TzzxhPH19TV79uyp5Mo942mf27dvNxMnTjRvvfWWadiwoXn++ecrt+BS8rTPe+65x7z00ktm165dJiUlxQwbNsyEhISYr7/+upIr94ynfX7yySdm5cqVZv/+/ebw4cNmwYIFxtvb26xdu7aSKy85T3s8Ly0tzTRu3Nh0797dDBw4sHKKLQNP+4yPjzcul8tkZGS4lxMnTlRy1Z7ztM+8vDzTpUsX069fP7Np0yaTlpZmkpKSTHJyciVX7hlP+/z++++LfJZ79+413t7eJj4+vnIL95CnfS5dutQ4nU6zdOlSk5aWZtatW2fCwsLMQw89VMmVF49gZ5EbbrjBPPDAA+7XBQUFplGjRmb27NnFjr/rrrtM//79i6zr2rWruf/++yu0zrLytM9fatq0abUJdmXp0xhjzp07Z4KDg83rr79eUSWWi7L2aYwxnTp1Mk888URFlFcuStPjuXPnTExMjHn11VfNfffdVy2Cnad9xsfHm5CQkEqqrvx42uff/vY3c80115j8/PzKKrFclPXP5vPPP2+Cg4NNbm5uRZVYLjzt84EHHjA333xzkXUPP/yw6datW4XWWVJcirVEfn6+du7cqT59+rjXeXl5qU+fPtq6dWux+2zdurXIeEmKjY296PgrQWn6rI7Ko88ff/xRZ8+eVd26dSuqzDIra5/GGCUmJurgwYPq0aNHRZZaaqXtcfr06apfv75GjhxZGWWWWWn7zM3NVdOmTRUeHq6BAwdq3759lVFuqZWmz/fee0/R0dF64IEH1KBBA7Vr106zZs1SQUFBZZXtsfL4O2jJkiUaPHiwatWqVVFllllp+oyJidHOnTvdl2uPHDmi1atXq1+/fpVS8+X4VHUBKB8nT55UQUGBGjRoUGR9gwYNdODAgWL3OXHiRLHjT5w4UWF1llVp+qyOyqPPxx57TI0aNbogvF9JSttnVlaWGjdurLy8PHl7e2vRokX69a9/XdHllkppety0aZOWLFmi5OTkSqiwfJSmz1atWum1115TVFSUsrKyNG/ePMXExGjfvn1q0qRJZZTtsdL0eeTIEX388ccaMmSIVq9ercOHD2vs2LE6e/aspkyZUhlle6ysfwdt375de/fu1ZIlSyqqxHJRmj7vuecenTx5UjfeeKOMMTp37pzGjBmjv/zlL5VR8mUR7AALzZkzR8uWLVNSUlK1uRndE8HBwUpOTlZubq4SExP18MMP65prrlGvXr2qurQyy8nJ0dChQ7V48WJdddVVVV1OhYqOjlZ0dLT7dUxMjNq0aaOXX35ZM2bMqMLKyldhYaHq16+vV155Rd7e3rruuuv0zTff6Nlnn71ig11ZLVmyRO3bt9cNN9xQ1aWUu6SkJM2aNUuLFi1S165ddfjwYf3pT3/SjBkz9OSTT1Z1eQQ7W1x11VXy9vbWt99+W2T9t99+q4YNGxa7T8OGDT0afyUoTZ/VUVn6nDdvnubMmaOPPvpIUVFRFVlmmZW2Ty8vL7Vo0UKS1LFjR6WkpGj27NlXZLDztMfU1FSlp6drwIAB7nWFhYWSJB8fHx08eFDNmzev2KJLoTz+bPr6+qpTp046fPhwRZRYLkrTZ1hYmHx9feXt7e1e16ZNG504cUL5+fny8/Or0JpLoyyf55kzZ7Rs2TJNnz69IkssF6Xp88knn9TQoUMVFxcnSWrfvr3OnDmj0aNHa/LkyfLyqtq73LjHzhJ+fn667rrrlJiY6F5XWFioxMTEIv9H/EvR0dFFxkvShx9+eNHxV4LS9FkdlbbPuXPnasaMGVq7dq26dOlSGaWWSXl9noWFhcrLy6uIEsvM0x5bt26tPXv2KDk52b389re/1U033aTk5GSFh4dXZvklVh6fZUFBgfbs2aOwsLCKKrPMStNnt27ddPjwYXdAl6RDhw4pLCzsigx1Utk+z3feeUd5eXn6wx/+UNFllllp+vzxxx8vCG/nQ7sxpuKKLakqfngD5WjZsmXG6XSahIQEs3//fjN69GhTu3Zt968PGDp0qHn88cfd4zdv3mx8fHzMvHnzTEpKipkyZUq1+XUnnvSZl5dndu3aZXbt2mXCwsLMxIkTza5du8yXX35ZVS2UiKd9zpkzx/j5+Znly5cX+ZUDOTk5VdVCiXja56xZs8z69etNamqq2b9/v5k3b57x8fExixcvrqoWLsvTHv+v6vJUrKd9Tps2zaxbt86kpqaanTt3msGDBxt/f3+zb9++qmqhRDzt8+jRoyY4ONiMGzfOHDx40KxatcrUr1/fzJw5s6paKJHS/nd74403mkGDBlV2uaXmaZ9TpkwxwcHB5q233jJHjhwx69evN82bNzd33XVXVbVQBMHOMi+++KK5+uqrjZ+fn7nhhhvMp59+6t7Ws2dPc9999xUZ//bbb5vIyEjj5+dnrr32WvPBBx9UcsWl40mfaWlpRtIFS8+ePSu/cA950mfTpk2L7XPKlCmVX7iHPOlz8uTJpkWLFsbf39/UqVPHREdHm2XLllVB1Z7x9M/mL1WXYGeMZ31OmDDBPbZBgwamX79+5vPPP6+Cqj3n6ee5ZcsW07VrV+N0Os0111xjnn76aXPu3LlKrtpznvZ54MABI8msX7++kistG0/6PHv2rJk6dapp3ry58ff3N+Hh4Wbs2LHm1KlTlV94MRzGXAnzhgAAACgr7rEDAACwBMEOAADAEgQ7AAAASxDsAAAALEGwAwAAsATBDgAAwBIEOwAAAEsQ7AAAACxBsAMAALAEwQ4AAMASBDsAAABLEOwAAAAs8f8Aro0BctUWX1YAAAAASUVORK5CYII=\n" }, "metadata": {} } ], "metadata": { "id": "nCIqrkixWTVe", "outputId": "0a7a1644-ec43-452f-e204-e5ac9dbed514", "colab": { "base_uri": "https://localhost:8080/", "height": 430 } } }, { "cell_type": "markdown", "source": [ "Ces scores reflettent -t-il bien les labels des phrases présents dans JIGSAW? (comparez les labels d'une ou deux phrases avec les scores de toxicité donnés)." ], "metadata": { "id": "3Wj6xXyTxDC1" } }, { "cell_type": "markdown", "source": [ "Pour savoir si Detoxify est un bon algorithme de classification, nous allons nous intéresser à la courbe AUC PR(Area Under the Curve - Precision Recall). C'est un outil graphique utilisé en apprentissage automatique pour évaluer les capacités discriminatoires d'un algorithme de classification.\n", "\n", "La courbe AUC PR est construite en traçant la précision (Precision) en fonction du rappel (Recall) à différents seuils de classification.\n", "\n", "La précision est la proportion de vrais positifs parmi tous les exemples classés comme positifs, tandis que le rappel (également appelé sensibilité) est la proportion de vrais positifs parmi tous les exemples réellement positifs.\n", "\n", "L'aire sous la courbe AUC PR (AUC pour Area Under the Curve) donne une mesure unique de la performance du modèle qui résume la qualité de la précision et du rappel pour tous les seuils possibles. Un AUC de 1.0 indique une performance parfaite, tandis qu'un AUC de 0.5 indique une performance équivalente à une classification aléatoire.\n", "\n", "N'hesitez pas à aller faire un tour sur ce site pour bien comprendre de quoi il s'agit 😉:\n", "https://kobia.fr/classification-metrics-precision-recall/\n", "\n", "Comme vous l'aurez sans doute compris, nous allons tracer la courbe AUC-PR de Detoxify pour voir à quel point le model est performant.🤯\n" ], "metadata": { "id": "_SOXwaJ0xzzW" } }, { "cell_type": "markdown", "source": [ "Dans un premier temps, pour tous les commentaires de JIGSAW, calculez le score de toxicité et stocker ces scores dans un array.(on ne s'interessera qu'au label toxicity)\n", "Dans un second temps, stocker dans un autre array les labels \"toxic\" des commentaire de Jigsaw." ], "metadata": { "id": "UDgMMhdr2_ic" } }, { "cell_type": "code", "execution_count": 10, "source": [ "SCORE = []\n", "labels = []\n", "i = 0\n", "for comment in dataset['comment_text']:\n", " toxicity_result = check_toxicity(comment)['toxicity']\n", " SCORE.append(toxicity_result)\n", " if toxicity_result > 0.5 :\n", " labels.append(1)\n", " else :\n", " labels.append(0)\n", "print(labels)\n" ], "outputs": [ { "output_type": "stream", "name": "stderr", "text": [ "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.\n", " warnings.warn(\n" ] }, { "output_type": "stream", "name": "stdout", "text": [ "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1]\n" ] } ], "metadata": { "id": "MODrXeMu2-5e", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "3f023ddd-14a3-4d52-9a06-7de537f3ab8b" } }, { "cell_type": "markdown", "source": [ "En utilisant la librairie scikit-learn, tracer AUC PR de Detoxify. Que vaut l'air sous cette courbe ? Concluez.\n" ], "metadata": { "id": "dlv5bQjj4ADM" } }, { "cell_type": "code", "execution_count": 11, "source": [ "!pip install detoxify\n", "\n", "\n", "import numpy as np\n", "from detoxify import Detoxify\n", "from sklearn.metrics import precision_recall_curve, auc\n", "import matplotlib.pyplot as plt\n", "\n", "# Calculer la courbe PR et l'aire sous la courbe\n", "precision, recall, _ = precision_recall_curve([0 for _ in range(25)]+[1 for i in range(25)], SCORE)\n", "auc_pr = auc(recall, precision)\n", "\n", "# Tracer la courbe PR\n", "plt.figure()\n", "plt.plot(recall, precision, label=f'AUC PR = {auc_pr:.2f}')\n", "plt.xlabel('Recall')\n", "plt.ylabel('Precision')\n", "plt.title('Precision-Recall Curve')\n", "plt.legend(loc='best')\n", "plt.show()\n", "\n", "print(f'L\\'aire sous la courbe PR (AUC PR) est de : {auc_pr:.2f}')\n" ], "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Requirement already satisfied: detoxify in /usr/local/lib/python3.10/dist-packages (0.5.2)\n", "Requirement already satisfied: transformers in /usr/local/lib/python3.10/dist-packages (from detoxify) (4.41.1)\n", "Requirement already satisfied: torch>=1.7.0 in /usr/local/lib/python3.10/dist-packages (from detoxify) (2.3.0+cu121)\n", "Requirement already satisfied: sentencepiece>=0.1.94 in /usr/local/lib/python3.10/dist-packages (from detoxify) (0.1.99)\n", "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (3.14.0)\n", "Requirement already satisfied: typing-extensions>=4.8.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (4.12.0)\n", "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (1.12.1)\n", "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (3.3)\n", "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (3.1.4)\n", "Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (2023.6.0)\n", "Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (12.1.105)\n", "Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (12.1.105)\n", "Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (12.1.105)\n", "Requirement already satisfied: nvidia-cudnn-cu12==8.9.2.26 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (8.9.2.26)\n", "Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (12.1.3.1)\n", "Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (11.0.2.54)\n", "Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (10.3.2.106)\n", "Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (11.4.5.107)\n", "Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (12.1.0.106)\n", "Requirement already satisfied: nvidia-nccl-cu12==2.20.5 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (2.20.5)\n", "Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (12.1.105)\n", "Requirement already satisfied: triton==2.3.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.7.0->detoxify) (2.3.0)\n", "Requirement already satisfied: nvidia-nvjitlink-cu12 in /usr/local/lib/python3.10/dist-packages (from nvidia-cusolver-cu12==11.4.5.107->torch>=1.7.0->detoxify) (12.5.40)\n", "Requirement already satisfied: huggingface-hub<1.0,>=0.23.0 in /usr/local/lib/python3.10/dist-packages (from transformers->detoxify) (0.23.2)\n", "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from transformers->detoxify) (1.25.2)\n", "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from transformers->detoxify) (24.0)\n", "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from transformers->detoxify) (6.0.1)\n", "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers->detoxify) (2024.5.15)\n", "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers->detoxify) (2.32.3)\n", "Requirement already satisfied: tokenizers<0.20,>=0.19 in /usr/local/lib/python3.10/dist-packages (from transformers->detoxify) (0.19.1)\n", "Requirement already satisfied: safetensors>=0.4.1 in /usr/local/lib/python3.10/dist-packages (from transformers->detoxify) (0.4.3)\n", "Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/dist-packages (from transformers->detoxify) (4.66.4)\n", "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch>=1.7.0->detoxify) (2.1.5)\n", "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->transformers->detoxify) (3.3.2)\n", "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->transformers->detoxify) (3.7)\n", "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->transformers->detoxify) (2.0.7)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->transformers->detoxify) (2024.2.2)\n", "Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=1.7.0->detoxify) (1.3.0)\n" ] }, { "output_type": "display_data", "data": { "text/plain": [ "
" ], "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAAHHCAYAAABDUnkqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABAOElEQVR4nO3deVxWZf7/8ffNdgOyqQgqkeReZi6YDJpZRpKajU4zOmaJTpqmfsekTcukZZJscqvcKpfq56Rp1lgaZpSV6Yxl2rS4pqWpoJYsgqz39fvDuOsOUEHghuPr+Xjcj7ivc51zPueSvN+ec51z24wxRgAAABbh4e4CAAAAqhLhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBrgIDR8+XFFRURVaZ+PGjbLZbNq4cWO11FTXXXfddbruuuuc77///nvZbDYtXbrUbTUBFyvCDVADli5dKpvN5nz5+vqqdevWGj9+vNLT091dXq1XEhRKXh4eHmrQoIH69OmjLVu2uLu8KpGenq777rtPbdu2lb+/v+rVq6fo6Gj94x//UEZGhrvLA+oUL3cXAFxMHn/8cV122WXKy8vTpk2bNH/+fK1bt05ff/21/P39a6yOF198UQ6Ho0LrXHvttTp9+rR8fHyqqapzGzJkiPr27avi4mLt2bNH8+bN0/XXX6/PPvtM7du3d1tdF+qzzz5T3759derUKd1+++2Kjo6WJH3++ed66qmn9PHHH+u9995zc5VA3UG4AWpQnz591KVLF0nSyJEj1bBhQ82cOVP//ve/NWTIkDLXycnJUb169aq0Dm9v7wqv4+HhIV9f3yqto6I6d+6s22+/3fm+R48e6tOnj+bPn6958+a5sbLKy8jI0MCBA+Xp6ant27erbdu2LsuffPJJvfjii1Wyr+r4XQJqIy5LAW7Uq1cvSdKBAwcknZkLExAQoO+++059+/ZVYGCghg4dKklyOByaPXu22rVrJ19fX4WHh2v06NE6efJkqe2+++676tmzpwIDAxUUFKSrr75a//rXv5zLy5pzs3z5ckVHRzvXad++vebMmeNcXt6cm5UrVyo6Olp+fn4KDQ3V7bffrsOHD7v0KTmuw4cPa8CAAQoICFCjRo103333qbi4uNLj16NHD0nSd99959KekZGhe+65R5GRkbLb7WrZsqWmT59e6myVw+HQnDlz1L59e/n6+qpRo0a66aab9Pnnnzv7LFmyRL169VJYWJjsdruuuOIKzZ8/v9I1/97ChQt1+PBhzZw5s1SwkaTw8HBNmTLF+d5ms+nRRx8t1S8qKkrDhw93vi+5FPrRRx9p7NixCgsL0yWXXKJVq1Y528uqxWaz6euvv3a27dq1S3/+85/VoEED+fr6qkuXLlqzZs2FHTRQzThzA7hRyYdyw4YNnW1FRUWKj4/XNddco2eeecZ5uWr06NFaunSpRowYob///e86cOCAnn/+eW3fvl2ffvqp82zM0qVL9be//U3t2rXT5MmTFRISou3btyslJUW33XZbmXVs2LBBQ4YM0Q033KDp06dLknbu3KlPP/1UEyZMKLf+knquvvpqJScnKz09XXPmzNGnn36q7du3KyQkxNm3uLhY8fHxiomJ0TPPPKP3339fM2bMUIsWLXT33XdXavy+//57SVL9+vWdbbm5uerZs6cOHz6s0aNH69JLL9XmzZs1efJkHT16VLNnz3b2vfPOO7V06VL16dNHI0eOVFFRkT755BP95z//cZ5hmz9/vtq1a6dbbrlFXl5eevvttzV27Fg5HA6NGzeuUnX/1po1a+Tn56c///nPF7ytsowdO1aNGjXS1KlTlZOTo379+ikgIECvv/66evbs6dJ3xYoVateuna688kpJ0jfffKPu3bsrIiJCkyZNUr169fT6669rwIABeuONNzRw4MBqqRm4YAZAtVuyZImRZN5//31z/Phxc+jQIbN8+XLTsGFD4+fnZ3788UdjjDEJCQlGkpk0aZLL+p988omRZJYtW+bSnpKS4tKekZFhAgMDTUxMjDl9+rRLX4fD4fw5ISHBNGvWzPl+woQJJigoyBQVFZV7DB9++KGRZD788ENjjDEFBQUmLCzMXHnllS77euedd4wkM3XqVJf9STKPP/64yzY7depkoqOjy91niQMHDhhJ5rHHHjPHjx83aWlp5pNPPjFXX321kWRWrlzp7PvEE0+YevXqmT179rhsY9KkScbT09McPHjQGGPMBx98YCSZv//976X299uxys3NLbU8Pj7eNG/e3KWtZ8+epmfPnqVqXrJkyVmPrX79+qZDhw5n7fNbkkxSUlKp9mbNmpmEhATn+5LfuWuuuabUn+uQIUNMWFiYS/vRo0eNh4eHy5/RDTfcYNq3b2/y8vKcbQ6Hw3Tr1s20atXqvGsGahqXpYAaFBcXp0aNGikyMlJ//etfFRAQoDfffFMREREu/X5/JmPlypUKDg7WjTfeqBMnTjhf0dHRCggI0IcffijpzBmY7OxsTZo0qdT8GJvNVm5dISEhysnJ0YYNG877WD7//HMdO3ZMY8eOddlXv3791LZtW61du7bUOmPGjHF536NHD+3fv/+895mUlKRGjRqpcePG6tGjh3bu3KkZM2a4nPVYuXKlevToofr167uMVVxcnIqLi/Xxxx9Lkt544w3ZbDYlJSWV2s9vx8rPz8/5c2Zmpk6cOKGePXtq//79yszMPO/ay5OVlaXAwMAL3k55Ro0aJU9PT5e2wYMH69ixYy6XGFetWiWHw6HBgwdLkn7++Wd98MEHGjRokLKzs53j+NNPPyk+Pl579+4tdfkRqC24LAXUoLlz56p169by8vJSeHi42rRpIw8P139jeHl56ZJLLnFp27t3rzIzMxUWFlbmdo8dOybp18tcJZcVztfYsWP1+uuvq0+fPoqIiFDv3r01aNAg3XTTTeWu88MPP0iS2rRpU2pZ27ZttWnTJpe2kjktv1W/fn2XOUPHjx93mYMTEBCggIAA5/u77rpLf/nLX5SXl6cPPvhAzz77bKk5O3v37tX//ve/Uvsq8duxatq0qRo0aFDuMUrSp59+qqSkJG3ZskW5ubkuyzIzMxUcHHzW9c8lKChI2dnZF7SNs7nssstKtd10000KDg7WihUrdMMNN0g6c0mqY8eOat26tSRp3759MsbokUce0SOPPFLmto8dO1YqmAO1AeEGqEFdu3Z1zuUoj91uLxV4HA6HwsLCtGzZsjLXKe+D/HyFhYVpx44dWr9+vd599129++67WrJkiYYNG6aXX375grZd4vdnD8py9dVXO0OTdOZMzW8nz7Zq1UpxcXGSpJtvvlmenp6aNGmSrr/+eue4OhwO3XjjjXrggQfK3EfJh/f5+O6773TDDTeobdu2mjlzpiIjI+Xj46N169Zp1qxZFb6dvixt27bVjh07VFBQcEG32Zc3Mfu3Z55K2O12DRgwQG+++abmzZun9PR0ffrpp5o2bZqzT8mx3XfffYqPjy9z2y1btqx0vUB1ItwAdUCLFi30/vvvq3v37mV+WP22nyR9/fXXFf7g8fHxUf/+/dW/f385HA6NHTtWCxcu1COPPFLmtpo1ayZJ2r17t/OurxK7d+92Lq+IZcuW6fTp0873zZs3P2v/hx9+WC+++KKmTJmilJQUSWfG4NSpU84QVJ4WLVpo/fr1+vnnn8s9e/P2228rPz9fa9as0aWXXupsL7kMWBX69++vLVu26I033ij3cQC/Vb9+/VIP9SsoKNDRo0crtN/Bgwfr5ZdfVmpqqnbu3CljjPOSlPTr2Ht7e59zLIHahjk3QB0waNAgFRcX64knnii1rKioyPlh17t3bwUGBio5OVl5eXku/Ywx5W7/p59+cnnv4eGhq666SpKUn59f5jpdunRRWFiYFixY4NLn3Xff1c6dO9WvX7/zOrbf6t69u+Li4pyvc4WbkJAQjR49WuvXr9eOHTsknRmrLVu2aP369aX6Z2RkqKioSJJ06623yhijxx57rFS/krEqOdv027HLzMzUkiVLKnxs5RkzZoyaNGmie++9V3v27Cm1/NixY/rHP/7hfN+iRQvnvKESL7zwQoVvqY+Li1ODBg20YsUKrVixQl27dnW5hBUWFqbrrrtOCxcuLDM4HT9+vEL7A2oSZ26AOqBnz54aPXq0kpOTtWPHDvXu3Vve3t7au3evVq5cqTlz5ujPf/6zgoKCNGvWLI0cOVJXX321brvtNtWvX19ffvmlcnNzy73ENHLkSP3888/q1auXLrnkEv3www967rnn1LFjR11++eVlruPt7a3p06drxIgR6tmzp4YMGeK8FTwqKkoTJ06sziFxmjBhgmbPnq2nnnpKy5cv1/333681a9bo5ptv1vDhwxUdHa2cnBx99dVXWrVqlb7//nuFhobq+uuv1x133KFnn31We/fu1U033SSHw6FPPvlE119/vcaPH6/evXs7z2iNHj1ap06d0osvvqiwsLAKnykpT/369fXmm2+qb9++6tixo8sTir/44gu99tprio2NdfYfOXKkxowZo1tvvVU33nijvvzyS61fv16hoaEV2q+3t7f+9Kc/afny5crJydEzzzxTqs/cuXN1zTXXqH379ho1apSaN2+u9PR0bdmyRT/++KO+/PLLCzt4oLq481Yt4GJRclvuZ599dtZ+CQkJpl69euUuf+GFF0x0dLTx8/MzgYGBpn379uaBBx4wR44ccem3Zs0a061bN+Pn52eCgoJM165dzWuvveayn9/eCr5q1SrTu3dvExYWZnx8fMyll15qRo8ebY4ePers8/tbwUusWLHCdOrUydjtdtOgQQMzdOhQ563t5zqupKQkcz5/DZXcVv3Pf/6zzOXDhw83np6eZt++fcYYY7Kzs83kyZNNy5YtjY+PjwkNDTXdunUzzzzzjCkoKHCuV1RUZP75z3+atm3bGh8fH9OoUSPTp08fs23bNpexvOqqq4yvr6+Jiooy06dPN4sXLzaSzIEDB5z9KnsreIkjR46YiRMnmtatWxtfX1/j7+9voqOjzZNPPmkyMzOd/YqLi82DDz5oQkNDjb+/v4mPjzf79u0r91bws/3ObdiwwUgyNpvNHDp0qMw+3333nRk2bJhp3Lix8fb2NhEREebmm282q1atOq/jAtzBZsxZzlUDAADUMcy5AQAAlkK4AQAAlkK4AQAAlkK4AQAAlkK4AQAAlkK4AQAAlnLRPcTP4XDoyJEjCgwMPOu3JAMAgNrDGKPs7Gw1bdq01Pfv/d5FF26OHDmiyMhId5cBAAAq4dChQ7rkkkvO2ueiCzeBgYGSzgxOUFCQm6sBAADnIysrS5GRkc7P8bO56MJNyaWooKAgwg0AAHXM+UwpYUIxAACwFMINAACwFMINAACwFMINAACwFMINAACwFMINAACwFMINAACwFMINAACwFMINAACwFMINAACwFLeGm48//lj9+/dX06ZNZbPZ9NZbb51znY0bN6pz586y2+1q2bKlli5dWu11AgCAusOt4SYnJ0cdOnTQ3Llzz6v/gQMH1K9fP11//fXasWOH7rnnHo0cOVLr16+v5koBAEBd4dYvzuzTp4/69Olz3v0XLFigyy67TDNmzJAkXX755dq0aZNmzZql+Pj46irzvOQXFet4dr5bawAAK/Lz9lTDALu7y0AdUqe+FXzLli2Ki4tzaYuPj9c999xT7jr5+fnKz/81dGRlZVVLbd8cydKf5m2ulm0DwMXMZpNmDeqoAZ0i3F0K6og6FW7S0tIUHh7u0hYeHq6srCydPn1afn5+pdZJTk7WY489Vu212STZvZifDQBVqchhVOww+upwJuEG561OhZvKmDx5shITE53vs7KyFBkZWeX76XRpfe3+x/lfYgMAnNv0lF2av/E7d5eBOqZOhZvGjRsrPT3dpS09PV1BQUFlnrWRJLvdLruda7UAAFws6tR1lNjYWKWmprq0bdiwQbGxsW6qCAAA1DZuDTenTp3Sjh07tGPHDklnbvXesWOHDh48KOnMJaVhw4Y5+48ZM0b79+/XAw88oF27dmnevHl6/fXXNXHiRHeUDwAAaiG3hpvPP/9cnTp1UqdOnSRJiYmJ6tSpk6ZOnSpJOnr0qDPoSNJll12mtWvXasOGDerQoYNmzJihl156ye23gQMAgNrDrXNurrvuOhljyl1e1tOHr7vuOm3fvr0aqwIAAHVZnZpzAwAAcC6EGwAAYCmEGwAAYCmEGwAAYCmEGwAAYCmEGwAAYCmEGwAAYCmEGwAAYCmEGwAAYCmEGwAAYCmEGwAAYCmEGwAAYCmEGwAAYCmEGwAAYCmEGwAAYCmEGwAAYCmEGwAAYCle7i4AAADUTcYY5RU6lJ1fqOy8ImXnFSmvsFgdI0Pk6+3ptroINwAAXISMMcotKFZWXqGyThcpO69QWXm/hpTsvCKdynd9n/3L8lP5v/5c5DCltn1Lh6Z6dkgnNxzVGYQbAADqoJJwknm60BlQspw/FyrrlzCSdbroTNtvf/5leXEZwaQybDYpwO4lTw+bMnIL9ePJ3CrZbmURbgAAcKO8wjMBpeSVkfvrz5m5Bb+2/6ZP1i/9yjprUlGeHjYF+Xop0NdbQX5eCrR7K8DXS4G+Xgry9Vagr5cC7GeWB/p6KcDXy9m/ZFk9Hy95eNj03jdpuuvVbVUwKheGcAMAQBUoLHYoI7dQGbkFyjhdqJM5BcrILdTJ3AKdLGn/5X3JfzNPFyq/yHFB+/X2tCnYz9sZRIJ++TnIz+uX/3q7hJdf28689/P2lM1mq6JRqB0INwAA/E5RsUMncwv1c07Br6/cAv18qkAnc8+8zzh9JrCczC1QRk6hsvOLKr0/D5sU7Od95uXv4/w5pKTNuezXn0N++dmK4eRCEW4AAJaXV1isE6fy9XNOgX7KKdDJXwJLyc8//fK+5OfM04WV3lewn7fq+3srxN9HIf7eqv+b/5a0l7SVBJaAXy7roGoQbgAAdY4xRtn5RTqRna8Tpwr006l8nTiVr+OnCnTiVP4v70t+LtCpSpxVsdmkED9v1a/no4b1zgSShgFn/tugns+Z8OLnrfr1fg0swX7e8iSkuB3hBgBQ623cfUwHTuToxKn8M4Emp0AFFZyr4uPpoQb1fM75aljPR/XrnQkuXp4867YuItwAAGqtej5nHgT33fEcfXc8p9TyALuXGgb4KDTArtAAHzUMsCs0wK5Gv/m5pD3I14u5KRcJwg0AoNYa0vVSFTskTw/9ElTsCg20q2G9M4HGz8d9T8FF7UW4AQDUWg0D7JoQ18rdZaCO4WIiAACwFMINAACwFMINAACwFMINAACwFMINAACwFMINAACwFMINAACwFMINAACwFMINAACwFMINAACwFMINAACwFMINAACwFMINAACwFMINAACwFMINAACwFMINAACwFMINAACwFMINAACwFMINAACwFMINAACwFMINAACwFMINAACwFMINAACwFMINAACwFMINAACwFMINAACwFMINAACwFLeHm7lz5yoqKkq+vr6KiYnR1q1by+1bWFioxx9/XC1atJCvr686dOiglJSUGqwWAADUdm4NNytWrFBiYqKSkpL0xRdfqEOHDoqPj9exY8fK7D9lyhQtXLhQzz33nL799luNGTNGAwcO1Pbt22u4cgAAUFu5NdzMnDlTo0aN0ogRI3TFFVdowYIF8vf31+LFi8vs/+qrr+qhhx5S37591bx5c919993q27evZsyYUcOVAwCA2spt4aagoEDbtm1TXFzcr8V4eCguLk5btmwpc538/Hz5+vq6tPn5+WnTpk3l7ic/P19ZWVkuLwAAYF1uCzcnTpxQcXGxwsPDXdrDw8OVlpZW5jrx8fGaOXOm9u7dK4fDoQ0bNmj16tU6evRouftJTk5WcHCw8xUZGVmlxwEAAGoXt08orog5c+aoVatWatu2rXx8fDR+/HiNGDFCHh7lH8bkyZOVmZnpfB06dKgGKwYAADXNbeEmNDRUnp6eSk9Pd2lPT09X48aNy1ynUaNGeuutt5STk6MffvhBu3btUkBAgJo3b17ufux2u4KCglxeAADAutwWbnx8fBQdHa3U1FRnm8PhUGpqqmJjY8+6rq+vryIiIlRUVKQ33nhDf/zjH6u7XAAAUEd4uXPniYmJSkhIUJcuXdS1a1fNnj1bOTk5GjFihCRp2LBhioiIUHJysiTpv//9rw4fPqyOHTvq8OHDevTRR+VwOPTAAw+48zAAAEAt4tZwM3jwYB0/flxTp05VWlqaOnbsqJSUFOck44MHD7rMp8nLy9OUKVO0f/9+BQQEqG/fvnr11VcVEhLipiMAAAC1jc0YY9xdRE3KyspScHCwMjMzmX8DAEAVeu+bNN316jZ1vjREq8d2r9JtV+Tzu07dLQUAAHAuhBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGAphBsAAGApbg83c+fOVVRUlHx9fRUTE6OtW7eetf/s2bPVpk0b+fn5KTIyUhMnTlReXl4NVQsAAGo7t4abFStWKDExUUlJSfriiy/UoUMHxcfH69ixY2X2/9e//qVJkyYpKSlJO3fu1KJFi7RixQo99NBDNVw5AACordwabmbOnKlRo0ZpxIgRuuKKK7RgwQL5+/tr8eLFZfbfvHmzunfvrttuu01RUVHq3bu3hgwZcs6zPQAA4OLhtnBTUFCgbdu2KS4u7tdiPDwUFxenLVu2lLlOt27dtG3bNmeY2b9/v9atW6e+ffuWu5/8/HxlZWW5vAAAgHV5uWvHJ06cUHFxscLDw13aw8PDtWvXrjLXue2223TixAldc801MsaoqKhIY8aMOetlqeTkZD322GNVWjsAAKi93D6huCI2btyoadOmad68efriiy+0evVqrV27Vk888US560yePFmZmZnO16FDh2qwYgAAUNPcduYmNDRUnp6eSk9Pd2lPT09X48aNy1znkUce0R133KGRI0dKktq3b6+cnBzdddddevjhh+XhUTqr2e122e32qj8AAABQK7ntzI2Pj4+io6OVmprqbHM4HEpNTVVsbGyZ6+Tm5pYKMJ6enpIkY0z1FQsAAOoMt525kaTExEQlJCSoS5cu6tq1q2bPnq2cnByNGDFCkjRs2DBFREQoOTlZktS/f3/NnDlTnTp1UkxMjPbt26dHHnlE/fv3d4YcAABwcXNruBk8eLCOHz+uqVOnKi0tTR07dlRKSopzkvHBgwddztRMmTJFNptNU6ZM0eHDh9WoUSP1799fTz75pLsOAQAA1DI2c5Fdz8nKylJwcLAyMzMVFBTk7nIAALCM975J012vblPnS0O0emz3Kt12RT6/K3Xmpri4WEuXLlVqaqqOHTsmh8PhsvyDDz6ozGYBAAAuWKXCzYQJE7R06VL169dPV155pWw2W1XXBQAAUCmVCjfLly/X66+/ftYnAwMAALhDpW4F9/HxUcuWLau6FgAAgAtWqXBz7733as6cOTxbBgAA1DqVuiy1adMmffjhh3r33XfVrl07eXt7uyxfvXp1lRQHAABQUZUKNyEhIRo4cGBV1wIAAHDBKhVulixZUtV1AAAAVIkLekLx8ePHtXv3bklSmzZt1KhRoyopCgAAoLIqNaE4JydHf/vb39SkSRNde+21uvbaa9W0aVPdeeedys3NreoaAQAAzlulwk1iYqI++ugjvf3228rIyFBGRob+/e9/66OPPtK9995b1TUCAACct0pdlnrjjTe0atUqXXfddc62vn37ys/PT4MGDdL8+fOrqj4AAIAKqdSZm9zcXOc3d/9WWFgYl6UAAIBbVSrcxMbGKikpSXl5ec6206dP67HHHlNsbGyVFQcAAFBRlbosNWfOHMXHx+uSSy5Rhw4dJElffvmlfH19tX79+iotEAAAoCIqFW6uvPJK7d27V8uWLdOuXbskSUOGDNHQoUPl5+dXpQUCAABURKWfc+Pv769Ro0ZVZS0AAAAX7LzDzZo1a9SnTx95e3trzZo1Z+17yy23XHBhAAAAlXHe4WbAgAFKS0tTWFiYBgwYUG4/m82m4uLiqqgNAACgws473DgcjjJ/BgAAqE0qdSt4WTIyMqpqUwAAAJVWqXAzffp0rVixwvn+L3/5ixo0aKCIiAh9+eWXVVYcAABARVUq3CxYsECRkZGSpA0bNuj9999XSkqK+vTpo/vvv79KCwQAAKiISt0KnpaW5gw377zzjgYNGqTevXsrKipKMTExVVogAABARVTqzE39+vV16NAhSVJKSori4uIkScYY7pQCAABuVakzN3/605902223qVWrVvrpp5/Up08fSdL27dvVsmXLKi0QAACgIioVbmbNmqWoqCgdOnRITz/9tAICAiRJR48e1dixY6u0QAAAgIqoVLjx9vbWfffdV6p94sSJF1wQAADAheDrFwAAgKXw9QsAAMBS+PoFAABgKVX29QsAAAC1QaXCzd///nc9++yzpdqff/553XPPPRdaEwAAQKVVKty88cYb6t69e6n2bt26adWqVRdcFAAAQGVVKtz89NNPCg4OLtUeFBSkEydOXHBRAAAAlVWpcNOyZUulpKSUan/33XfVvHnzCy4KAACgsir1EL/ExESNHz9ex48fV69evSRJqampmjFjhmbPnl2V9QEAAFRIpcLN3/72N+Xn5+vJJ5/UE088IUmKiorS/PnzNWzYsCotEAAAoCIqFW4k6e6779bdd9+t48ePy8/Pz/n9UgAAAO5U6efcFBUV6f3339fq1atljJEkHTlyRKdOnaqy4gAAACqqUmdufvjhB9100006ePCg8vPzdeONNyowMFDTp09Xfn6+FixYUNV1AgAAnJdKnbmZMGGCunTpopMnT8rPz8/ZPnDgQKWmplZZcQAAABVVqTM3n3zyiTZv3iwfHx+X9qioKB0+fLhKCgMAAKiMSp25cTgcZX7z948//qjAwMALLgoAAKCyKhVuevfu7fI8G5vNplOnTikpKUl9+/atqtoAAAAqrFKXpZ555hnddNNNuuKKK5SXl6fbbrtNe/fuVWhoqF577bWqrhEAAOC8VSrcREZG6ssvv9SKFSv05Zdf6tSpU7rzzjs1dOhQlwnGAAAANa3C4aawsFBt27bVO++8o6FDh2ro0KHVURcAAEClVHjOjbe3t/Ly8qqjFgAAgAtWqQnF48aN0/Tp01VUVFTV9QAAAFyQSs25+eyzz5Samqr33ntP7du3V7169VyWr169ukqKAwAAqKhKhZuQkBDdeuutVV0LAADABatQuHE4HPrnP/+pPXv2qKCgQL169dKjjz7KHVIAAKDWqNCcmyeffFIPPfSQAgICFBERoWeffVbjxo2rrtoAAAAqrELh5pVXXtG8efO0fv16vfXWW3r77be1bNkyORyO6qoPAACgQioUbg4ePOjy9QpxcXGy2Ww6cuRIlRcGAABQGRUKN0VFRfL19XVp8/b2VmFh4QUVMXfuXEVFRcnX11cxMTHaunVruX2vu+462Wy2Uq9+/fpdUA0AAMAaKjSh2Bij4cOHy263O9vy8vI0ZswYl9vBK3Ir+IoVK5SYmKgFCxYoJiZGs2fPVnx8vHbv3q2wsLBS/VevXq2CggLn+59++kkdOnTQX/7yl4ocCgAAsKgKhZuEhIRSbbfffvsFFTBz5kyNGjVKI0aMkCQtWLBAa9eu1eLFizVp0qRS/Rs0aODyfvny5fL39yfcAAAASRUMN0uWLKnSnRcUFGjbtm2aPHmys83Dw0NxcXHasmXLeW1j0aJF+utf/1rqQYIAAODiVKmH+FWVEydOqLi4WOHh4S7t4eHh2rVr1znX37p1q77++mstWrSo3D75+fnKz893vs/Kyqp8wQAAoNar1HdL1RaLFi1S+/bt1bVr13L7JCcnKzg42PmKjIyswQoBAEBNc2u4CQ0Nlaenp9LT013a09PT1bhx47Oum5OTo+XLl+vOO+88a7/JkycrMzPT+Tp06NAF1w0AAGovt4YbHx8fRUdHKzU11dnmcDiUmpqq2NjYs667cuVK5efnn3NCs91uV1BQkMsLAABYl1vn3EhSYmKiEhIS1KVLF3Xt2lWzZ89WTk6O8+6pYcOGKSIiQsnJyS7rLVq0SAMGDFDDhg3dUTYAAKil3B5uBg8erOPHj2vq1KlKS0tTx44dlZKS4pxkfPDgQXl4uJ5g2r17tzZt2qT33nvPHSUDAIBazO3hRpLGjx+v8ePHl7ls48aNpdratGkjY0w1VwUAAOqiOn23FAAAwO8RbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKW4PdzMnTtXUVFR8vX1VUxMjLZu3XrW/hkZGRo3bpyaNGkiu92u1q1ba926dTVULQAAqO283LnzFStWKDExUQsWLFBMTIxmz56t+Ph47d69W2FhYaX6FxQU6MYbb1RYWJhWrVqliIgI/fDDDwoJCan54gEAQK3k1nAzc+ZMjRo1SiNGjJAkLViwQGvXrtXixYs1adKkUv0XL16sn3/+WZs3b5a3t7ckKSoqqiZLBgAAtZzbLksVFBRo27ZtiouL+7UYDw/FxcVpy5YtZa6zZs0axcbGaty4cQoPD9eVV16padOmqbi4uKbKBgAAtZzbztycOHFCxcXFCg8Pd2kPDw/Xrl27ylxn//79+uCDDzR06FCtW7dO+/bt09ixY1VYWKikpKQy18nPz1d+fr7zfVZWVtUdBAAAqHXcPqG4IhwOh8LCwvTCCy8oOjpagwcP1sMPP6wFCxaUu05ycrKCg4Odr8jIyBqsGAAA1DS3hZvQ0FB5enoqPT3dpT09PV2NGzcuc50mTZqodevW8vT0dLZdfvnlSktLU0FBQZnrTJ48WZmZmc7XoUOHqu4gAABAreO2cOPj46Po6GilpqY62xwOh1JTUxUbG1vmOt27d9e+ffvkcDicbXv27FGTJk3k4+NT5jp2u11BQUEuLwAAYF1uvSyVmJioF198US+//LJ27typu+++Wzk5Oc67p4YNG6bJkyc7+9999936+eefNWHCBO3Zs0dr167VtGnTNG7cOHcdAgAAqGXceiv44MGDdfz4cU2dOlVpaWnq2LGjUlJSnJOMDx48KA+PX/NXZGSk1q9fr4kTJ+qqq65SRESEJkyYoAcffNBdhwAAAGoZmzHGuLuImpSVlaXg4GBlZmZyiQoAgCr03jdpuuvVbep8aYhWj+1epduuyOd3nbpbCgAA4FwINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFK83F1AbWSMUVFRkYqLi91dCizE09NTXl5estls7i4FACyNcPM7BQUFOnr0qHJzc91dCizI399fTZo0kY+Pj7tLAQDLItz8hsPh0IEDB+Tp6ammTZvKx8eHf2WjShhjVFBQoOPHj+vAgQNq1aqVPDy4KgwA1YFw8xsFBQVyOByKjIyUv7+/u8uBxfj5+cnb21s//PCDCgoK5Ovr6+6SAMCS+KdjGfgXNaoLv1sAUP34mxYAAFgK4QYAAFgK4cZitmzZIk9PT/Xr16/Uso0bN8pmsykjI6PUsqioKM2ePdul7cMPP1Tfvn3VsGFD+fv764orrtC9996rw4cPl7v/qKgo2Ww22Ww21atXT507d9bKlSudyx999FHnck9PT0VGRuquu+7Szz//XOljPl8rV65U27Zt5evrq/bt22vdunXnXGfu3Lm6/PLL5efnpzZt2uiVV15xWV5YWKjHH39cLVq0kK+vrzp06KCUlJTqOgQAwHkg3FjMokWL9H//93/6+OOPdeTIkUpvZ+HChYqLi1Pjxo31xhtv6Ntvv9WCBQuUmZmpGTNmnHXdxx9/XEePHtX27dt19dVXa/Dgwdq8ebNzebt27XT06FEdPHhQS5YsUUpKiu6+++5K13o+Nm/erCFDhujOO+/U9u3bNWDAAA0YMEBff/11uevMnz9fkydP1qOPPqpvvvlGjz32mMaNG6e3337b2WfKlClauHChnnvuOX377bcaM2aMBg4cqO3bt1fr8QAAzsJcZDIzM40kk5mZWWrZ6dOnzbfffmtOnz7thsouXHZ2tgkICDC7du0ygwcPNk8++aTL8g8//NBIMidPniy1brNmzcysWbOMMcYcOnTI+Pj4mHvuuafM/ZS1flnbMcaYwsJC4+/vbyZNmmSMMSYpKcl06NDBZZ3ExERTv379cx7fhRg0aJDp16+fS1tMTIwZPXp0uevExsaa++67z6UtMTHRdO/e3fm+SZMm5vnnn3fp86c//ckMHTq0zG3W9d8xADib9V8fNc0efMcMnLupyrd9ts/v3+PMzTkYY5RbUOSWlzGmQrW+/vrratu2rdq0aaPbb79dixcvrvA2pDOXbwoKCvTAAw+UuTwkJOS8t+Xl5SVvb28VFBSUufz777/X+vXrz/lQu2XLlikgIOCsr08++aTc9bds2aK4uDiXtvj4eG3ZsqXcdfLz80vdru3n56etW7eqsLDwrH02bdp01uMBAFQfnnNzDqcLi3XF1PVu2fe3j8fL3+f8/4gWLVqk22+/XZJ00003KTMzUx999JGuu+66Cu137969CgoKUpMmTSq03u8VFBRoxowZyszMVK9evZztX331lQICAlRcXKy8vDxJ0syZM8+6rVtuuUUxMTFn7RMREVHusrS0NIWHh7u0hYeHKy0trdx14uPj9dJLL2nAgAHq3Lmztm3bppdeekmFhYU6ceKEmjRpovj4eM2cOVPXXnutWrRoodTUVK1evZqv7gAAN6oVZ27mzp2rqKgo+fr6KiYmRlu3bi2379KlS50TUktePAxN2r17t7Zu3aohQ4ZIOnPGZPDgwVq0aFGFt2WMuaAnMz/44IMKCAiQv7+/pk+frqeeesplgnObNm20Y8cOffbZZ3rwwQcVHx+v//u//zvrNgMDA9WyZcuzvvz8/Cpdc1keeeQR9enTR3/4wx/k7e2tP/7xj0pISJD06/Nq5syZo1atWqlt27by8fHR+PHjNWLECJ5nAwBu5PYzNytWrFBiYqIWLFigmJgYzZ49W/Hx8dq9e7fCwsLKXCcoKEi7d+92vq/Or0jw8/bUt4/HV9v2z7Xv87Vo0SIVFRWpadOmzjZjjOx2u55//nkFBwcrKChIkpSZmVnq0lJGRoaCg4MlSa1bt1ZmZqaOHj1aqbM3999/v4YPH66AgACFh4eX+vPx8fFRy5YtJckZfB577DE98cQT5W5z2bJlGj169Fn3++6776pHjx5lLmvcuLHS09Nd2tLT09W4ceNyt+fn56fFixdr4cKFSk9PV5MmTfTCCy8oMDBQjRo1kiQ1atRIb731lvLy8vTTTz+padOmmjRpkpo3b37WWgEA1cft4WbmzJkaNWqURowYIUlasGCB1q5dq8WLF2vSpEllrmOz2c76oVSVbDZbhS4NuUNRUZFeeeUVzZgxQ71793ZZNmDAAL322msaM2aM8/uMtm3bpmbNmjn77N+/X5mZmWrdurUk6c9//rMmTZqkp59+WrNmzSq1v4yMjLPOuwkNDXWGl/MxZcoU9erVS3fffbdLOPutC70sFRsbq9TUVN1zzz3Otg0bNig2Nvac9Xl7e+uSSy6RJC1fvlw333xzqTMzvr6+ioiIUGFhod544w0NGjTonNsFAFQPt35qFxQUaNu2bZo8ebKzzcPDQ3FxcWed6Hnq1Ck1a9ZMDodDnTt31rRp09SuXbsy++bn5ys/P9/5Pisrq+oOoJZ45513dPLkSd15553Osy8lbr31Vi1atEhjxoxRYGCgRo4cqXvvvVdeXl5q3769Dh06pAcffFB/+MMf1K1bN0lSZGSkZs2apfHjxysrK0vDhg1TVFSUfvzxR73yyisKCAg45+3gFREbG6urrrpK06ZN0/PPP19mn8DAQAUGBlZ6HxMmTFDPnj01Y8YM9evXT8uXL9fnn3+uF154wdln8uTJOnz4sPNZNnv27NHWrVsVExOjkydPaubMmfr666/18ssvO9f573//q8OHD6tjx446fPiwHn30UTkcjnInYwMAqp9bJwacOHFCxcXFFZro2aZNGy1evFj//ve/9f/+3/+Tw+FQt27d9OOPP5bZPzk5WcHBwc5XZGRklR+Huy1atEhxcXGlgo10Jtx8/vnn+t///ifpzByRhIQEPfjgg2rXrp2GDx+uq666Sm+//bbL5aOxY8fqvffe0+HDhzVw4EC1bdtWI0eOVFBQkO67774qP4aJEyfqpZde0qFDh6p825LUrVs3/etf/9ILL7ygDh06aNWqVXrrrbd05ZVXOvuUPHunRHFxsWbMmKEOHTroxhtvVF5enjZv3qyoqChnn7y8PE2ZMkVXXHGFBg4cqIiICG3atKlCd5QBgFV42Gyye3nI29O98w5tpjL3CleRI0eOKCIiQps3b3a5PPDAAw/oo48+0n//+99zbqOwsFCXX365hgwZUuacjbLO3ERGRiozM9M5B6VEXl6eDhw4oMsuu4xJyqgW/I4BQOVkZWUpODi4zM/v33PrZanQ0FB5enpWeKLnb3l7e6tTp07at29fmcvtdrvsdvsF1woAAOoGt5438vHxUXR0tFJTU51tDodDqamp5zXRUzpz6eCrr7664GeyAAAAa3D7bUCJiYlKSEhQly5d1LVrV82ePVs5OTnOu6eGDRumiIgIJScnSzrzvUV/+MMf1LJlS2VkZOif//ynfvjhB40cOdKdhwEAAGoJt4ebwYMH6/jx45o6darS0tLUsWNHpaSkOCcZHzx40OW225MnT2rUqFFKS0tT/fr1FR0drc2bN+uKK65w1yEAAIBaxK0Tit3hbBOSmOyJ6sbvGABUTkUmFPOM+DJcZHkPNYjfLQCofoSb3/D29pYk5ebmurkSWFXJ71bJ7xoAoOq5fc5NbeLp6amQkBAdO3ZMkuTv71+t31uFi4cxRrm5uTp27JhCQkLk6Xn+3xsGAKgYws3vlDxfpyTgAFUpJCSkxr4XDQAuVoSb37HZbGrSpInCwsJUWFjo7nJgId7e3pyxAYAaQLgph6enJx9EAADUQUwoBgAAlkK4AQAAlkK4AQAAlnLRzbkpeYhaVlaWmysBAADnq+Rz+3wehnrRhZvs7GxJUmRkpJsrAQAAFZWdna3g4OCz9rnovlvK4XDoyJEjCgwMrPIH9GVlZSkyMlKHDh065/deoPIY55rBONcMxrnmMNY1o7rG2Rij7OxsNW3a1OULtcty0Z258fDw0CWXXFKt+wgKCuJ/nBrAONcMxrlmMM41h7GuGdUxzuc6Y1OCCcUAAMBSCDcAAMBSCDdVyG63KykpSXa73d2lWBrjXDMY55rBONccxrpm1IZxvugmFAMAAGvjzA0AALAUwg0AALAUwg0AALAUwg0AALAUwk0FzZ07V1FRUfL19VVMTIy2bt161v4rV65U27Zt5evrq/bt22vdunU1VGndVpFxfvHFF9WjRw/Vr19f9evXV1xc3Dn/XHBGRX+fSyxfvlw2m00DBgyo3gItoqLjnJGRoXHjxqlJkyay2+1q3bo1f3ech4qO8+zZs9WmTRv5+fkpMjJSEydOVF5eXg1VWzd9/PHH6t+/v5o2bSqbzaa33nrrnOts3LhRnTt3lt1uV8uWLbV06dJqr1MG52358uXGx8fHLF682HzzzTdm1KhRJiQkxKSnp5fZ/9NPPzWenp7m6aefNt9++62ZMmWK8fb2Nl999VUNV163VHScb7vtNjN37lyzfft2s3PnTjN8+HATHBxsfvzxxxquvG6p6DiXOHDggImIiDA9evQwf/zjH2um2DqsouOcn59vunTpYvr27Ws2bdpkDhw4YDZu3Gh27NhRw5XXLRUd52XLlhm73W6WLVtmDhw4YNavX2+aNGliJk6cWMOV1y3r1q0zDz/8sFm9erWRZN58882z9t+/f7/x9/c3iYmJ5ttvvzXPPfec8fT0NCkpKdVaJ+GmArp27WrGjRvnfF9cXGyaNm1qkpOTy+w/aNAg069fP5e2mJgYM3r06Gqts66r6Dj/XlFRkQkMDDQvv/xydZVoCZUZ56KiItOtWzfz0ksvmYSEBMLNeajoOM+fP980b97cFBQU1FSJllDRcR43bpzp1auXS1tiYqLp3r17tdZpJecTbh544AHTrl07l7bBgweb+Pj4aqzMGC5LnaeCggJt27ZNcXFxzjYPDw/FxcVpy5YtZa6zZcsWl/6SFB8fX25/VG6cfy83N1eFhYVq0KBBdZVZ51V2nB9//HGFhYXpzjvvrIky67zKjPOaNWsUGxurcePGKTw8XFdeeaWmTZum4uLimiq7zqnMOHfr1k3btm1zXrrav3+/1q1bp759+9ZIzRcLd30OXnRfnFlZJ06cUHFxscLDw13aw8PDtWvXrjLXSUtLK7N/WlpatdVZ11VmnH/vwQcfVNOmTUv9D4VfVWacN23apEWLFmnHjh01UKE1VGac9+/frw8++EBDhw7VunXrtG/fPo0dO1aFhYVKSkqqibLrnMqM82233aYTJ07ommuukTFGRUVFGjNmjB566KGaKPmiUd7nYFZWlk6fPi0/P79q2S9nbmApTz31lJYvX64333xTvr6+7i7HMrKzs3XHHXfoxRdfVGhoqLvLsTSHw6GwsDC98MILio6O1uDBg/Xwww9rwYIF7i7NUjZu3Khp06Zp3rx5+uKLL7R69WqtXbtWTzzxhLtLQxXgzM15Cg0Nlaenp9LT013a09PT1bhx4zLXady4cYX6o3LjXOKZZ57RU089pffff19XXXVVdZZZ51V0nL/77jt9//336t+/v7PN4XBIkry8vLR79261aNGieouugyrz+9ykSRN5e3vL09PT2Xb55ZcrLS1NBQUF8vHxqdaa66LKjPMjjzyiO+64QyNHjpQktW/fXjk5Obrrrrv08MMPy8ODf/tXhfI+B4OCgqrtrI3EmZvz5uPjo+joaKWmpjrbHA6HUlNTFRsbW+Y6sbGxLv0lacOGDeX2R+XGWZKefvppPfHEE0pJSVGXLl1qotQ6raLj3LZtW3311VfasWOH83XLLbfo+uuv144dOxQZGVmT5dcZlfl97t69u/bt2+cMj5K0Z88eNWnShGBTjsqMc25ubqkAUxIoDV+5WGXc9jlYrdOVLWb58uXGbrebpUuXmm+//dbcddddJiQkxKSlpRljjLnjjjvMpEmTnP0//fRT4+XlZZ555hmzc+dOk5SUxK3g56Gi4/zUU08ZHx8fs2rVKnP06FHnKzs7212HUCdUdJx/j7ulzk9Fx/ngwYMmMDDQjB8/3uzevdu88847JiwszPzjH/9w1yHUCRUd56SkJBMYGGhee+01s3//fvPee++ZFi1amEGDBrnrEOqE7Oxss337drN9+3YjycycOdNs377d/PDDD8YYYyZNmmTuuOMOZ/+SW8Hvv/9+s3PnTjN37lxuBa+NnnvuOXPppZcaHx8f07VrV/Of//zHuaxnz54mISHBpf/rr79uWrdubXx8fEy7du3M2rVra7jiuqki49ysWTMjqdQrKSmp5guvYyr6+/xbhJvzV9Fx3rx5s4mJiTF2u900b97cPPnkk6aoqKiGq657KjLOhYWF5tFHHzUtWrQwvr6+JjIy0owdO9acPHmy5guvQz788MMy/74tGduEhATTs2fPUut07NjR+Pj4mObNm5slS5ZUe502Yzj/BgAArIM5NwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAAwFIINwAgyWaz6a233pIkff/997LZbHwDOlBHEW4AuN3w4cNls9lks9nk7e2tyy67TA888IDy8vLcXRqAOohvBQdQK9x0001asmSJCgsLtW3bNiUkJMhms2n69OnuLg1AHcOZGwC1gt1uV+PGjRUZGakBAwYoLi5OGzZskHTmG56Tk5N12WWXyc/PTx06dNCqVatc1v/mm2908803KygoSIGBgerRo4e+++47SdJnn32mG2+8UaGhoQoODlbPnj31xRdf1PgxAqgZhBsAtc7XX3+tzZs3y8fHR5KUnJysV155RQsWLNA333yjiRMn6vbbb9dHH30kSTp8+LCuvfZa2e12ffDBB9q2bZv+9re/qaioSJKUnZ2thIQEbdq0Sf/5z3/UqlUr9e3bV9nZ2W47RgDVh8tSAGqFd955RwEBASoqKlJ+fr48PDz0/PPPKz8/X9OmTdP777+v2NhYSVLz5s21adMmLVy4UD179tTcuXMVHBys5cuXy9vbW5LUunVr57Z79erlsq8XXnhBISEh+uijj3TzzTfX3EECqBGEGwC1wvXXX6/58+crJydHs2bNkpeXl2699VZ98803ys3N1Y033ujSv6CgQJ06dZIk7dixQz169HAGm99LT0/XlClTtHHjRh07dkzFxcXKzc3VwYMHq/24ANQ8wg2AWqFevXpq2bKlJGnx4sXq0KGDFi1apCuvvFKStHbtWkVERLisY7fbJUl+fn5n3XZCQoJ++uknzZkzR82aNZPdbldsbKwKCgqq4UgAuBvhBkCt4+HhoYceekiJiYnas2eP7Ha7Dh48qJ49e5bZ/6qrrtLLL7+swsLCMs/efPrpp5o3b5769u0rSTp06JBOnDhRrccAwH2YUAygVvrLX/4iT09PLVy4UPfdd58mTpyol19+Wd99952++OILPffcc3r55ZclSePHj1dWVpb++te/6vPPP9fevXv16quvavfu3ZKkVq1a6dVXX9XOnTv13//+V0OHDj3n2R4AdRdnbgDUSl5eXho/fryefvppHThwQI0aNVJycrL279+vkJAQde7cWQ899JAkqWHDhvrggw90//33q2fPnvL09FTHjh3VvXt3SdKiRYt01113qXPnzoqMjNS0adN03333ufPwAFQjmzHGuLsIAACAqsJlKQAAYCmEGwAAYCmEGwAAYCmEGwAAYCmEGwAAYCmEGwAAYCmEGwAAYCmEGwAAYCmEGwAAYCmEGwAAYCmEGwAAYCmEGwAAYCn/Hzu2g3+giqgNAAAAAElFTkSuQmCC\n" }, "metadata": {} }, { "output_type": "stream", "name": "stdout", "text": [ "L'aire sous la courbe PR (AUC PR) est de : 0.99\n" ] } ], "metadata": { "id": "vcps_UPP7-5y", "colab": { "base_uri": "https://localhost:8080/", "height": 1000 }, "outputId": "df141f02-e090-41af-cde8-b9dc1a08d600" } }, { "cell_type": "markdown", "source": [ "Ci-dessous, nous importons deux autres modèles de détection de toxicité. Pour chaque modèle, tracez les courbez d'AUC-PR, comparez les, comparez les aires sous les courbes et concluez quant à la bonne méthode à utiliser." ], "metadata": { "id": "gl4ifVPk9KEy" } }, { "cell_type": "code", "execution_count": 7, "source": [ "from transformers import pipeline\n", "\n", "toxic_rob = pipeline(\n", " \"text-classification\", model=\"s-nlp/roberta_toxicity_classifier\", device=device\n", " )\n", "toxic_r4 = evaluate.load(\"toxicity\")\n", "\n", "test = 'Fuck you'\n", "print(toxic_r4.compute(predictions=[test], aggregation=None),toxic_rob(test))" ], "outputs": [ { "output_type": "stream", "name": "stderr", "text": [ "Some weights of the model checkpoint at s-nlp/roberta_toxicity_classifier were not used when initializing RobertaForSequenceClassification: ['roberta.pooler.dense.bias', 'roberta.pooler.dense.weight']\n", "- This IS expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", "- This IS NOT expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", "WARNING:evaluate_modules.metrics.evaluate-measurement--toxicity.2390290fa0bf6d78480143547c6b08f3d4f8805b249df8c7a8e80d0ce8e3778b.toxicity:Using default facebook/roberta-hate-speech-dynabench-r4-target checkpoint\n" ] }, { "output_type": "stream", "name": "stdout", "text": [ "{'toxicity': [0.0027796211652457714]} [{'label': 'toxic', 'score': 0.9995612502098083}]\n" ] } ], "metadata": { "id": "v5Z6lsn89IVt", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "15d66dc6-fbbd-462a-91c2-07e3f3b00018" } }, { "cell_type": "markdown", "source": [ "##Peut-on détécter la toxicité avec un LLM ?\n", "\n", "\n", "Dans cette partie , nous allons explorer l'application des modèles de langage, ou large Language Models (LLM), à la détection de la toxicité dans les textes. Pour cela exécuter la cellule suivante. Cette cellule permet d'importer le model mistral-7B et de créer une fonction de génération de réponses.\n" ], "metadata": { "id": "FUmJM-Lm-pnT" } }, { "cell_type": "code", "execution_count": 8, "source": [ "!pip install langchain\n", "!pip install langchain_community\n", "!pip install huggingface-cli\n", "!pip install llama-cpp-python\n", "!huggingface-cli download TheBloke/Mistral-7B-Instruct-v0.2-GGUF mistral-7b-instruct-v0.2.Q4_K_M.gguf --local-dir . --local-dir-use-symlinks False\n", "\n", "from langchain.llms import LlamaCpp\n", "from langchain.chains import LLMChain\n", "from langchain.prompts import PromptTemplate\n", "from langchain_core.output_parsers import StrOutputParser\n", "import os\n", "os.environ[\"CUDA_VISIBLE_DEVICES\"] = \"0\"\n", "\n", "def create_chain(model_path):\n", " llm = LlamaCpp(\n", " model_path=model_path, stop=[\"Question\"], max_tokens=10, temperature=0,\n", "\t\t\t\tn_ctx=8000, n_batch=1024, n_gpu_layers=-1,\n", " )\n", " template = \"\"\"Query:{query}\n", " {prompt}\n", " Answer yes/no:\n", " \"\"\"\n", " prompt = PromptTemplate(\n", " input_variables=[\"instruction\",'chat_history'], template=template\n", " )\n", "\n", " llm_chain = prompt |llm| StrOutputParser()\n", " return llm_chain\n", "\n", "llm = create_chain('mistral-7b-instruct-v0.2.Q4_K_M.gguf')\n", "\n", "def gen(query,prompt):\n", " response = llm.invoke({\"query\":query,\"prompt\":prompt})\n", " return response\n", "\n" ], "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Requirement already satisfied: langchain in /usr/local/lib/python3.10/dist-packages (0.2.2)\n", "Requirement already satisfied: PyYAML>=5.3 in /usr/local/lib/python3.10/dist-packages (from langchain) (6.0.1)\n", "Requirement already satisfied: SQLAlchemy<3,>=1.4 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.0.30)\n", "Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /usr/local/lib/python3.10/dist-packages (from langchain) (3.9.5)\n", "Requirement already satisfied: async-timeout<5.0.0,>=4.0.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (4.0.3)\n", "Requirement already satisfied: langchain-core<0.3.0,>=0.2.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (0.2.4)\n", "Requirement already satisfied: langchain-text-splitters<0.3.0,>=0.2.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (0.2.1)\n", "Requirement already satisfied: langsmith<0.2.0,>=0.1.17 in /usr/local/lib/python3.10/dist-packages (from langchain) (0.1.71)\n", "Requirement already satisfied: numpy<2,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain) (1.25.2)\n", "Requirement already satisfied: pydantic<3,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.7.2)\n", "Requirement already satisfied: requests<3,>=2 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.32.3)\n", "Requirement already satisfied: tenacity<9.0.0,>=8.1.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (8.3.0)\n", "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.3.1)\n", "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (23.2.0)\n", "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.4.1)\n", "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (6.0.5)\n", "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.9.4)\n", "Requirement already satisfied: jsonpatch<2.0,>=1.33 in /usr/local/lib/python3.10/dist-packages (from langchain-core<0.3.0,>=0.2.0->langchain) (1.33)\n", "Requirement already satisfied: packaging<24.0,>=23.2 in /usr/local/lib/python3.10/dist-packages (from langchain-core<0.3.0,>=0.2.0->langchain) (23.2)\n", "Requirement already satisfied: orjson<4.0.0,>=3.9.14 in /usr/local/lib/python3.10/dist-packages (from langsmith<0.2.0,>=0.1.17->langchain) (3.10.3)\n", "Requirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1->langchain) (0.7.0)\n", "Requirement already satisfied: pydantic-core==2.18.3 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1->langchain) (2.18.3)\n", "Requirement already satisfied: typing-extensions>=4.6.1 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1->langchain) (4.12.0)\n", "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (3.3.2)\n", "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (3.7)\n", "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (2.0.7)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (2024.2.2)\n", "Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy<3,>=1.4->langchain) (3.0.3)\n", "Requirement already satisfied: jsonpointer>=1.9 in /usr/local/lib/python3.10/dist-packages (from jsonpatch<2.0,>=1.33->langchain-core<0.3.0,>=0.2.0->langchain) (2.4)\n", "Requirement already satisfied: langchain_community in /usr/local/lib/python3.10/dist-packages (0.2.2)\n", "Requirement already satisfied: PyYAML>=5.3 in /usr/local/lib/python3.10/dist-packages (from langchain_community) (6.0.1)\n", "Requirement already satisfied: SQLAlchemy<3,>=1.4 in /usr/local/lib/python3.10/dist-packages (from langchain_community) (2.0.30)\n", "Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /usr/local/lib/python3.10/dist-packages (from langchain_community) (3.9.5)\n", "Requirement already satisfied: dataclasses-json<0.7,>=0.5.7 in /usr/local/lib/python3.10/dist-packages (from langchain_community) (0.6.6)\n", "Requirement already satisfied: langchain<0.3.0,>=0.2.0 in /usr/local/lib/python3.10/dist-packages (from langchain_community) (0.2.2)\n", "Requirement already satisfied: langchain-core<0.3.0,>=0.2.0 in /usr/local/lib/python3.10/dist-packages (from langchain_community) (0.2.4)\n", "Requirement already satisfied: langsmith<0.2.0,>=0.1.0 in /usr/local/lib/python3.10/dist-packages (from langchain_community) (0.1.71)\n", "Requirement already satisfied: numpy<2,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain_community) (1.25.2)\n", "Requirement already satisfied: requests<3,>=2 in /usr/local/lib/python3.10/dist-packages (from langchain_community) (2.32.3)\n", "Requirement already satisfied: tenacity<9.0.0,>=8.1.0 in /usr/local/lib/python3.10/dist-packages (from langchain_community) (8.3.0)\n", "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain_community) (1.3.1)\n", "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain_community) (23.2.0)\n", "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain_community) (1.4.1)\n", "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain_community) (6.0.5)\n", "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain_community) (1.9.4)\n", "Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain_community) (4.0.3)\n", "Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /usr/local/lib/python3.10/dist-packages (from dataclasses-json<0.7,>=0.5.7->langchain_community) (3.21.2)\n", "Requirement already satisfied: typing-inspect<1,>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from dataclasses-json<0.7,>=0.5.7->langchain_community) (0.9.0)\n", "Requirement already satisfied: langchain-text-splitters<0.3.0,>=0.2.0 in /usr/local/lib/python3.10/dist-packages (from langchain<0.3.0,>=0.2.0->langchain_community) (0.2.1)\n", "Requirement already satisfied: pydantic<3,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain<0.3.0,>=0.2.0->langchain_community) (2.7.2)\n", "Requirement already satisfied: jsonpatch<2.0,>=1.33 in /usr/local/lib/python3.10/dist-packages (from langchain-core<0.3.0,>=0.2.0->langchain_community) (1.33)\n", "Requirement already satisfied: packaging<24.0,>=23.2 in /usr/local/lib/python3.10/dist-packages (from langchain-core<0.3.0,>=0.2.0->langchain_community) (23.2)\n", "Requirement already satisfied: orjson<4.0.0,>=3.9.14 in /usr/local/lib/python3.10/dist-packages (from langsmith<0.2.0,>=0.1.0->langchain_community) (3.10.3)\n", "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain_community) (3.3.2)\n", "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain_community) (3.7)\n", "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain_community) (2.0.7)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain_community) (2024.2.2)\n", "Requirement already satisfied: typing-extensions>=4.6.0 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy<3,>=1.4->langchain_community) (4.12.0)\n", "Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy<3,>=1.4->langchain_community) (3.0.3)\n", "Requirement already satisfied: jsonpointer>=1.9 in /usr/local/lib/python3.10/dist-packages (from jsonpatch<2.0,>=1.33->langchain-core<0.3.0,>=0.2.0->langchain_community) (2.4)\n", "Requirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1->langchain<0.3.0,>=0.2.0->langchain_community) (0.7.0)\n", "Requirement already satisfied: pydantic-core==2.18.3 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1->langchain<0.3.0,>=0.2.0->langchain_community) (2.18.3)\n", "Requirement already satisfied: mypy-extensions>=0.3.0 in /usr/local/lib/python3.10/dist-packages (from typing-inspect<1,>=0.4.0->dataclasses-json<0.7,>=0.5.7->langchain_community) (1.0.0)\n", "Requirement already satisfied: huggingface-cli in /usr/local/lib/python3.10/dist-packages (0.1)\n", "Requirement already satisfied: llama-cpp-python in /usr/local/lib/python3.10/dist-packages (0.2.77)\n", "Requirement already satisfied: typing-extensions>=4.5.0 in /usr/local/lib/python3.10/dist-packages (from llama-cpp-python) (4.12.0)\n", "Requirement already satisfied: numpy>=1.20.0 in /usr/local/lib/python3.10/dist-packages (from llama-cpp-python) (1.25.2)\n", "Requirement already satisfied: diskcache>=5.6.1 in /usr/local/lib/python3.10/dist-packages (from llama-cpp-python) (5.6.3)\n", "Requirement already satisfied: jinja2>=2.11.3 in /usr/local/lib/python3.10/dist-packages (from llama-cpp-python) (3.1.4)\n", "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2>=2.11.3->llama-cpp-python) (2.1.5)\n", "/usr/local/lib/python3.10/dist-packages/huggingface_hub/commands/download.py:132: FutureWarning: Ignoring --local-dir-use-symlinks. Downloading to a local directory does not use symlinks anymore.\n", " warnings.warn(\n", "mistral-7b-instruct-v0.2.Q4_K_M.gguf\n" ] }, { "output_type": "stream", "name": "stderr", "text": [ "llama_model_loader: loaded meta data with 24 key-value pairs and 291 tensors from mistral-7b-instruct-v0.2.Q4_K_M.gguf (version GGUF V3 (latest))\n", "llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.\n", "llama_model_loader: - kv 0: general.architecture str = llama\n", "llama_model_loader: - kv 1: general.name str = mistralai_mistral-7b-instruct-v0.2\n", "llama_model_loader: - kv 2: llama.context_length u32 = 32768\n", "llama_model_loader: - kv 3: llama.embedding_length u32 = 4096\n", "llama_model_loader: - kv 4: llama.block_count u32 = 32\n", "llama_model_loader: - kv 5: llama.feed_forward_length u32 = 14336\n", "llama_model_loader: - kv 6: llama.rope.dimension_count u32 = 128\n", "llama_model_loader: - kv 7: llama.attention.head_count u32 = 32\n", "llama_model_loader: - kv 8: llama.attention.head_count_kv u32 = 8\n", "llama_model_loader: - kv 9: llama.attention.layer_norm_rms_epsilon f32 = 0.000010\n", "llama_model_loader: - kv 10: llama.rope.freq_base f32 = 1000000.000000\n", "llama_model_loader: - kv 11: general.file_type u32 = 15\n", "llama_model_loader: - kv 12: tokenizer.ggml.model str = llama\n", "llama_model_loader: - kv 13: tokenizer.ggml.tokens arr[str,32000] = [\"\", \"\", \"\", \"<0x00>\", \"<...\n", "llama_model_loader: - kv 14: tokenizer.ggml.scores arr[f32,32000] = [0.000000, 0.000000, 0.000000, 0.0000...\n", "llama_model_loader: - kv 15: tokenizer.ggml.token_type arr[i32,32000] = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ...\n", "llama_model_loader: - kv 16: tokenizer.ggml.bos_token_id u32 = 1\n", "llama_model_loader: - kv 17: tokenizer.ggml.eos_token_id u32 = 2\n", "llama_model_loader: - kv 18: tokenizer.ggml.unknown_token_id u32 = 0\n", "llama_model_loader: - kv 19: tokenizer.ggml.padding_token_id u32 = 0\n", "llama_model_loader: - kv 20: tokenizer.ggml.add_bos_token bool = true\n", "llama_model_loader: - kv 21: tokenizer.ggml.add_eos_token bool = false\n", "llama_model_loader: - kv 22: tokenizer.chat_template str = {{ bos_token }}{% for message in mess...\n", "llama_model_loader: - kv 23: general.quantization_version u32 = 2\n", "llama_model_loader: - type f32: 65 tensors\n", "llama_model_loader: - type q4_K: 193 tensors\n", "llama_model_loader: - type q6_K: 33 tensors\n", "llm_load_vocab: special tokens cache size = 259\n", "llm_load_vocab: token to piece cache size = 0.1637 MB\n", "llm_load_print_meta: format = GGUF V3 (latest)\n", "llm_load_print_meta: arch = llama\n", "llm_load_print_meta: vocab type = SPM\n", "llm_load_print_meta: n_vocab = 32000\n", "llm_load_print_meta: n_merges = 0\n", "llm_load_print_meta: n_ctx_train = 32768\n", "llm_load_print_meta: n_embd = 4096\n", "llm_load_print_meta: n_head = 32\n", "llm_load_print_meta: n_head_kv = 8\n", "llm_load_print_meta: n_layer = 32\n", "llm_load_print_meta: n_rot = 128\n", "llm_load_print_meta: n_embd_head_k = 128\n", "llm_load_print_meta: n_embd_head_v = 128\n", "llm_load_print_meta: n_gqa = 4\n", "llm_load_print_meta: n_embd_k_gqa = 1024\n", "llm_load_print_meta: n_embd_v_gqa = 1024\n", "llm_load_print_meta: f_norm_eps = 0.0e+00\n", "llm_load_print_meta: f_norm_rms_eps = 1.0e-05\n", "llm_load_print_meta: f_clamp_kqv = 0.0e+00\n", "llm_load_print_meta: f_max_alibi_bias = 0.0e+00\n", "llm_load_print_meta: f_logit_scale = 0.0e+00\n", "llm_load_print_meta: n_ff = 14336\n", "llm_load_print_meta: n_expert = 0\n", "llm_load_print_meta: n_expert_used = 0\n", "llm_load_print_meta: causal attn = 1\n", "llm_load_print_meta: pooling type = 0\n", "llm_load_print_meta: rope type = 0\n", "llm_load_print_meta: rope scaling = linear\n", "llm_load_print_meta: freq_base_train = 1000000.0\n", "llm_load_print_meta: freq_scale_train = 1\n", "llm_load_print_meta: n_yarn_orig_ctx = 32768\n", "llm_load_print_meta: rope_finetuned = unknown\n", "llm_load_print_meta: ssm_d_conv = 0\n", "llm_load_print_meta: ssm_d_inner = 0\n", "llm_load_print_meta: ssm_d_state = 0\n", "llm_load_print_meta: ssm_dt_rank = 0\n", "llm_load_print_meta: model type = 7B\n", "llm_load_print_meta: model ftype = Q4_K - Medium\n", "llm_load_print_meta: model params = 7.24 B\n", "llm_load_print_meta: model size = 4.07 GiB (4.83 BPW) \n", "llm_load_print_meta: general.name = mistralai_mistral-7b-instruct-v0.2\n", "llm_load_print_meta: BOS token = 1 ''\n", "llm_load_print_meta: EOS token = 2 ''\n", "llm_load_print_meta: UNK token = 0 ''\n", "llm_load_print_meta: PAD token = 0 ''\n", "llm_load_print_meta: LF token = 13 '<0x0A>'\n", "llm_load_tensors: ggml ctx size = 0.15 MiB\n", "llm_load_tensors: CPU buffer size = 4165.37 MiB\n", ".................................................................................................\n", "llama_new_context_with_model: n_ctx = 8000\n", "llama_new_context_with_model: n_batch = 1024\n", "llama_new_context_with_model: n_ubatch = 512\n", "llama_new_context_with_model: flash_attn = 0\n", "llama_new_context_with_model: freq_base = 10000.0\n", "llama_new_context_with_model: freq_scale = 1\n", "llama_kv_cache_init: CPU KV buffer size = 1000.00 MiB\n", "llama_new_context_with_model: KV self size = 1000.00 MiB, K (f16): 500.00 MiB, V (f16): 500.00 MiB\n", "llama_new_context_with_model: CPU output buffer size = 0.12 MiB\n", "llama_new_context_with_model: CPU compute buffer size = 547.63 MiB\n", "llama_new_context_with_model: graph nodes = 1030\n", "llama_new_context_with_model: graph splits = 1\n", "AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 1 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | \n", "Model metadata: {'tokenizer.chat_template': \"{{ bos_token }}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if message['role'] == 'user' %}{{ '[INST] ' + message['content'] + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ message['content'] + eos_token}}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %}\", 'tokenizer.ggml.add_eos_token': 'false', 'tokenizer.ggml.padding_token_id': '0', 'tokenizer.ggml.unknown_token_id': '0', 'tokenizer.ggml.eos_token_id': '2', 'general.architecture': 'llama', 'llama.rope.freq_base': '1000000.000000', 'llama.context_length': '32768', 'general.name': 'mistralai_mistral-7b-instruct-v0.2', 'tokenizer.ggml.add_bos_token': 'true', 'llama.embedding_length': '4096', 'llama.feed_forward_length': '14336', 'llama.attention.layer_norm_rms_epsilon': '0.000010', 'llama.rope.dimension_count': '128', 'tokenizer.ggml.bos_token_id': '1', 'llama.attention.head_count': '32', 'llama.block_count': '32', 'llama.attention.head_count_kv': '8', 'general.quantization_version': '2', 'tokenizer.ggml.model': 'llama', 'general.file_type': '15'}\n", "Available chat formats from metadata: chat_template.default\n", "Guessed chat format: mistral-instruct\n" ] } ], "metadata": { "id": "J6WwOxXPUG01", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "d3947be4-56cf-41ee-fc0e-dff0338404dc" } }, { "cell_type": "markdown", "source": [ "Cherchez un prompt permettant de détecter le langage toxique. Par exemple : \"Is it injuring ?\". Testez-le avec quelques exemples de JIGSAW.\n", "\n", "Le but est d'évaluer les performances de cette méthode. Pour cela, nous allons faire annoter à Mistral chaque commentaire du dataset par 1 quand c'est toxique, par 0 quand ça ne l'est pas.\n", "\n", "Par la suite, on calculera le F1-score.\n", "\n", "Dans un premier temps, créez une fonction permettant de récupérer dans la réponse le \"yes/no\" et renvoyer un booleen 1/0" ], "metadata": { "id": "EaiKXDWzWTVe" } }, { "cell_type": "code", "execution_count": 9, "source": [ "def response(reponse) :\n", " if reponse == \"yes\" :\n", " return 1\n", " else :\n", " return 0\n" ], "outputs": [], "metadata": { "id": "fqJkJ2fpfAER" } }, { "cell_type": "markdown", "source": [ "Faites annoter les commentaire du dataset par Mistral par 1 quand c'est toxique, par 0 quand ca ne l'est pas. Stockez les dans un array." ], "metadata": { "id": "Or9zdujOhSUx" } }, { "cell_type": "code", "execution_count": null, "source": [ "liste = []\n", "for comment in dataset['comment_text']:\n", " liste.append(response(gen(\"Is it injuring ?\", comment)))\n", "print(liste)" ], "outputs": [ { "output_type": "stream", "name": "stderr", "text": [ "\n", "llama_print_timings: load time = 265697.83 ms\n", "llama_print_timings: sample time = 6.23 ms / 10 runs ( 0.62 ms per token, 1606.17 tokens per second)\n", "llama_print_timings: prompt eval time = 265688.59 ms / 381 tokens ( 697.35 ms per token, 1.43 tokens per second)\n", "llama_print_timings: eval time = 10709.36 ms / 9 runs ( 1189.93 ms per token, 0.84 tokens per second)\n", "llama_print_timings: total time = 276436.20 ms / 390 tokens\n", "Llama.generate: prefix-match hit\n", "\n", "llama_print_timings: load time = 265697.83 ms\n", "llama_print_timings: sample time = 7.00 ms / 10 runs ( 0.70 ms per token, 1427.55 tokens per second)\n", "llama_print_timings: prompt eval time = 24281.34 ms / 33 tokens ( 735.80 ms per token, 1.36 tokens per second)\n", "llama_print_timings: eval time = 10438.11 ms / 9 runs ( 1159.79 ms per token, 0.86 tokens per second)\n", "llama_print_timings: total time = 34752.53 ms / 42 tokens\n", "Llama.generate: prefix-match hit\n", "\n", "llama_print_timings: load time = 265697.83 ms\n", "llama_print_timings: sample time = 6.92 ms / 10 runs ( 0.69 ms per token, 1445.50 tokens per second)\n", "llama_print_timings: prompt eval time = 8957.96 ms / 21 tokens ( 426.57 ms per token, 2.34 tokens per second)\n", "llama_print_timings: eval time = 7521.59 ms / 9 runs ( 835.73 ms per token, 1.20 tokens per second)\n", "llama_print_timings: total time = 16500.05 ms / 30 tokens\n", "Llama.generate: prefix-match hit\n", "\n", "llama_print_timings: load time = 265697.83 ms\n", "llama_print_timings: sample time = 6.08 ms / 10 runs ( 0.61 ms per token, 1643.39 tokens per second)\n", "llama_print_timings: prompt eval time = 16266.95 ms / 36 tokens ( 451.86 ms per token, 2.21 tokens per second)\n", "llama_print_timings: eval time = 7736.29 ms / 9 runs ( 859.59 ms per token, 1.16 tokens per second)\n", "llama_print_timings: total time = 24020.95 ms / 45 tokens\n", "Llama.generate: prefix-match hit\n", "\n", "llama_print_timings: load time = 265697.83 ms\n", "llama_print_timings: sample time = 6.68 ms / 10 runs ( 0.67 ms per token, 1497.01 tokens per second)\n", "llama_print_timings: prompt eval time = 34243.51 ms / 77 tokens ( 444.72 ms per token, 2.25 tokens per second)\n", "llama_print_timings: eval time = 7637.30 ms / 9 runs ( 848.59 ms per token, 1.18 tokens per second)\n", "llama_print_timings: total time = 41899.72 ms / 86 tokens\n", "Llama.generate: prefix-match hit\n", "\n", "llama_print_timings: load time = 265697.83 ms\n", "llama_print_timings: sample time = 7.36 ms / 10 runs ( 0.74 ms per token, 1358.70 tokens per second)\n", "llama_print_timings: prompt eval time = 32143.66 ms / 66 tokens ( 487.03 ms per token, 2.05 tokens per second)\n", "llama_print_timings: eval time = 9700.41 ms / 9 runs ( 1077.82 ms per token, 0.93 tokens per second)\n", "llama_print_timings: total time = 41864.61 ms / 75 tokens\n", "Llama.generate: prefix-match hit\n", "\n", "llama_print_timings: load time = 265697.83 ms\n", "llama_print_timings: sample time = 6.86 ms / 10 runs ( 0.69 ms per token, 1457.94 tokens per second)\n", "llama_print_timings: prompt eval time = 32399.87 ms / 69 tokens ( 469.56 ms per token, 2.13 tokens per second)\n", "llama_print_timings: eval time = 6854.11 ms / 9 runs ( 761.57 ms per token, 1.31 tokens per second)\n", "llama_print_timings: total time = 39270.58 ms / 78 tokens\n", "Llama.generate: prefix-match hit\n", "\n", "llama_print_timings: load time = 265697.83 ms\n", "llama_print_timings: sample time = 6.68 ms / 10 runs ( 0.67 ms per token, 1496.33 tokens per second)\n", "llama_print_timings: prompt eval time = 18102.71 ms / 30 tokens ( 603.42 ms per token, 1.66 tokens per second)\n", "llama_print_timings: eval time = 11512.61 ms / 9 runs ( 1279.18 ms per token, 0.78 tokens per second)\n", "llama_print_timings: total time = 29641.54 ms / 39 tokens\n", "Llama.generate: prefix-match hit\n" ] } ], "metadata": { "id": "kdLS9fmigCKR", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "8f9bb0b9-20d1-4d65-b703-c44a9b93c6e1" } }, { "cell_type": "markdown", "source": [ "Calculez le F1-Score et concluez sur l'efficacité de la méthode." ], "metadata": { "id": "UOl2Q138h1RG" } }, { "cell_type": "code", "execution_count": null, "source": [ "labels_test = [0 for _ in range(25)] + [1 for _ in range(25)]\n", "predicted_labels = liste\n", "from sklearn.metrics import f1_score\n", "f1 = f1_score(labels_test, predicted_labels)\n", "print(f1)\n" ], "outputs": [], "metadata": { "id": "yrJ60HN0hqcy" } }, { "cell_type": "markdown", "source": [ "Pour cloturer ce TP, créez un filtre de toxicité sur le RAG que vous avez implémenté durant les dernières séances avec la méthode de votre choix. Ce filtre devra filtrer le langage toxique de l'utilisateur." ], "metadata": { "id": "6kmocZvpiMNg" } }, { "cell_type": "code", "execution_count": null, "source": [ "from detoxify import Detoxify\n", "from transformers import AutoTokenizer, AutoModelForSeq2SeqLM\n", "\n", "# Initialisation du modèle Detoxify\n", "detoxify_model = Detoxify('original')\n", "\n", "# Initialisation du modèle RAG (assurez-vous de charger votre propre modèle RAG)\n", "rag_model_name = \"facebook/rag-token-base\"\n", "tokenizer = AutoTokenizer.from_pretrained(rag_model_name)\n", "model = AutoModelForSeq2SeqLM.from_pretrained(rag_model_name)\n", "\n", "# Fonction pour détecter la toxicité\n", "def is_toxic(text):\n", " predictions = detoxify_model.predict(text)\n", " return predictions['toxicity'] > 0.5 # Seuil de toxicité (peut être ajusté)\n", "\n", "# Fonction pour filtrer les messages\n", "def filter_message(message):\n", " if is_toxic(message):\n", " return \"Votre message a été détecté comme toxique et ne sera pas transmis.\"\n", " return message\n", "\n", "# Exemple d'utilisation avec RAG\n", "def rag_response(input_text):\n", " # Filtrage du message\n", " filtered_message = filter_message(input_text)\n", " if \"toxique\" in filtered_message:\n", " return filtered_message\n", "\n", " # Préparation de l'entrée pour le modèle RAG\n", " inputs = tokenizer([filtered_message], return_tensors='pt')\n", "\n", " # Génération de la réponse avec RAG\n", " outputs = model.generate(**inputs)\n", " response = tokenizer.decode(outputs[0], skip_special_tokens=True)\n", " return response\n", "\n", "# Exemple d'interaction\n", "user_message = input(\"Entrez votre message: \")\n", "response = rag_response(user_message)\n", "print(\"Réponse du système:\", response)\n" ], "outputs": [], "metadata": { "id": "0xgyS61riIHj" } }, { "cell_type": "markdown", "source": [ "#Bonus:\n", "\n", "Determinez avec la méthode de votre choix le seuil optimal pour filtrer un maximum de langage toxique avec Roberta." ], "metadata": { "id": "qoWo-wrpirwm" } }, { "cell_type": "code", "execution_count": null, "source": [], "outputs": [], "metadata": { "id": "e8pXYU2EjSYH" } } ], "metadata": { "kernelspec": { "name": "python3", "display_name": "Python 3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.12" }, "colab": { "provenance": [] }, "interpreter": { "hash": "1a85d0143f93c4f67f394d062034989b2c9fca60c1f84a050bdb6946630bae93" } }, "nbformat": 4, "nbformat_minor": 0 }