Improve model card with pipeline tag, library name, and Github link
#1
by
nielsr HF Staff - opened
README.md
CHANGED
|
@@ -1,9 +1,11 @@
|
|
| 1 |
---
|
| 2 |
license: mit
|
|
|
|
|
|
|
| 3 |
---
|
| 4 |
|
| 5 |
<div align="center">
|
| 6 |
-
|
| 7 |
<div style="display: flex; justify-content: center; align-items: center; text-align: center;">
|
| 8 |
<div style="display: flex; align-items: center; margin: auto;">
|
| 9 |
<strong style="margin-left: 0px; font-size: 24px;">Bohdi: Heterogeneous LLM Fusion with Automatic Data Exploration</strong>
|
|
@@ -29,7 +31,7 @@ _<sup>β </sup> Corresponding Author_
|
|
| 29 |
|
| 30 |
### π Introduction
|
| 31 |
Bohdi is a novel framework for heterogeneous Large Language Model (LLM) fusion that integrates the strengths of multiple source LLMs into a target LLM through adaptive knowledge exploration and automatic data generation. Unlike existing methods that rely on real data from limited domains and use fixed data allocation proportions, Bohdi dynamically adjusts sampling based on the target LLM's performance and generates data automatically through a hierarchical knowledge tree structure. This ensures comprehensive domain coverage and balanced capability enhancement without the need for real data. Our github page is [Bohdi](https://github.com/gjq100/Bohdi).
|
| 32 |
-
|
| 33 |
|
| 34 |
### β¨ Features
|
| 35 |
|
|
@@ -54,7 +56,7 @@ conda env create -f opencompass_env.yaml
|
|
| 54 |
```bash
|
| 55 |
# The version we used: opencompass 0.3.4
|
| 56 |
git clone https://github.com/open-compass/opencompass opencompass
|
| 57 |
-
cd opencompass
|
| 58 |
pip install -e .
|
| 59 |
```
|
| 60 |
|
|
@@ -64,7 +66,7 @@ pip install -e .
|
|
| 64 |
To train the target LLM using Bohdi, follow these steps:
|
| 65 |
|
| 66 |
1. **Prepare Source LLMs**: Ensure you have access to the source LLMs you want to fuse. If you want to follow our setup, please download the following models:
|
| 67 |
-
```
|
| 68 |
# Source Models
|
| 69 |
Qwen/Qwen2.5-14B-Instruct
|
| 70 |
mistralai/Mistral-Small-24B-Instruct-2501
|
|
@@ -79,7 +81,7 @@ To train the target LLM using Bohdi, follow these steps:
|
|
| 79 |
Please first configure the relevant paths in `run_bohdi.sh` according to your actual paths, and then run:
|
| 80 |
```bash
|
| 81 |
source activate bohdi
|
| 82 |
-
cd your project path
|
| 83 |
bash run_bohdi.sh
|
| 84 |
```
|
| 85 |
|
|
@@ -87,10 +89,20 @@ Please first configure the relevant paths in `run_bohdi.sh` according to your ac
|
|
| 87 |
We use <a href="https://github.com/open-compass/opencompass/tree/main">OpenCompass</a> for evaluation and perform inference based on VLLM. To evaluate your model, please configure the relevant paths in `eval_opencompass.sh` according to your actual paths, and then run:
|
| 88 |
```bash
|
| 89 |
source activate opencompass
|
| 90 |
-
cd your project path
|
| 91 |
bash eval_opencompass.sh
|
| 92 |
```
|
| 93 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 94 |
### π Citation
|
| 95 |
```
|
| 96 |
@article{gao2025bohdi,
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
+
library_name: transformers
|
| 4 |
+
pipeline_tag: text-generation
|
| 5 |
---
|
| 6 |
|
| 7 |
<div align="center">
|
| 8 |
+
|
| 9 |
<div style="display: flex; justify-content: center; align-items: center; text-align: center;">
|
| 10 |
<div style="display: flex; align-items: center; margin: auto;">
|
| 11 |
<strong style="margin-left: 0px; font-size: 24px;">Bohdi: Heterogeneous LLM Fusion with Automatic Data Exploration</strong>
|
|
|
|
| 31 |
|
| 32 |
### π Introduction
|
| 33 |
Bohdi is a novel framework for heterogeneous Large Language Model (LLM) fusion that integrates the strengths of multiple source LLMs into a target LLM through adaptive knowledge exploration and automatic data generation. Unlike existing methods that rely on real data from limited domains and use fixed data allocation proportions, Bohdi dynamically adjusts sampling based on the target LLM's performance and generates data automatically through a hierarchical knowledge tree structure. This ensures comprehensive domain coverage and balanced capability enhancement without the need for real data. Our github page is [Bohdi](https://github.com/gjq100/Bohdi).
|
| 34 |
+
We release model weights of the resulting LLMs which are finetuned with Bohdi.
|
| 35 |
|
| 36 |
### β¨ Features
|
| 37 |
|
|
|
|
| 56 |
```bash
|
| 57 |
# The version we used: opencompass 0.3.4
|
| 58 |
git clone https://github.com/open-compass/opencompass opencompass
|
| 59 |
+
cd [your project path]/opencompass
|
| 60 |
pip install -e .
|
| 61 |
```
|
| 62 |
|
|
|
|
| 66 |
To train the target LLM using Bohdi, follow these steps:
|
| 67 |
|
| 68 |
1. **Prepare Source LLMs**: Ensure you have access to the source LLMs you want to fuse. If you want to follow our setup, please download the following models:
|
| 69 |
+
```python
|
| 70 |
# Source Models
|
| 71 |
Qwen/Qwen2.5-14B-Instruct
|
| 72 |
mistralai/Mistral-Small-24B-Instruct-2501
|
|
|
|
| 81 |
Please first configure the relevant paths in `run_bohdi.sh` according to your actual paths, and then run:
|
| 82 |
```bash
|
| 83 |
source activate bohdi
|
| 84 |
+
cd [your project path]/Bohdi
|
| 85 |
bash run_bohdi.sh
|
| 86 |
```
|
| 87 |
|
|
|
|
| 89 |
We use <a href="https://github.com/open-compass/opencompass/tree/main">OpenCompass</a> for evaluation and perform inference based on VLLM. To evaluate your model, please configure the relevant paths in `eval_opencompass.sh` according to your actual paths, and then run:
|
| 90 |
```bash
|
| 91 |
source activate opencompass
|
| 92 |
+
cd [your project path]/opencompass
|
| 93 |
bash eval_opencompass.sh
|
| 94 |
```
|
| 95 |
|
| 96 |
+
### Direct Download and Usage
|
| 97 |
+
If you would like to directly use the distilled models for evaluation, our distilled models can be found directly on Hugging Face:
|
| 98 |
+
```python
|
| 99 |
+
ChetKao/Bohdi-Llama-3.2-3B-Instruct
|
| 100 |
+
ChetKao/Bohdi-Llama-3.1-8B-Instruct
|
| 101 |
+
ChetKao/Bohdi-Qwen2.5-7B-Instruct
|
| 102 |
+
ChetKao/Bohdi-gemma-2-9b-it
|
| 103 |
+
```
|
| 104 |
+
|
| 105 |
+
|
| 106 |
### π Citation
|
| 107 |
```
|
| 108 |
@article{gao2025bohdi,
|