Improve model card with pipeline tag, library name, and Github link

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +18 -6
README.md CHANGED
@@ -1,9 +1,11 @@
1
  ---
2
  license: mit
 
 
3
  ---
4
 
5
  <div align="center">
6
-
7
  <div style="display: flex; justify-content: center; align-items: center; text-align: center;">
8
  <div style="display: flex; align-items: center; margin: auto;">
9
  <strong style="margin-left: 0px; font-size: 24px;">Bohdi: Heterogeneous LLM Fusion with Automatic Data Exploration</strong>
@@ -29,7 +31,7 @@ _<sup>†</sup> Corresponding Author_
29
 
30
  ### πŸ“„ Introduction
31
  Bohdi is a novel framework for heterogeneous Large Language Model (LLM) fusion that integrates the strengths of multiple source LLMs into a target LLM through adaptive knowledge exploration and automatic data generation. Unlike existing methods that rely on real data from limited domains and use fixed data allocation proportions, Bohdi dynamically adjusts sampling based on the target LLM's performance and generates data automatically through a hierarchical knowledge tree structure. This ensures comprehensive domain coverage and balanced capability enhancement without the need for real data. Our github page is [Bohdi](https://github.com/gjq100/Bohdi).
32
-
33
 
34
  ### ✨ Features
35
 
@@ -54,7 +56,7 @@ conda env create -f opencompass_env.yaml
54
  ```bash
55
  # The version we used: opencompass 0.3.4
56
  git clone https://github.com/open-compass/opencompass opencompass
57
- cd opencompass
58
  pip install -e .
59
  ```
60
 
@@ -64,7 +66,7 @@ pip install -e .
64
  To train the target LLM using Bohdi, follow these steps:
65
 
66
  1. **Prepare Source LLMs**: Ensure you have access to the source LLMs you want to fuse. If you want to follow our setup, please download the following models:
67
- ```Python
68
  # Source Models
69
  Qwen/Qwen2.5-14B-Instruct
70
  mistralai/Mistral-Small-24B-Instruct-2501
@@ -79,7 +81,7 @@ To train the target LLM using Bohdi, follow these steps:
79
  Please first configure the relevant paths in `run_bohdi.sh` according to your actual paths, and then run:
80
  ```bash
81
  source activate bohdi
82
- cd your project path
83
  bash run_bohdi.sh
84
  ```
85
 
@@ -87,10 +89,20 @@ Please first configure the relevant paths in `run_bohdi.sh` according to your ac
87
  We use <a href="https://github.com/open-compass/opencompass/tree/main">OpenCompass</a> for evaluation and perform inference based on VLLM. To evaluate your model, please configure the relevant paths in `eval_opencompass.sh` according to your actual paths, and then run:
88
  ```bash
89
  source activate opencompass
90
- cd your project path
91
  bash eval_opencompass.sh
92
  ```
93
 
 
 
 
 
 
 
 
 
 
 
94
  ### πŸ“š Citation
95
  ```
96
  @article{gao2025bohdi,
 
1
  ---
2
  license: mit
3
+ library_name: transformers
4
+ pipeline_tag: text-generation
5
  ---
6
 
7
  <div align="center">
8
+
9
  <div style="display: flex; justify-content: center; align-items: center; text-align: center;">
10
  <div style="display: flex; align-items: center; margin: auto;">
11
  <strong style="margin-left: 0px; font-size: 24px;">Bohdi: Heterogeneous LLM Fusion with Automatic Data Exploration</strong>
 
31
 
32
  ### πŸ“„ Introduction
33
  Bohdi is a novel framework for heterogeneous Large Language Model (LLM) fusion that integrates the strengths of multiple source LLMs into a target LLM through adaptive knowledge exploration and automatic data generation. Unlike existing methods that rely on real data from limited domains and use fixed data allocation proportions, Bohdi dynamically adjusts sampling based on the target LLM's performance and generates data automatically through a hierarchical knowledge tree structure. This ensures comprehensive domain coverage and balanced capability enhancement without the need for real data. Our github page is [Bohdi](https://github.com/gjq100/Bohdi).
34
+ We release model weights of the resulting LLMs which are finetuned with Bohdi.
35
 
36
  ### ✨ Features
37
 
 
56
  ```bash
57
  # The version we used: opencompass 0.3.4
58
  git clone https://github.com/open-compass/opencompass opencompass
59
+ cd [your project path]/opencompass
60
  pip install -e .
61
  ```
62
 
 
66
  To train the target LLM using Bohdi, follow these steps:
67
 
68
  1. **Prepare Source LLMs**: Ensure you have access to the source LLMs you want to fuse. If you want to follow our setup, please download the following models:
69
+ ```python
70
  # Source Models
71
  Qwen/Qwen2.5-14B-Instruct
72
  mistralai/Mistral-Small-24B-Instruct-2501
 
81
  Please first configure the relevant paths in `run_bohdi.sh` according to your actual paths, and then run:
82
  ```bash
83
  source activate bohdi
84
+ cd [your project path]/Bohdi
85
  bash run_bohdi.sh
86
  ```
87
 
 
89
  We use <a href="https://github.com/open-compass/opencompass/tree/main">OpenCompass</a> for evaluation and perform inference based on VLLM. To evaluate your model, please configure the relevant paths in `eval_opencompass.sh` according to your actual paths, and then run:
90
  ```bash
91
  source activate opencompass
92
+ cd [your project path]/opencompass
93
  bash eval_opencompass.sh
94
  ```
95
 
96
+ ### Direct Download and Usage
97
+ If you would like to directly use the distilled models for evaluation, our distilled models can be found directly on Hugging Face:
98
+ ```python
99
+ ChetKao/Bohdi-Llama-3.2-3B-Instruct
100
+ ChetKao/Bohdi-Llama-3.1-8B-Instruct
101
+ ChetKao/Bohdi-Qwen2.5-7B-Instruct
102
+ ChetKao/Bohdi-gemma-2-9b-it
103
+ ```
104
+
105
+
106
  ### πŸ“š Citation
107
  ```
108
  @article{gao2025bohdi,