【Training tutorial】MiniCPM-Llama3-V 2.5
输入“/”快速插入内容
【Training tutorial】MiniCPM-Llama3-V 2.5
【Training tutorial】MiniCPM-Llama3-V 2.5
用户3092024年8月26日修改
🙌
Suitable for: Those with a basic understanding of Linux and capable of reading Python scripts, modifying parameters, and handling data.
一、Model introduction
MiniCPM-Llama3-V 2.5 can process images with any aspect ratio and up to 1.8 million pixels (e.g., 1344x1344), achieving a 700+ score on OCRBench, surpassing proprietary models such as GPT-4o, GPT-4V-0409, Qwen-VL-Max and Gemini Pro. Based on recent user feedback, MiniCPM-Llama3-V 2.5 has now enhanced full-text OCR extraction, table-to-markdown conversion, and other high-utility capabilities, and has further strengthened its instruction-following and complex reasoning abilities, enhancing multimodal interaction experiences.
二、Model download：
1.
Install git lfs
a.
ubuntu install git lfs:
代码块
apt-get install git-lfs
b.
mac install git lfs
代码块
brew install git-lfs
c.
centos install git lfs
代码块
yum install git-lfs
2.
Download the model
Method 1: Use git command to download on huggingface
代码块
git clone https://huggingface.co/openbmb/MiniCPM-Llama3-V-2_5
Method 2: Use git command to download in modelscope
代码块
git clone https://modelscope.cn/models/OpenBMB/MiniCPM-Llama3-V-2_5
3.
Get the github code of minicpmv：
代码块
git clone https://github.com/OpenBMB/MiniCPM-V.git
三、Official code model training：
1.
Install dependency packages
代码块
cd MiniCPM-V
pip install -r requirements.txt
## deepspeed recommends downloading the latest version from the official website and pip install e. Installation
2.
Process the data set
代码块
[
{ "id": "0",
"conversations": [
{
"content": "<image>\nWho are they?",
"role": "user"
},
{
"content": "They're Kane and Gretzka from Bayern Munich.",
"role": "assistant"
},
{
"content": "What are they doing?",
"role": "user"
},
{
"content": "They are celebrating on the soccer field.",
"role": "assistant"
}
],
"image":
"/root/ld/ld_project/LLaMA-Factory/data/mllm_demo_data/1.jpg"
}
...以上是单个条数据，列表中可存在多个相同格式的数据
]