【Deployment Tutorial】MiniCPM-Llama3-V 2.5
输入“/”快速插入内容
【Deployment Tutorial】MiniCPM-Llama3-V 2.5
【Deployment Tutorial】MiniCPM-Llama3-V 2.5
用户3092024年8月26日修改
🙌
Bilibili accompanying video: https://space.bilibili.com/493282299/channel/collectiondetail?sid=3433564
Suitable for: Those who can modify simple parameters based on Python scripts and use the most basic Bash language.
一、Python single GPU code inference
Equipment requirements: single card 24g 20 series or above nvidia graphics card
1.
Get the github code of MiniCPM-V:：
代码块
git clone https://github.com/OpenBMB/MiniCPM-V
2.
Create the following python file in the path of MiniCPM-V, and make sure there is a chat.py file in the same path.
代码块
from chat import MiniCPMVChat, img2base64
import torch
import json
torch.manual_seed(0)
chat_model = MiniCPMVChat('openbmb/MiniCPM-Llama3-V-2_5') # or local model path
im_64 = img2base64('/root/ld/ld_project/MiniCPM-V/assets/zhihu.webp') # image path
# First round chat
msgs = [{"role": "user", "content": "Tell me the model of this aircraft."}] # content写入user的提问
inputs = {"image": im_64, "question": json.dumps(msgs)}
answer = chat_model.chat(inputs)
print(answer)
3.
You can modify the above code by yourself to generate text for multiple images.
二、LMDeploy batch inference
Equipment requirements: single card 24g 20 series or above nvidia graphics card, or more than 2 12g 20 series or above graphics cards
1.
Install deploy, be careful not to use source code compilation.
代码块
pip install deploy
2.
并发推理代码如下：
代码块
from lmdeploy import pipeline, TurbomindEngineConfig
from lmdeploy.vl import load_image
# MiniCPM-Llama3-V 2.5 can be replaced with a local path
#session_len=2048 represents the context length
#tp=8 represents the number of graphics cards used, which must be 2**n, such as 1, 2, 4, 8
pipe = pipeline('MiniCPM-Llama3-V 2.5',
backend_config=TurbomindEngineConfig(session_len=2048,tp=8),)
# image url or local path
image_urls=[
"/root/ld/ld_project/MiniCPM-V/assets/minicpmv2-cases.png",
"/root/ld/ld_project/MiniCPM-V/assets/llavabench_compare_phi3.png",
"/root/ld/ld_project/MiniCPM-V/assets/MiniCPM-Llama3-V-2.5-peformance.png",
"/root/ld/ld_project/MiniCPM-V/assets/zhihu.webp",
"/root/ld/ld_project/MiniCPM-V/assets/thunlp.png"
]
prompts = [('describe this image', load_image(img_url)) for img_url in image_urls]
response = pipe(prompts)
print([i.text for i in response])