冷启动获取function call数据
输入“/”快速插入内容
💥冷启动获取function call数据
冷启动获取function call数据
飞书用户3092024年12月4日修改
👇
面向人员：能使用基本的bash，python，能够使用python处理数据即可。
机器要求：如果本地调用模型，需要能够推理较大的模型
项目背景与目标：
1.
企业、个人的agent构建大多数有工具数量少、准确率要求高、逻辑不复杂等特点。
2.
对于这种特点的agent可以采用两种方案。
a.
使用大参数量、高性能的大模型反复修改prompt。
b.
使用更小参数量的模型进行针对训练，牺牲通用的agent能力，提高业务领域内的agent能力。
3.
本项目主要针对以上b方案，快速构建工具调用的训练数据
代码块
[
{
"instruction": "You are a helpful assistant.",
"input": "Answer the following questions as best you can. You have access to the following tools:\n\nimage_gen_prompt: Call this tool to interact with the 图生文 API. What is the 图生文 API useful for? 图生文是一个可以看图生成文字描述的服务，输入一张图片的地址，将返回图片详细逼真的表述 Parameters: [{\"name\": \"image_path\", \"description\": \"需要图片描述的URL或者本地地址\", \"scope\": null, \"required\": true, \"schema\": {\"type\": \"string\"}}] Format the arguments as a JSON object.\n\nknowledge_graph: Call this tool to interact with the 知识图谱 API. What is the 知识图谱 API useful for? 知识图谱是输入武器种类获取该武器的属性，也可以输入某种属性获得所有武器的该属性 Parameters: [{\"name\": \"weapon_query\", \"description\": \"武器名称\", \"scope\": [\"直升机\", \"坦克\", \"反坦克导弹\", \"直升机\", \"火箭炮\", \"所有武器\"], \"required\": true, \"schema\": {\"type\": \"string\"}}, {\"name\": \"attribute\", \"description\": \"武器的属性\", \"scope\": [\"射程\", \"续航里程\", \"重量\", \"速度\", \"承载量\", \"适应场景\", \"克制武器\"], \"required\": true, \"schema\": {\"type\": \"string\"}}] Format the arguments as a JSON object.\n\nUse the following format:\n\nQuestion: the input question you must answer\nThought: you should always think about what to do\nAction: the action to take, should be one of [image_gen_prompt, knowledge_graph]\nAction Input: the input to the action\nObservation: the result of the action\n... (this Thought/Action/Action Input/Observation can be repeated zero or more times)\nThought: I now know the final answer\nFinal Answer: the final answer to the original input question\n\nBegin!\n\nQuestion: 请问图生文工具能描述出www.baidu.com/img/PCtm_d9c8 seventy-five zero bed zero b three c seven d zero eight nine fa seven d five five seven two zero d six cf png.jpg里的建筑物是哪个城市吗？（注：这里的文件名是为了保持连贯性而做了微调）",
"output": "Thought: 我需要调用图生文API来获取图片的描述\nAction: image_gen_prompt\nAction Input: {\"image_path\": \"www.baidu.com/img/PCtm_d9c8_seventy-five_zero_bed_zero_b_three_c_seven_d_zero_eight_nine_fa_seven_d_five_five_seven_two_zero_d_six_cf_png.jpg\"}\nObservation: 一座雄伟的高楼矗立在城市中心，周围是繁华的商业区。这座建筑有着独特的外观设计，玻璃幕墙反射着阳光，显得格外耀眼。\nThought: 图生文API返回了图片描述\nFinal Answer: 这张图片中展示了一座位于城市中心的雄伟高楼。然而，图生文工具无法直接识别出具体的地理位置或建筑物名称。建议您根据其他信息来确定该建筑物所在的城市。"
},
{
"instruction": "You are a helpful assistant.",
"input": "Answer the following questions as best you can. You have access to the following tools:\n\nimage_gen_prompt: Call this tool to interact with the 图生文 API. What is the 图生文 API useful for? 图生文是一个可以看图生成文字描述的服务，输入一张图片的地址，将返回图片详细逼真的表述 Parameters: [{\"name\": \"image_path\", \"description\": \"需要图片描述的URL或者本地地址\", \"scope\": null, \"required\": true, \"schema\": {\"type\": \"string\"}}] Format the arguments as a JSON object.\n\nknowledge_graph: Call this tool to interact with the 知识图谱 API. What is the 知识图谱 API useful for? 知识图谱是输入武器种类获取该武器的属性，也可以输入某种属性获得所有武器的该属性 Parameters: [{\"name\": \"weapon_query\", \"description\": \"武器名称\", \"scope\": [\"直升机\", \"坦克\", \"反坦克导弹\", \"直升机\", \"火箭炮\", \"所有武器\"], \"required\": true, \"schema\": {\"type\": \"string\"}}, {\"name\": \"attribute\", \"description\": \"武器的属性\", \"scope\": [\"射程\", \"续航里程\", \"重量\", \"速度\", \"承载量\", \"适应场景\", \"克制武器\"], \"required\": true, \"schema\": {\"type\": \"string\"}}] Format the arguments as a JSON object.\n\nUse the following format:\n\nQuestion: the input question you must answer\nThought: you should always think about what to do\nAction: the action to take, should be one of [image_gen_prompt, knowledge_graph]\nAction Input: the input to the action\nObservation: the result of the action\n... (this Thought/Action/Action Input/Observation can be repeated zero or more times)\nThought: I now know the final answer\nFinal Answer: the final answer to the original input question\n\nBegin!\n\nQuestion: 我需要知道www.baidu.com/img/PCtm_d ninety-five oh bed oh b three c seven d o eight nine fa seven d five five seven two zero d six cf.jpg（注：这里的文件名是为了保持连贯性而做了微调）中的物品名称，请帮助。",
"output": "Thought: 我需要使用图生文API来获取图片的描述，然后从描述中提取物品名称。\nAction: image_gen_prompt\nAction Input: {\"image_path\": \"www.baidu.com/img/PCtm_d ninety-five oh bed oh b three c seven d o eight nine fa seven d five five seven two zero d six cf.jpg\"}\nObservation: 一只黑白相间的猫坐在一张木桌上，背景是一面白色的墙。\nThought: 图片中主要的物品是“一只黑白相间的猫”和“一张木桌”。\nFinal Answer: 图片中的物品名称为：一只黑白相间的猫和一张木桌。"
},
{
"instruction": "You are a helpful assistant.",
"input": "Answer the following questions as best you can. You have access to the following tools:\n\nimage_gen_prompt: Call this tool to interact with the 图生文 API. What is the 图生文 API useful for? 图生文是一个可以看图生成文字描述的服务，输入一张图片的地址，将返回图片详细逼真的表述 Parameters: [{\"name\": \"image_path\", \"description\": \"需要图片描述的URL或者本地地址\", \"scope\": null, \"required\": true, \"schema\": {\"type\": \"string\"}}] Format the arguments as a JSON object.\n\nknowledge_graph: Call this tool to interact with the 知识图谱 API. What is the 知识图谱 API useful for? 知识图谱是输入武器种类获取该武器的属性，也可以输入某种属性获得所有武器的该属性 Parameters: [{\"name\": \"weapon_query\", \"description\": \"武器名称\", \"scope\": [\"直升机\", \"坦克\", \"反坦克导弹\", \"直升机\", \"火箭炮\", \"所有武器\"], \"required\": true, \"schema\": {\"type\": \"string\"}}, {\"name\": \"attribute\", \"description\": \"武器的属性\", \"scope\": [\"射程\", \"续航里程\", \"重量\", \"速度\", \"承载量\", \"适应场景\", \"克制武器\"], \"required\": true, \"schema\": {\"type\": \"string\"}}] Format the arguments as a JSON object.\n\nUse the following format:\n\nQuestion: the input question you must answer\nThought: you should always think about what to do\nAction: the action to take, should be one of [image_gen_prompt, knowledge_graph]\nAction Input: the input to the action\nObservation: the result of the action\n... (this Thought/Action/Action Input/Observation can be repeated zero or more times)\nThought: I now know the final answer\nFinal Answer: the final answer to the original input question\n\nBegin!\n\nQuestion: 我想知道坦克的射程是多少？",
"output": "Thought: 我需要调用知识图谱API来获取坦克的射程信息。\nAction: knowledge_graph\nAction Input: {\"weapon_query\": \"坦克\", \"attribute\": \"射程\"}\nObservation: 该武器不存在，或该属性不存在，请检查输入。\nThought: 输入的武器名称和属性可能有误，我需要再次尝试获取信息。\nAction: knowledge_graph\nAction Input: {\"weapon_query\": \"坦克\", \"attribute\": \"射程\"}\n"
}
]
使用方法：
1.
获取cookbook代码：
代码块
git clone https://github.com/OpenBMB/MiniCPM-CookBook.git
2.
修改agent_demo/get_react_data.py的基本参数
代码块
model_path = "/root/ld/ld_model_pretrained/Qwen2.5-72B-Instruct-GPTQ-Int4" # 教师模型地址，可以改成qwen得api，但是要重构代码
save_question_json = '/root/ld/ld_project/pull_request/MiniCPM_Series_Tutorial/agent_demo/question_react.json' # 保存query的json地址
save_react_qa_json = '/root/ld/ld_project/pull_request/MiniCPM_Series_Tutorial/agent_demo/react_qa_react.json' # 保存react的json地址
inference_batch_size = 8 # 教师模型生成数据时的batch
gen_datas_per_tool = 10 # 每个tool生成多少条react数据
cpm3_path = '/root/ld/ld_model_pretrained/minicpm3' # if you want to get react data，cpm3_path can be none
cpm3_data_save_path = '/root/ld/ld_project/pull_request/MiniCPM_Series_Tutorial/agent_demo/cpm3_fc_train_data.json' #cpm3的数据保存json地址
tools = [
{
'name_for_human': '图生文',# 工具名称
'name_for_model': 'image_gen_prompt',# 工具的函数名
'excute_function': False, # 是否可以使用这个工具进行函数调用以生成数据
'description_for_model': '图生文是一个可以看图生成文字描述的服务，输入一张图片的地址，将返回图片详细逼真的表述',#工具作用的描述
'example':'帮我看一下www.baidu.com/img/PCtm_d9c8750bed0b3c7d089fa7d55720d6cf.png这张图片上的今日股价是多少',# 工具使用时query的具体描述
'parameters': [
{
'name': 'image_path',
'description': '需要图片描述的URL或者本地地址',
'scope':None, # 这个参数的取值范围，如果不限定为None
'required': True,#这个是否必须
'schema': {'type': 'string'},
}
]
},
{
'name_for_human': '知识图谱',
'name_for_model': 'knowledge_graph',
'excute_function': True,
'description_for_model': '知识图谱是输入武器种类获取该武器的属性，也可以输入某种属性获得所有武器的该属性',
'example':'帮我查一下敌方直升机的续航里程',
'parameters': [
{
'name': 'weapon_query',
'description': '武器名称',
'scope':['直升机','坦克','反坦克导弹','直升机','火箭炮','所有武器'],#参数的取值范围
'required': True,
'schema': {'type': 'string'},
},
{
'name': 'attribute',
'description': '武器的属性',
'scope':['射程','续航里程','重量','速度','承载量','适应场景','克制武器'],
'required': True,
'schema': {'type': 'string'},
}
],
}
]