本地部署之后，AgentFabric上操作就会报错。preview_send_message user_agent = _state['user_agent'] KeyError: 'user_agent' about modelscope-agent HOT 11 OPEN

模型配的是:Qwen2-72B-Instruct-AWQ

"qwen2-72b-instruct-awq": {
"type": "openai",
"model": "qwen/Qwen2-72B-Instruct-AWQ",
"api_base": "http://localhost:8000/v1",
"is_chat": true,
"is_function_call": false
}

from modelscope-agent.

在环境变量定义DASHSCOPE_API_KEY了之后能往下跑了，但是还是会报错：
During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/data/work/modelscope-agent/apps/agentfabric/builder_core.py", line 60, in gen_response_and_process
for s in response:
File "/data/work/modelscope-agent/modelscope_agent/llm/dashscope.py", line 20, in stream_output
for trunk in response:
File "/data/work/modelscope-agent/modelscope_agent/llm/dashscope.py", line 142, in stat_last_call_token_info
if not chunk.usage.get('total_tokens'):
AttributeError: 'NoneType' object has no attribute 'get'
| step: | error: llm result is not valid
2024-07-07 14:59:58.225 - modelscope-agent - INFO - | message: frame | uuid: local_user | details: {'frame': '{'error': "llm result is not valid: 'NoneType' object has no attribute 'get'. Please reset and try again."}'} | step: | error:
2024-07-07 14:59:58.233 - modelscope-agent - INFO - | message: frame | uuid: local_user | details: {'frame': "{'error': 'llm result is not valid. parse RichConfig error. Please reset and try again.'}"} | step: | error:
2024-07-07 14:59:58.241 - modelscope-agent - ERROR - | message: | uuid: local_user | details: {} | step: | error: parse RichConfig error
No valid document. Return Empty Response.
2024-07-07 15:00:09.693 - modelscope-agent - INFO - | message: call dashscope generation api | uuid: | details: {'model': 'qwen-max', 'messages': [{'role': 'user', 'content': 'What is the weather like in Boston?'}], 'stop': [{'type': 'function', 'function': {'name': 'get_current_weather', 'description': 'Get the current weather in a given location.', 'parameters': {'type': 'object', 'properties': {'location': {'type': 'string', 'description': 'The city and state, e.g. San Francisco, CA'}, 'unit': {'type': 'string', 'enum': ['celsius', 'fahrenheit']}}, 'required': ['location']}}}], 'top_p': 0.8, 'result_format': 'message', 'stream': True} | step: | error:
2024-07-07 15:00:09.700 - modelscope-agent - INFO - | message: call dashscope generation api | uuid: | details: {'model': 'qwen-max', 'messages': [{'role': 'user', 'content': '<|im_start|>system\n\n\n# 知识库\n\nEmpty Response\n\n\n# 工具\n\n## 你拥有如下工具：\n\nimage_gen: image_gen API。AI绘画（图像生成）服务，输入文本描述和图像分辨率，返回根据文本信息绘制的图片URL。 输入参数: {"type": "object", "properties": {"text": {"type": "string", "description": "详细描述了希望生成的图像具有什么内容，例如人物、环境、动作等细节描述"}, "resolution": {"type": "string", "description": "格式是 数字数字，表示希望生成的图像的分辨率大小，选项有[10241024, 7201280, 1280720]"}, "lora_index": {"type": "string", "description": "如果用户要求使用lora的情况下，则使用该参数，没有指定的情况下默认为wanx1.4.5_textlora_huiben2_20240518"}}, "required": ["text", "resolution"]} Format the arguments as a JSON object.\n\n## 当你需要调用工具时，请在你的回复中穿插如下的工具调用命令，可以根据需求调用零次或多次：\n\n工具调用\nAction: 工具的名称，必须是[image_gen]之一\nAction Input: 工具的输入\nObservation: 工具返回的结果\nAnswer: 根据Observation总结本次工具调用返回的结果，如果结果中出现url，请使用如下格式展示出来：\n\n\n# 指令\n\n你扮演AI-Agent，\n你具有下列具体功能：\n下面你将开始扮演\n\n请注意：你具有图像和视频的展示能力，也具有运行代码的能力，不要在回复中说你做不到。\n<|im_end|>\n<|im_start|>user\n(你正在扮演。你可以使用工具：[image_gen]。请查看前面的知识库)你可以做什么？<|im_end|>\n<|im_start|>assistant\n'}], 'stop': ['Observation:', 'Observation:\n'], 'top_p': 0.8, 'result_format': 'message', 'stream': True} | step: | error:
2024-07-07 15:00:09.701 - modelscope-agent - INFO - | message: call llm 1 times output: <generator object stream_output at 0x79f841e9b060>

from modelscope-agent.

2024-07-07 15:44:18.618 - modelscope-agent - INFO - | message: using builder model qwen2-72b-instruct-awq | uuid: local_user | details: {} | step: | error:
2024-07-07 15:44:18.619 - modelscope-agent - INFO - | message: client url http://localhost:8000/v1, client key: EMPTY
2024-07-07 15:44:18,667 - modelscope - INFO - initiate model from /data/work/nlp_gte_sentence-embedding_chinese-base
2024-07-07 15:44:18,667 - modelscope - INFO - initiate model from location /data/work/nlp_gte_sentence-embedding_chinese-base.
2024-07-07 15:44:18,668 - modelscope - INFO - initialize model from /data/work/nlp_gte_sentence-embedding_chinese-base
2024-07-07 15:44:19,958 - modelscope - WARNING - No preprocessor field found in cfg.
2024-07-07 15:44:19,958 - modelscope - WARNING - No val key and type key found in preprocessor domain of configuration.json file.
2024-07-07 15:44:19,958 - modelscope - WARNING - Cannot find available config to build preprocessor at mode inference, current config: {'model_dir': '/data/work/nlp_gte_sentence-embedding_chinese-base'}. trying to build by task and model information.
2024-07-07 15:44:19,984 - modelscope - WARNING - No preprocessor field found in cfg.
2024-07-07 15:44:19,984 - modelscope - WARNING - No val key and type key found in preprocessor domain of configuration.json file.
2024-07-07 15:44:19,984 - modelscope - WARNING - Cannot find available config to build preprocessor at mode inference, current config: {'model_dir': '/data/work/nlp_gte_sentence-embedding_chinese-base', 'sequence_length': 128}. trying to build by task and model information.
2024-07-07 15:44:20.020 - modelscope-agent - INFO - | message: using model qwen2-72b-instruct-awq with tool Config (path: ./config/tool_config.json): {'image_gen': {'name': 'Wanx Image Generation', 'is_active': True, 'use': False, 'is_remote_tool': True}, 'code_interpreter': {'name': 'Code Interpreter', 'is_active': True, 'use': False, 'is_remote_tool': False, 'max_output': 2000}, 'web_browser': {'name': 'Web Browsing', 'is_active': True, 'use': False, 'max_browser_length': 2000}, 'amap_weather': {'name': '高德天气', 'is_active': True, 'use': False}, 'paraformer_asr': {'name': 'Paraformer语音识别', 'is_active': True, 'use': False, 'is_remote_tool': True}, 'sambert_tts': {'name': 'Sambert语音合成', 'is_active': True, 'use': False, 'is_remote_tool': True}, 'wordart_texture_generation': {'name': '艺术字纹理生成', 'is_active': True, 'use': False}, 'web_search': {'name': 'Web Searching', 'is_active': True, 'use': False, 'searcher': 'bing'}, 'qwen_vl': {'name': 'Qwen-VL识图', 'is_active': True, 'use': False}, 'style_repaint': {'name': '人物风格重绘', 'is_active': True, 'use': False}, 'image_enhancement': {'name': '追影-放大镜', 'is_active': True, 'use': False}, 'text-address': {'name': '地址解析', 'url': 'https://api-inference.modelscope.cn/api-inference/v1/models/damo/mgeo_geographic_elements_tagging_chinese_base', 'use': False, 'is_active': True, 'is_remote_tool': True}, 'text-ner': {'name': '命名实体识别', 'url': 'https://api-inference.modelscope.cn/api-inference/v1/models/damo/nlp_raner_named-entity-recognition_chinese-base-cmeee', 'use': False, 'is_active': False, 'is_remote_tool': True}, 'speech-generation': {'name': '语音生成', 'url': 'https://api-inference.modelscope.cn/api-inference/v1/models/damo/speech_sambert-hifigan_tts_zh-cn_16k', 'use': False, 'is_active': True, 'is_remote_tool': True}, 'video-generation': {'name': '视频生成', 'url': 'https://api-inference.modelscope.cn/api-inference/v1/models/damo/text-to-video-synthesis', 'use': False, 'is_active': True, 'is_remote_tool': True}, 'text-translation-en2zh': {'name': '英译中', 'url': 'https://api-inference.modelscope.cn/api-inference/v1/models/damo/nlp_csanmt_translation_en2zh', 'use': False, 'is_active': False, 'is_remote_tool': True}, 'text-translation-zh2en': {'name': '中译英', 'url': 'https://api-inference.modelscope.cn/api-inference/v1/models/damo/nlp_csanmt_translation_zh2en', 'use': False, 'is_active': False, 'is_remote_tool': True}} and function list [] | uuid: local_user | details: {'model_config': {'type': 'openai', 'model': 'qwen/Qwen2-72B-Instruct-AWQ', 'api_base': 'http://localhost:8000/v1', 'is_chat': True, 'is_function_call': False, 'generate_cfg': {'top_p': 0.5, 'stop': 'Observation'}}} | step: | error:
2024-07-07 15:44:20.020 - modelscope-agent - INFO - | message: client url http://localhost:8000/v1, client key: EMPTY
Neither documents nor cache_dir.
No valid document. Return Empty Response.
2024-07-07 15:44:23.533 - modelscope-agent - INFO - | message: call llm 1 times output: <generator object OpenAi._chat_stream at 0x71e1c22cfd10>
2024-07-07 15:44:23.534 - modelscope-agent - INFO - | message: call openai api, model: qwen/Qwen2-72B-Instruct-AWQ, messages: [{'role': 'system', 'content': "\n\n# 知识库\n\nEmpty Response\n\n\n# 指令\n\n你扮演AI-Agent，你的名字是agent-demo。you're an agent \n你具有下列具体功能：\n下面你将开始扮演agent-demo\n\n请注意：你具有图像和视频的展示能力，也具有运行代码的能力，不要在回复中说你做不到。\n"}, {'role': 'user', 'content': '(你正在扮演agent-demo。请查看前面的知识库)你有什么功能？'}], stop: ['Observation:', 'Observation:\n'], stream: True, args: {'user_token': ''}

from modelscope-agent.

what is your modelscope-agent version?

from modelscope-agent.

模型配的是:Qwen2-72B-Instruct-AWQ

"qwen2-72b-instruct-awq": { "type": "openai", "model": "qwen/Qwen2-72B-Instruct-AWQ", "api_base": "http://localhost:8000/v1", "is_chat": true, "is_function_call": false }

what is your llm_config?

from modelscope-agent.

what is your modelscope-agent version?

用的是本地部署哪个流程，因为有docker缺少新版本的引用，所以用的官方推荐的 git checkout 8deef6d 这个版本。

from modelscope-agent.

模型配的是:Qwen2-72B-Instruct-AWQ
"qwen2-72b-instruct-awq": { "type": "openai", "model": "qwen/Qwen2-72B-Instruct-AWQ", "api_base": "http://localhost:8000/v1", "is_chat": true, "is_function_call": false }

what is your llm_config?

官方本地话部署里面没写要单独修改llm_config啊。只是编辑 modelscope-agent/apps/agentfabric/config/model_config.json， 增加了本地模型的配置。顺便问一下，咱们框架跑起来好多地方的默认都会要 DASHSCOPE_API_KEY 。对于开源框架的使用来说感觉不太解耦。纯本地化部署有点跑不起来

from modelscope-agent.

直接用最新的master分支，然后 pip install -e . 安装一下试试。

what is your modelscope-agent version?

用的是本地部署哪个流程，因为有docker缺少新版本的引用，所以用的官方推荐的 git checkout 8deef6d 这个版本。

from modelscope-agent.

模型配的是:Qwen2-72B-Instruct-AWQ
"qwen2-72b-instruct-awq": { "type": "openai", "model": "qwen/Qwen2-72B-Instruct-AWQ", "api_base": "http://localhost:8000/v1", "is_chat": true, "is_function_call": false }

what is your llm_config?

官方本地话部署里面没写要单独修改llm_config啊。只是编辑 modelscope-agent/apps/agentfabric/config/model_config.json， 增加了本地模型的配置。顺便问一下，咱们框架跑起来好多地方的默认都会要 DASHSCOPE_API_KEY 。对于开源框架的使用来说感觉不太解耦。纯本地化部署有点跑不起来

嗯，主要就是基础模型调用的是qwen-max 需要dashscope。
那你在agentfabric里面有没有选择用你这个模型进行推理，而不是用默认的，默认的会请求qwen-max，就会依赖dashscope

from modelscope-agent.

直接用最新的master分支，然后 pip install -e . 安装一下试试。

what is your modelscope-agent version?

用的是本地部署哪个流程，因为有docker缺少新版本的引用，所以用的官方推荐的 git checkout 8deef6d 这个版本。

from modelscope-agent.

gradio==4.36.1
升级一下。

from modelscope-agent.