zho-zho-zho / comfyui-gemini Goto Github PK

View Code? Open in Web Editor NEW

546.0 5.0 49.0 158 KB

Using Gemini in ComfyUI

License: GNU General Public License v3.0

Python 92.64% JavaScript 7.36%

comfyui stable-diffusion gemini-pro google-gemini

comfyui-gemini's Introduction

👋 Hi，我是 Zho ！

中文 | English

🏛️ 职业：

职业建筑师 | 公共建筑
罗杰斯✅福斯特❌ | 盖里✅扎哈❌ | 贝聿铭✅路易斯康❌

❣️ 兴趣：

AIGC
建筑设计&理论 | Archigram ✅ 结构主义✅
艺术 | 蔡国强✅
工业设计、交互设计

🤖 焦点：

目前独立从事 AIGC 研究与实践 | 图像✅ 视频✅ 3D✅ LLM✅
论文解读 | 发展梳理 | 视频制作

🏗️重心：

Stable Diffusion 生态 + ComfyUI：
- ComfyPark 社区筹备中 | 安装与部署 | 系统教程 | 模块+模组 | 工作流设计 | 定制化UI
- 编写、汉化或优化自定义节点
- 设计工作流
- 完成2000+节点测试
- 未来推出ComfyUI的一个优化版本
AIGC 理论发展与梳理

🌟 技能：

ChatGPT（哈哈哈）
Python

🔥 热门：

ComfyUI Workflows ZHO: 我的所有工作流（目前共 19 类 46 项）
简体中文版 ComfyUI: 主库，完整工作内容和时间线
ComfyUI Assistant：GPTs帮助创建或解释自定义节点或工作流，上榜 GPTs 全球对话数据第 165 名（12.29数据），目前已达1.7k！
ComfyUI Portrait Master 简体中文版: 人物肖像提示词生成模块，选择永远比填空更适合人类！
Stable Diffusion 3 API ComfyUI: 通过 API 将 Stable Diffusion 3 引入 ComfyUI!
Phi-3-mini in ComfyUI: 将闪电般速度的 Phi-3-mini-4k 模型引入到 ComfyUI 中，生成/补全提示词，解数学题，畅聊人生！
Gemini in ComfyUI: 将 Gemini Pro 双模型引入到 ComfyUI 中，生成提示词、描述图像，也可与它畅聊人生！
QWen-VL in ComfyUI: 将目前最好的开源视觉模型 QWen-VL-Plus & Max 引入到 ComfyUI 中，超强细节、文字识别能力，支持单/多轮对话双模式！
ComfyUI ArtGallery | Prompt Visualization: 将近千种艺术提示词分类可视化，只需选择喜欢的图像 + 滑动选择权重，节点就会自动输出提示词，自由的在艺术世界翱翔吧！
APISR in ComfyUI:专门用于动漫的超分模型，包含 2x 和 4x 双模型，速度飞快，效果很好
BiRefNet in ComfyUI:目前最好的开源可商用背景抠除模型，同时支持图像和视频处理
BRIA_RMBG 1.4 in ComfyUI：BRIA 开发的目前最好的背景去除模型，已支持批量处理（可去除视频背景）
ComfyUI YoloWorld-EfficientSAM：利用全新的 YOLO-World 与 EfficientSAM 实现高效的对象检测 + 分割，支持选择指定蒙版单独输出，同时支持图像和视频
DepthFM IN ComfyUI：CompVis 推出的高效 + 多功能深度估计模型，同时支持图像和视频深度估计
ComfyUI TripoSR ZHO：TripoSR 在 ComfyUI 中的优化版，自带背景去除功能，修复原版黑色背景块bug，优化为按文件名保存 3D 模型
ComfyUI PhotoMaker: 人物特征迁移与融合，支持单/多张参考图，轻松保持角色一致性！
ComfyUI InstantID: 仅需一张图就可实现高质量的角色保持！多种风格随心变！
ComfyUI Text_+ Image：将文字转化为图像的 ComfyUI 节点，还可配合视频生成字幕或帧计数
ComfyUI Selector + CoInput Multilingual：CoInput 综合输入：高效整合 + 利于复用 + 简化界面 + 中英双语 + 避免efficiency-nodes重复加载的缺陷
ComfyUI SegMoE：三个臭皮匠顶个诸葛亮！ SD 的 MoE 专家混合模型框架！
ComfyUI PixArt-alpha Diffusers：对 PixArt-alpha-Diffusers实现，填补原本支持插件失效的空白
Stable Video Diffusion in ComfyUI：我为 SVD 设计的工作流和节点，文生视频 + 图生视频，更好的交互！
ComfyUI I2VGenXL：目前动效自然程度和光影最好的高清视频模型
ComfyUI Differential_Diffusion Workflows：我设计的 Differential Diffusion 工作流

📬 联系我：

邮箱：[email protected]
QQ 群：839821928

🔗 社交媒体：

个人页：-Zho-
Bilibili：我的B站主页
X（Twitter）：我的Twitter
小红书：我的小红书主页

📅 未来计划：

ComfyPark社区
发布 ComfyUI 2400+ 节点测试报告
开放精选期刊栏目（博客+订阅）
推出 ComfyUI 的特殊优化版本

💡 支持我：

B站：B站充电
爱发电：为我充电

comfyui-gemini's People

Contributors

Stargazers

Watchers

comfyui-gemini's Issues

Gemini已经搭建好可用，但还是红字报错

Error occurred when executing Gemini_API_S_Zho:

The response.parts quick accessor only works for a single candidate, but none were returned. Check the response.prompt_feedback to see if the prompt was blocked.

Connection failed : Time out

Error occurred when executing Gemini_API_S_Zho:

HTTPSConnectionPool(host='generativelanguage.googleapis.com', port=443): Max retries exceeded with url: /v1beta/models/gemini-pro-vision:generateContent?%24alt=json%3Benum-encoding%3Dint (Caused by ConnectTimeoutError(, 'Connection to generativelanguage.googleapis.com timed out. (connect timeout=60.0)'))

想请问出现这个错误如何解决？梯子挂到美国，全局，Google gemini 网页端是能用的。

google-generativeai需要比较新的protobuf，但是其他都需要旧的。

其他都要3.几

大佬能做一个moonshotAI的插件吗？

翻墙太麻烦了能给写一个吗？

报错了

!!! Exception during processing !!!
Traceback (most recent call last):
File "H:\ComfyUI-qiuye\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "H:\ComfyUI-qiuye\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "H:\ComfyUI-qiuye\ComfyUI\custom_nodes\ComfyUI-0246\utils.py", line 381, in new_func
res_value = old_func(*final_args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "H:\ComfyUI-qiuye\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "H:\ComfyUI-qiuye\ComfyUI\custom_nodes\ComfyUI-Gemini\GeminiAPINode.py", line 91, in generate_content
response = model.generate_content([prompt, pil_image])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "H:\ComfyUI-qiuye\ComfyUI.ext\Lib\site-packages\google\generativeai\generative_models.py", line 248, in generate_content
response = self.client.generate_content(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "H:\ComfyUI-qiuye\ComfyUI.ext\Lib\site-packages\google\ai\generativelanguage_v1beta\services\generative_service\client.py", line 566, in generate_content
response = rpc(
^^^^
File "H:\ComfyUI-qiuye\ComfyUI.ext\Lib\site-packages\google\api_core\gapic_v1\method.py", line 131, in call
return wrapped_func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "H:\ComfyUI-qiuye\ComfyUI.ext\Lib\site-packages\google\api_core\retry.py", line 372, in retry_wrapped_func
return retry_target(
^^^^^^^^^^^^^
File "H:\ComfyUI-qiuye\ComfyUI.ext\Lib\site-packages\google\api_core\retry.py", line 207, in retry_target
result = target()
^^^^^^^^
File "H:\ComfyUI-qiuye\ComfyUI.ext\Lib\site-packages\google\api_core\timeout.py", line 120, in func_with_timeout
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "H:\ComfyUI-qiuye\ComfyUI.ext\Lib\site-packages\google\api_core\grpc_helpers.py", line 79, in error_remapped_callable
return callable(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "H:\ComfyUI-qiuye\ComfyUI.ext\Lib\site-packages\google\ai\generativelanguage_v1beta\services\generative_service\transports\rest.py", line 822, in call
body = json_format.MessageToJson(
^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: MessageToJson() got an unexpected keyword argument 'including_default_value_fields'

同一张图片，在不更改prompt的情况下重复运行好像并不会重复的向gemini发出解读？

上传一张图片后，第一次运行会向gemini发出请求。
但是如果不更换图片或者提示词的话，好像无论运行多少次都不会重复向gemini发出运行请求了？
第二次开始都是直接用原来生成的提示语进行图片生成

Header too Large?

Error occurred when executing CheckpointLoaderSimple:

Error while deserializing header: HeaderTooLarge

File "H:\ComfyUI-qiuye\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "H:\ComfyUI-qiuye\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "H:\ComfyUI-qiuye\ComfyUI\custom_nodes\ComfyUI-0246\utils.py", line 381, in new_func
res_value = old_func(*final_args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "H:\ComfyUI-qiuye\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "H:\ComfyUI-qiuye\ComfyUI\nodes.py", line 516, in load_checkpoint
out = comfy.sd.load_checkpoint_guess_config(ckpt_path, output_vae=True, output_clip=True, embedding_directory=folder_paths.get_folder_paths("embeddings"))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "H:\ComfyUI-qiuye\ComfyUI\comfy\sd.py", line 505, in load_checkpoint_guess_config
sd = comfy.utils.load_torch_file(ckpt_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "H:\ComfyUI-qiuye\ComfyUI\comfy\utils.py", line 14, in load_torch_file
sd = safetensors.torch.load_file(ckpt, device=device.type)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "H:\ComfyUI-qiuye\ComfyUI.ext\Lib\site-packages\safetensors\torch.py", line 311, in load_file
with safe_open(filename, framework="pt", device=device) as f:

Error occurred when executing Gemini_API_Zho: MessageToJson() got an unexpected keyword argument 'including_default_value_fields'

Error occurred when executing Gemini_API_Zho:

MessageToJson() got an unexpected keyword argument 'including_default_value_fields'

This response is not simple text.Use the `result.parts` accessor or the full `result.candidates[index].content.parts` lookup instead.

ERROR:root:!!! Exception during processing !!!
ERROR:root:Traceback (most recent call last):
File "E:\IMAGE\ComfyUI_windows_portable\ComfyUI\execution.py", line 155, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\IMAGE\ComfyUI_windows_portable\ComfyUI\execution.py", line 85, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\IMAGE\ComfyUI_windows_portable\ComfyUI\execution.py", line 78, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\IMAGE\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Gemini\GeminiAPINode.py", line 92, in generate_content
textoutput = response.text
^^^^^^^^^^^^^
File "E:\IMAGE\ComfyUI_windows_portable\python_embeded\Lib\site-packages\google\generativeai\types\generation_types.py", line 328, in text
raise ValueError(
ValueError: The response.text quick accessor only works for simple (single-Part) text responses. This response is not simple text.Use the result.parts accessor or the full result.candidates[index].content.parts lookup instead.

where do i entere the api?

sorry about this but dont see where to place it in the workflow?

在使用图生文的过程中得到这个报错提醒。能帮忙看看问题出在哪里吗？

got prompt
[rgthree] Using rgthree's optimized recursive execution.
!!! Exception during processing !!!
Traceback (most recent call last):
File "D:\ComfyUI\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI\ComfyUI\custom_nodes\ComfyUI-Gemini\GeminiAPINode.py", line 262, in generate_content
textoutput = response.text
^^^^^^^^^^^^^
File "D:\ComfyUI\python_embeded\Lib\site-packages\google\generativeai\types\generation_types.py", line 347, in text
raise ValueError(
ValueError: The response.text quick accessor only works when the response contains a valid Part, but none was returned. Check the candidate.safety_ratings to see if the response was blocked.

Prompt executed in 16.68 seconds

config.json 每次更新都会被重置，可不可以单独做个文件放key，更新的时候跳过去

Gemini_API_Zho : argument of type 'Part' is not iterable

Error occurred when executing Gemini_API_Zho:

argument of type 'Part' is not iterable

File "D:\Stable-Diffusion\StabilityMatrix\Data\Packages\ComfyUI\execution.py", line 153, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "D:\Stable-Diffusion\StabilityMatrix\Data\Packages\ComfyUI\execution.py", line 83, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "D:\Stable-Diffusion\StabilityMatrix\Data\Packages\ComfyUI\execution.py", line 76, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "D:\Stable-Diffusion\StabilityMatrix\Data\Packages\ComfyUI\custom_nodes\ComfyUI-Gemini\GeminiAPINode.py", line 77, in generate_content
textoutput = response.text
File "D:\Stable-Diffusion\StabilityMatrix\Data\Packages\ComfyUI\venv\lib\site-packages\google\generativeai\types\generation_types.py", line 327, in text
if len(parts) != 1 or "text" not in parts[0]:

the following node types were not found: Save Text File_mne

i use your sample of workflow of training
"When loading the graph, the following node types were not found:
Save Text File_mne
Nodes that have failed to load will show as red on the graph."

Install Missing Custom Nodes, i can not find it
i do miss a text file
how to fix it?

config.json

放秘钥的这个文件每次开启comfyui都得输入一遍吗，老是忘记好麻烦

我上传txt文件，提示不支持格式，需要什么类型呢？

400 POST https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-pro-latest:generateContent?%24alt=json%3Benum-encoding%3Dint: Unsupported MIME type: text/plain

链接不上呀，一直报Cannot connect to proxy

Error occurred when executing Gemini_API_S_Zho:

HTTPSConnectionPool(host='generativelanguage.googleapis.com', port=443): Max retries exceeded with url: /v1beta/models/gemini-pro-vision:generateContent?%24alt=json%3Benum-encoding%3Dint (Caused by ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 405 Method Not Allowed')))

File "D:\Program Files\ComfyUI-aki-v1.3\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "D:\Program Files\ComfyUI-aki-v1.3\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "D:\Program Files\ComfyUI-aki-v1.3\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "D:\Program Files\ComfyUI-aki-v1.3\custom_nodes\ComfyUI-Gemini\GeminiAPINode.py", line 298, in generate_content
response = model.generate_content([prompt, pil_image])
File "D:\Program Files\ComfyUI-aki-v1.3\python\lib\site-packages\google\generativeai\generative_models.py", line 262, in generate_content
response = self._client.generate_content(
File "D:\Program Files\ComfyUI-aki-v1.3\python\lib\site-packages\google\ai\generativelanguage_v1beta\services\generative_service\client.py", line 791, in generate_content
response = rpc(
File "D:\Program Files\ComfyUI-aki-v1.3\python\lib\site-packages\google\api_core\gapic_v1\method.py", line 131, in call
return wrapped_func(*args, **kwargs)
File "D:\Program Files\ComfyUI-aki-v1.3\python\lib\site-packages\google\api_core\retry\retry_unary.py", line 293, in retry_wrapped_func
return retry_target(
File "D:\Program Files\ComfyUI-aki-v1.3\python\lib\site-packages\google\api_core\retry\retry_unary.py", line 153, in retry_target
_retry_error_helper(
File "D:\Program Files\ComfyUI-aki-v1.3\python\lib\site-packages\google\api_core\retry\retry_base.py", line 212, in retry_error_helper
raise final_exc from source_exc
File "D:\Program Files\ComfyUI-aki-v1.3\python\lib\site-packages\google\api_core\retry\retry_unary.py", line 144, in retry_target
result = target()
File "D:\Program Files\ComfyUI-aki-v1.3\python\lib\site-packages\google\api_core\timeout.py", line 120, in func_with_timeout
return func(*args, **kwargs)
File "D:\Program Files\ComfyUI-aki-v1.3\python\lib\site-packages\google\api_core\grpc_helpers.py", line 76, in error_remapped_callable
return callable(*args, **kwargs)
File "D:\Program Files\ComfyUI-aki-v1.3\python\lib\site-packages\google\ai\generativelanguage_v1beta\services\generative_service\transports\rest.py", line 835, in call
response = getattr(self._session, method)(
File "D:\Program Files\ComfyUI-aki-v1.3\python\lib\site-packages\requests\sessions.py", line 637, in post
return self.request("POST", url, data=data, json=json, **kwargs)
File "D:\Program Files\ComfyUI-aki-v1.3\python\lib\site-packages\google\auth\transport\requests.py", line 541, in request
response = super(AuthorizedSession, self).request(
File "D:\Program Files\ComfyUI-aki-v1.3\python\lib\site-packages\requests\sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
File "D:\Program Files\ComfyUI-aki-v1.3\python\lib\site-packages\requests\sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
File "D:\Program Files\ComfyUI-aki-v1.3\python\lib\site-packages\requests\adapters.py", line 513, in send
raise ProxyError(e, request=request)

ComfyUI-Gemini Import Failed

I think this is a great node! However, I encountered some issues when I tried to install it. ComfyUI could not import this node, and the CMD window showed: ModuleNotFoundError: No module named 'google.generativeai'. But I have already installed the module, some of my friends also faced the same problem. Could you help us solve this issue please? Thank you very much.

报错连不上

400 POST https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-pro-latest:generateContent?%24alt=json%3Benum-encoding%3Dint: User location is not supported for the API use.

使用Gemini 1.5 pro 生成图片后全是雪花点

后端没有任何报错信息，大模型选择的是cosXL

希望增加proxy输入框，gemini真的太难连了

有proxy框会方便很多哦

梯子能登录gemini,写入config.json连不上，且每次启动config.json会重置！

Error occurred when executing Gemini_API_S_Zho:

400 POST https://generativelanguage.googleapis.com/v1beta/models/gemini-pro-vision:generateContent?%24alt=json%3Benum-encoding%3Dint: API key expired. Please renew the API key. [{'@type': 'type.googleapis.com/google.rpc.ErrorInfo', 'reason': 'API_KEY_INVALID', 'domain': 'googleapis.com', 'metadata': {'service': 'generativelanguage.googleapis.com'}}],用梯子登录gemini可以，但是把api写入config.json里，每次重启都会重置config.json文件，希望大佬解决下，谢谢！

Cannot import name 'cygrpc'

Cannot import C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Gemini module for custom nodes: cannot import name 'cygrpc' from 'grpc.cython' (C:\ComfyUI_windows_portable\python_embeded\lib\site-packages\grpc_cython_init.py)

User location is not supported for the API use

请问下这是墙的问题吗？尽管路由挂了SSR，开了全局好像也没办法

Error occurred when executing Gemini_API_S_Zho:

400 POST https://generativelanguage.googleapis.com/v1beta/models/gemini-pro-vision:generateContent?%24alt=json%3Benum-encoding%3Dint: User location is not supported for the API use.

zho-zho-zho / comfyui-gemini Goto Github PK

comfyui-gemini's Introduction

👋 Hi，我是 Zho ！

comfyui-gemini's People

Contributors

Stargazers

Watchers

Forkers

comfyui-gemini's Issues

Recommend Projects

Recommend Topics

Recommend Org