Giter VIP home page Giter VIP logo

Comments (4)

gobshyte avatar gobshyte commented on September 17, 2024 1

Hi
In the meantime I basically culled as many of the Context Big nodes as I could, and it is working for now.
I had been using input/output/intermediate Context Big's for all sections of workflow and this really didn't help the situation!
Regardless, if/when it happens again (I am still adding to the same workflow) I will definitely try the workaround you suggest.
Many thanks, awesome nodes.

from rgthree-comfy.

gobshyte avatar gobshyte commented on September 17, 2024

Trying to figure this out, my guess so far is some kind of infinite/lengthy loop in to_execute, I have tried adding some debugging as follows:

https://stackoverflow.com/questions/60340710/len-cant-return-big-numbers

Line 385 of execution.py

                #to_execute = sorted(list(map(lambda a: (len(recursive_will_execute(prompt, self.outputs, a[-1], memo)), a[-1]), to_execute)))
                
                # Step through the map function
                mapped = []
                for a in to_execute:
                    # Debug: Print each item in to_execute
                    print("Processing item:", a)
                    
                    # Call recursive_will_execute and store the result
                    result = recursive_will_execute(prompt, self.outputs, a[-1], memo)
                    
                    # Debug: Print the result of recursive_will_execute
                    print("Result of recursive_will_execute:", result)
                    
                    # Calculate the length and form the tuple
                    length = len(result)
                    print("Length:", length)
                    print("a[-1]:", a[-1])
                    mapped_item = (length, a[-1])
                    
                    # Debug: Print the mapped item
                    print("Mapped item:", mapped_item)
                    
                    # Append to the mapped list
                    mapped.append(mapped_item)
                
                # Convert the mapped list to a list
                mapped_list = list(mapped)
                
                # Debug: Print the mapped list before sorting
                print("Mapped list before sorting:", mapped_list)
                
                # Sort the list
                to_execute = sorted(mapped_list)

Which results in:

[rgthree] Using rgthree's optimized recursive execution.
[rgthree] First run patching recursive_output_delete_if_changed and recursive_will_execute.
[rgthree] Note: If execution seems broken due to forward ComfyUI changes, you can disable the optimization from rgthree settings in ComfyUI.
Processing item: (0, '2959')
Result of recursive_will_execute: (6228, '2959')
Length: 6228
a[-1]: 2959
Mapped item: (6228, '2959')
Processing item: (0, '3326')
Result of recursive_will_execute: (11811208024791, '3326')
Length: 11811208024791
a[-1]: 3326
Mapped item: (11811208024791, '3326')
Processing item: (0, '2500')
Result of recursive_will_execute: (205763, '2500')
Length: 205763
a[-1]: 2500
Mapped item: (205763, '2500')
Processing item: (0, '2524')
Result of recursive_will_execute: (1172019, '2524')
Length: 1172019
a[-1]: 2524
Mapped item: (1172019, '2524')
Processing item: (0, '2832')
Result of recursive_will_execute: (30758354243, '2832')
Length: 30758354243
a[-1]: 2832
Mapped item: (30758354243, '2832')
Processing item: (0, '3072')
Result of recursive_will_execute: (11515927824164927, '3072')
Length: 11515927824164927
a[-1]: 3072
Mapped item: (11515927824164927, '3072')
Processing item: (0, '3330')
Result of recursive_will_execute: (383864260805491, '3330')
Length: 383864260805491
a[-1]: 3330
Mapped item: (383864260805491, '3330')
Processing item: (0, '3319')
Result of recursive_will_execute: (6, '3319')
Length: 6
a[-1]: 3319
Mapped item: (6, '3319')
Processing item: (0, '2807')
Result of recursive_will_execute: (63288799, '2807')
Length: 63288799
a[-1]: 2807
Mapped item: (63288799, '2807')
Processing item: (0, '3194')
Result of recursive_will_execute: (6, '3194')
Length: 6
a[-1]: 3194
Mapped item: (6, '3194')
Processing item: (0, '464')
Result of recursive_will_execute: (30402049455795419551, '464')
Exception in thread Thread-12 (prompt_worker):
Traceback (most recent call last):
  File "threading.py", line 1016, in _bootstrap_inner
  File "threading.py", line 953, in run
  File "C:\SDAI\ComfyUI\ComfyUI\main.py", line 111, in prompt_worker
    e.execute(item[2], prompt_id, item[3], item[4])
  File "C:\SDAI\ComfyUI\ComfyUI\custom_nodes\rgthree-comfy\__init__.py", line 211, in rgthree_execute
    return self.rgthree_old_execute(*args, **kwargs)
  File "C:\SDAI\ComfyUI\ComfyUI\execution.py", line 400, in execute
    length = len(result)
OverflowError: cannot fit 'int' into an index-sized integer

Specifically I think len(result) is failing, presumably because:
https://stackoverflow.com/questions/60340710/len-cant-return-big-numbers

I will have a look at it some more later today.

from rgthree-comfy.

gobshyte avatar gobshyte commented on September 17, 2024

I'm thinking it is similar to the issues here:
comfyanonymous/ComfyUI#1502

Trung0246/ComfyUI-0246#20

(I do have a lot of context pipeline nodes in the workflow...)

from rgthree-comfy.

rgthree avatar rgthree commented on September 17, 2024

Ok, as I mentioned in comfyanonymous/ComfyUI#3384, you must have a very large workflow. While the error appears to come from rgthree-comfy, that's really only because the optimization lets ComfyUI get to the error faster. I have a suspicion if you ran without the optimization you would see it hang for many, many minutes and either run out of memory or get the same error in the end.

Good news: We may be able to fix this for you because the rgthree-comfy optimization doesn't actually need to use len; I only do that so it can shim into the normal ComfyUI process with minimal changes.

However, some slightly bad news, is the fix requires making a change directly to comfyui code, meaning it's not something I can fix and submit in rgthree-comfy (and comfyanonymous hasn't responded to 7 month old PR). It also means you'll have to always use rgthree-comfy's optimization since the changed code would require it.

The change is actually quite simple. We just want to change the call site to not use len and, since rgthree-comfy's optimization keeps track of the length separately, we can use a count property.

In /execution.py, around line 385, you'll want to change this line:

# Change this:
to_execute = sorted(list(map(lambda a: (len(recursive_will_execute(prompt, self.outputs, a[-1], memo)), a[-1]), to_execute)))

# To this:
to_execute = sorted(list(map(lambda a: ((recursive_will_execute(prompt, self.outputs, a[-1], memo)).count, a[-1]), to_execute)))

Also, in /custom_nodes/rgthree-comfy/__init__.py around line 246:

# Change this:
will_execute.add(len(will_execute_value))

# To this:
will_execute.add(will_execute_value.count)

Let me know if this works, it's just an idea, and it's not impossible something else wouldn't break afterwards with such a large workflow.

from rgthree-comfy.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.