Add RAM Pressure cache mode

[rattus128]

Implement RAM Pressure cache

Implement a cache sensitive to RAM pressure. When RAM headroom drops
down below a certain threshold, evict RAM-expensive nodes from the
cache.

Models and tensors are measured directly for RAM usage. An OOM score
is then computed based on the RAM usage of the node and workflow staleness.

Note the due to indirection through shared objects (like a model
patcher), multiple nodes can account the same RAM as their individual
usage. The intent is this will free chains of nodes particularly
model loaders and associate loras as they all score similar and are
sorted in close to each other.

Has a helpful bias towards unloading model nodes mid flow while being able
to keep results like text encodings and VAE.

Example:

Linux, 64GB RAM system, RTX3060
WAN I2V template with FP16 Models
python main.py --novram --cache-ram 32.0

NOTE: You want to set the headroom significantly greater than your largest model.

At this point in time, it is running the low noise of WAN I2V after evicting the high noise.

After running another trivial workflow, and then returning the WAN, it recommences at the model loading and is able to use the cached TE and VAE results. The first nodes run are the UNETLoader (for high noise) -> Lora -> ModelSampling -> KSampler.

[Kosinkadink]

@guill hey, would you be able to take a look to see if some of the changes (like ui cache being removed) seem all good?

[asagi4]

I tried this and it seems to have some problems with either graph expansion or subgraphs.
I have a workflow where I get this:

!!! Exception during processing !!! 'NoneType' object is not subscriptable
Traceback (most recent call last):
  File "/home/sd/git/ComfyUI/execution.py", line 445, in execute
    node_output = execution_list.get_output_cache(source_node, unique_id)[source_output]
                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not subscriptable

Adding some debug logging and a dumb workaround to force the execution to continue and it logs this:

Output cache is none for source_node='322:166.0.0.1' unique_id='322:166'
Output cache is none for source_node='322:166.0.0.1' unique_id='322:166'

Node 322:166 is my PCLazyLoRALoader node that dynamically expands into a LoRALoader. With the workaround it fails with an NoneType error later because the output of the node becomes None.

I'll try to see if I can get a simpler workflow to fail.

[asagi4]

No subgraphs involved, it fails even with this simple workflow (--ram-cache 28 with 32GB of RAM on the host and 24GB of VRAM); works with the default cache.
fail_cache.json
In that workflow I get the following debug print:

Output cache is none for source_node='3.0.0.1' unique_id='3'

Just using LazyLoRALoader with a Preview Any node for the model output doesn't seem to be enough to trigger it, so I'm not sure what exactly the problem is.