Diffusion Single File
comfyui

How much GPU memory is required for BF16?

#6
by ZXCZXC13 - opened

How much GPU memory is required for BF16?

I'm running about 29.6 gb with a 5 second, .4megapixel gen, which is eating up about 100 gb of shared gpu memory (ram).

what is the minimun vram + ram config

I will be testing all the models soon on my rtx pro 6000.
Will see how much each of them takes

I'm able to run the 55gb text encoder and the 66.3gb model file together on my 128gb system ram and 24gb 4090.

I'm able to run the 55gb text encoder and the 66.3gb model file together on my 128gb system ram and 24gb 4090.

I am curious about generation speeds, I downloading the prunned Convrot files hoping to get good speeds. also what res are you pushing?

Well, speed is not the goal when juggling these beasts. lol. It is waaaaaaaaaaaaaay faster running the quantized versions. 4s 720p vid: 1st run was 15:01 then subsequent were 416-422s. The load times are brutal.

Well, speed is not the goal when juggling these beasts. lol. It is waaaaaaaaaaaaaay faster running the quantized versions. 4s 720p vid: 1st run was 15:01 then subsequent were 416-422s. The load times are brutal.

did you anyone checked the quality difference between the bf16, int8 and int8_pruned? any speed and quality comparison?

The quality is very close between the 3 versions. In fact, you can run the small one to see what kind of output you will get, and if you love the video, then re-run it with the same random number seed, with the high quality version of the model. You can think of it as a way to get a master copy, with the quantized versions offering very close almost identical previews. I posted a side by side comparison on my X acct, but I don't think I can post that link here.

ζˆ‘5090+256gε†…ε­˜οΌŒζ΅‹θ―•δΊ†ε„η§η‰ˆζœ¬οΌŒε‘ηŽ°ιƒ½δΈ€ζ ·οΌŒιƒ½θƒ½θ·‘οΌŒεͺζ˜―η”¨ηš„ε†…ε­˜ε˜ε€§δΊ†γ€‚ζˆ‘ζ΅‹θ―•εŽε‘ηŽ°η”Ÿζˆζ—Άι—΄δΉ‹η±»ηš„ε˜εŒ–εΎˆε°γ€‚ζˆ‘ζ„Ÿθ§‰δ»–θΏ™δΈͺζ¨‘εž‹ζœ¬θΊ«ζƒι‡ε₯½εƒε―Ήζ˜Ύε­˜δΈζ•ζ„ŸοΌŒδ½†ζ˜―εˆ†θΎ¨ηŽ‡ε’Œζ—Άι•Ώε₯½εƒζ²‘费显存

Mine runs well on 196gb ram and NVIDIA GeForce RTX 5090 32 gb vram. Able to make 30 second videos with loosing quality. Im pushing 45 seconds now. This one is far better than ltx so far

mine at 128gb RAM and rtx 4090 any 12second 1x1 and 16x9 HD videos are averaging between 10 to 13mins. Have sageattention and the workflow is the DasiwaMinimaxH3WorkflowsT2VA_cMMH3V11 from civitai. I still dont believe this is running locally πŸ˜‚

24GB in my 3090 and 16GB of normal RAM... the fact that VAE quants don't go lower than 5GB hurts a lot! I thought I could put the DiT, audio and video VAEs within my VRAM with 2GB to spare, then put the text encoder and mmproj into normal RAM with 3GB to spare.

The quality is very close between the 3 versions. In fact, you can run the small one to see what kind of output you will get, and if you love the video, then re-run it with the same random number seed, with the high quality version of the model. You can think of it as a way to get a master copy, with the quantized versions offering very close almost identical previews. I posted a side by side comparison on my X acct, but I don't think I can post that link here.

I think this is the most understated comment on the page! This is exactly what is awesome about this drop. Like people quickly forget how impossible it was to get free opensource local outputs like this even a few months ago! Like this is incredible!

Not a BF16 answer β€” I only tested the pruned int8 set β€” but the thread has moved to
"minimum VRAM + RAM", so here are measured numbers for that.

RTX 5070 Ti (16 GB VRAM), 125 GB system RAM, ComfyUI v0.30.1, torch 2.12.0+cu130.
Models: fl2va_pruned_int8_convrot + qwen3vl nvfp4 + both VAEs = 42.5 GB on disk.

One startup flag changes the RAM requirement by ~3.6x. Same job, only the flag differs:

flag ComfyUI RSS peak system RAM used page cache
none 45.4 GiB 51.0 GiB 11.9 GiB
--fast-disk 12.6 GiB 17.7 GiB 41.5 GiB
--cache-none 45.1 GiB 50.5 GiB 41.3 GiB

--fast-disk moves the weights out of anonymous memory and into page cache. Anonymous
memory cannot be reclaimed β€” if it does not fit you swap or OOM. Page cache can be
reclaimed: the OS drops it and re-reads from NVMe. --cache-none does nothing here;
it controls node-output caching, not weight residency.

@homeworkace β€” with 16 GB of RAM, --fast-disk is probably what you want. One thing
that breaks the budget you described: ComfyUI never releases the text encoder from
host RAM.
Sampled at 1 s intervals:
7.3 GiB idle
24.1 GiB after TE load (VRAM 15,219 MiB, 100% util = encoding)
45.6 GiB after DiT load (VRAM drops to 2,999 MiB β€” TE evicted from VRAM, host copy stays)
51.0 GiB after VAE decode

RAM never goes back down, so "DiT + VAEs in VRAM, text encoder in RAM" does not hold β€”
the host copies coexist regardless of what is in VRAM.

@Joe357683878 β€” your observation matches what I measured. VRAM peak was 14,197 MiB for
30 s @ 640x480 and 14,437 MiB for 5 s @ 1344x768. ComfyUI's DynamicVRAM keeps the ceiling
roughly constant and pays the difference in time rather than VRAM.

Full writeup with speed numbers, the sage-attention noise bug, and the measurement
scripts (EN/JA/ZH): https://github.com/Tomiigo/minimax-h3-16gb

Caveats: no BF16 tested, and I have not verified any of this on an actual 16 GB or
32 GB RAM machine β€” it is inference from a 125 GB box.

Sign up or log in to comment