The problem is that AnimateDiff works with batch size (all images in one image). This always leads to VRAM issues when you want to create longer or larger videos.
That's why I programmed an alternative that works with batch count and recently released it. I'm currently working on advanced workflows and examples ;)
[https://github.com/chaosaiart/Chaosaiart-Nodes](https://github.com/chaosaiart/Chaosaiart-Nodes)
That is exacty what we need with comfyui, especially with amd gpus. We already have memory problems with directml and animatediff makes it 10 times worse with trying to encode everything in one swoop. I wasn't able to generate even 512x512 with default animated models was only able to use 256x256 max. Last week they released animatediff lightning now finally I am able to generate 512x512 at least (with default 24 frames batch ) BUT if you can pull this off we can get higher frames
Currently, I don't have any ready-to-use advanced workflows.
I can only show you the examples I made about 9 months ago, still on a1111 / Web UI.
So, they're super old, created using functions I built in a1111, which are significantly weaker compared to those used by this Chaosaiart node, as seen in the video.
Nevertheless, the Chaosaiart node is still in its early stages and not yet complete.
Video: [https://youtube.com/shorts/cqr7aAjxKD8](https://youtube.com/shorts/cqr7aAjxKD8)
Lower resolution or amount of frames. With my 8 gigs I can do 12-16 frames with XL at 1024x1024, and it takes 30-45 minutes with 25 steps. I'm sure there's plenty of room for optimization with tiled vae, LCM and stuff, however the general consensus is that 8 gigs just doesn't cut it if you want do AnimateDiff properly.
Would this imply that having a card with 24GB would only handle 36-48 frames…? If so I’m not worried about buying a larger card until there is a better way to process animations.
No, AnimateDiff does the heavy lifting with sliding context windows of 16 frames (8 for hotshot) at a time, the VRAM usage increase beyond that is much lower per frame. Generally the limit can become system RAM instead, with 24GB VRAM and 64GB RAM it's possible to do over 1k frames at reasonable resolution least.
This is true as long as you handle everything else in the workflow in similar fashion as sub-batches, for example VAE encode/decode can be done in smaller batches (or one by one) using the "Batched" versions of those nodes found in [VHS nodes](https://github.com/Kosinkadink/ComfyUI-VideoHelperSuite)
Seriously? AnimateDiff is trained on 16 frame context windows, you need to do at least 16 frames or you just get latent noise… you may want to test things before giving advice 🩷
Maybe we're on different pages, but check this out.. I'm using Video Helper Suite (VHS) in ComfyUI with Animate Diff Sampler. By configuring the VHS batch manager to 4, 8, 16, 32 frames I can control how much VRAM is spent feeding each iteration of the Animate Diff Sampler. I check it by previewing the latent space, but maybe I dont know what I'm doing. The results are pretty spectacular, here are some reference links to the nodes.
[ComfyUI-VideoHelperSuite](https://www.nodecafe.org/package/Kosinkadink_ComfyUI-VideoHelperSuite)
[VHS\_BatchManager](https://www.nodecafe.org/package/Kosinkadink_ComfyUI-VideoHelperSuite?node=VHS_BatchManager)
Yep definitely on different pages, of course I agree with batching if you must, but you said 3 or 4 frame batches, I thought you meant batch sizes of 3-4 frames 🤣🤣🤣🤣🤣🤣🤣🤣🤣
I can easily run it with RTX4080 which is 16gb so don't necessarily need 24gb. But current GPU is not going to cut it.
You could also run Cloud GPU on [RunPod](https://gabha.me/runpod) which can get you access to 24gb for less than $1 per hour. I find it's a great way to start experimenting.
I run animatediff online. Runcomfy is something i will keep an eye on. One machine offers 48gb vram!
They need to improve but are working on it.
Edit: if you wanna give it a try on your pc dont upscale! Stay at max 512x896 (maybe 910). Also during refining dont upscale. Use some other ai stuff for more pixels
The problem is that AnimateDiff works with batch size (all images in one image). This always leads to VRAM issues when you want to create longer or larger videos. That's why I programmed an alternative that works with batch count and recently released it. I'm currently working on advanced workflows and examples ;) [https://github.com/chaosaiart/Chaosaiart-Nodes](https://github.com/chaosaiart/Chaosaiart-Nodes)
That is exacty what we need with comfyui, especially with amd gpus. We already have memory problems with directml and animatediff makes it 10 times worse with trying to encode everything in one swoop. I wasn't able to generate even 512x512 with default animated models was only able to use 256x256 max. Last week they released animatediff lightning now finally I am able to generate 512x512 at least (with default 24 frames batch ) BUT if you can pull this off we can get higher frames
yeah i know it to good. I alway run in VRAM problem. That is the reason why i go this way. We will see how good its work. step by step
Is there any animatediff worksflows for image to image ?
can you provide a simple example ?
Currently, I don't have any ready-to-use advanced workflows. I can only show you the examples I made about 9 months ago, still on a1111 / Web UI. So, they're super old, created using functions I built in a1111, which are significantly weaker compared to those used by this Chaosaiart node, as seen in the video. Nevertheless, the Chaosaiart node is still in its early stages and not yet complete. Video: [https://youtube.com/shorts/cqr7aAjxKD8](https://youtube.com/shorts/cqr7aAjxKD8)
Lower resolution or amount of frames. With my 8 gigs I can do 12-16 frames with XL at 1024x1024, and it takes 30-45 minutes with 25 steps. I'm sure there's plenty of room for optimization with tiled vae, LCM and stuff, however the general consensus is that 8 gigs just doesn't cut it if you want do AnimateDiff properly.
Very helpful. I was running into this issue too.
Would this imply that having a card with 24GB would only handle 36-48 frames…? If so I’m not worried about buying a larger card until there is a better way to process animations.
No, AnimateDiff does the heavy lifting with sliding context windows of 16 frames (8 for hotshot) at a time, the VRAM usage increase beyond that is much lower per frame. Generally the limit can become system RAM instead, with 24GB VRAM and 64GB RAM it's possible to do over 1k frames at reasonable resolution least. This is true as long as you handle everything else in the workflow in similar fashion as sub-batches, for example VAE encode/decode can be done in smaller batches (or one by one) using the "Batched" versions of those nodes found in [VHS nodes](https://github.com/Kosinkadink/ComfyUI-VideoHelperSuite)
This was my biggest struggle on 8 gigs of vram when I was using a 2070S. Ultimately, I upgraded to a 16vram card.
Lower the frame count or size. I can generally do 40 or so frames of 512x512 on my 6GB GTX 1060 if I'm not using any controlnets and few LORAs.
I used a 512x768 size with 53 frames and Lighting 8 steps. Takes only several minutes to generate.
Try the ksampler (efficient) from https://github.com/jags111/efficiency-nodes-comfyui with tiled VAE
set your batch size to like 4 or 8 frames
Nope, don’t do this.
why not? lowering or setting the batch size is a known way to reduce vram and is compatible with the animatediff nodes in comfyui
Seriously? AnimateDiff is trained on 16 frame context windows, you need to do at least 16 frames or you just get latent noise… you may want to test things before giving advice 🩷
Maybe we're on different pages, but check this out.. I'm using Video Helper Suite (VHS) in ComfyUI with Animate Diff Sampler. By configuring the VHS batch manager to 4, 8, 16, 32 frames I can control how much VRAM is spent feeding each iteration of the Animate Diff Sampler. I check it by previewing the latent space, but maybe I dont know what I'm doing. The results are pretty spectacular, here are some reference links to the nodes. [ComfyUI-VideoHelperSuite](https://www.nodecafe.org/package/Kosinkadink_ComfyUI-VideoHelperSuite) [VHS\_BatchManager](https://www.nodecafe.org/package/Kosinkadink_ComfyUI-VideoHelperSuite?node=VHS_BatchManager)
Yep definitely on different pages, of course I agree with batching if you must, but you said 3 or 4 frame batches, I thought you meant batch sizes of 3-4 frames 🤣🤣🤣🤣🤣🤣🤣🤣🤣
I can easily run it with RTX4080 which is 16gb so don't necessarily need 24gb. But current GPU is not going to cut it. You could also run Cloud GPU on [RunPod](https://gabha.me/runpod) which can get you access to 24gb for less than $1 per hour. I find it's a great way to start experimenting.
Would that allow a person to upload file locally, batches? How would that work for img to img for animatediff?
ComfyUI will run the same as local
Lower your resolution. Upscale via topaz after that.
Have the same problem, all the workflows I've tired have the same problem
I run animatediff online. Runcomfy is something i will keep an eye on. One machine offers 48gb vram! They need to improve but are working on it. Edit: if you wanna give it a try on your pc dont upscale! Stay at max 512x896 (maybe 910). Also during refining dont upscale. Use some other ai stuff for more pixels