I have been setting up Zram, Swap, Swappiness and EasyOOM daemon on 16gb ram boxes, or lower. Someone asked me about 32gb of ram, or more, and I’m unsure. Wondering if others have experimented with this!
I have been setting up Zram, Swap, Swappiness and EasyOOM daemon on 16gb ram boxes, or lower. Someone asked me about 32gb of ram, or more, and I’m unsure. Wondering if others have experimented with this!
Local AI can chew it up. Wasn’t able to run certain jobs on 64Gb until I switched to zswap.
What kind of token per second are you getting with your model partially on disk?
I think it was a video workflow, and it wasn’t too bad as it kept the main model in vram and was efficiently passing out whatever it was passing out. But it took nearly all of my 64GB of ram. From memory it was about 20min for a 5 second clip. Not great but not horrible.