Hello, I’m currently thinking back and forth about which new home server to build. What I’ve stumbled across: the i9 and new Core 9 Ultra all only support a maximum of 192GB RAM. However, some of the mainboards support 256GB (with 4 RAM banks and dual channel). Why?
I want to have the option of maxing out the RAM later.
I could buy 4x48GB RAM now and be at 192GB. Maybe I would be annoyed later that 48 GB of RAM is still “missing”. But what if I buy 4x 64GB RAM? 3x64 GB RAM makes no sense, because then dual channel is not used. 4x64 is probably not recognized by the processor?
Or are there LGA1851 or LGA1700 processors, capable of handling 256GB RAM?
Well, the numbers I find on google are: a Nvidia 4090 can transfer 1008 GB/s. And a i9 does something like 90 GB/s. So you’d expect the CPU to be roughly 11 times slower than that GPU at fetching numbers from memory.
I think if you double the amount of DDR channels for your CPU, and if that also meant your transfer rate would double to 180 GB/s, you’d be roughly 6 times slower than the 4090. I’m not sure if it works exactly like that. But I’d guess so. And a larger model also means more numbers to transfer. So if you now also use your larger memory to use a 70B parameter model instead of an 12B parameter model (or whatever fits on a GPU), your tokens will come in at a 65th of the speed.