Open models on a laptop: what runs today
Which models fit, how much memory they need and where the speed comes from.
Models with a few billion parameters now run on an ordinary laptop.1 Four-bit quantisation cuts the memory they need to about a quarter.2 Speed depends more on memory bandwidth than on the processor.2 3




