Local, consumer-hardware SLM deployment

4-bit quantization and llama.cpp

Quantization makes running 32B-class SLMs feasible on Apple Silicon and similar devices.

1 source videos