In a major step toward democratizing artificial intelligence, Indian startup Ziroh Labs, in collaboration with IIT Madras, has unveiled a breakthrough system that allows large AI models to run on regular CPUs—without relying on expensive GPUs from companies like Nvidia. Their new framework, called Kompact AI, was showcased running advanced models like Meta’s Llama 2 and Alibaba’s Qwen 2.5 on a basic laptop powered by a shelf-bought Intel Xeon processor.
Unlike traditional AI development, which often requires high-performance GPU setups, Kompact AI focuses on optimizing inference processes, enabling AI to operate efficiently post-training on everyday machines. Ziroh claims that despite using CPUs, the system still delivers high-quality results, making AI significantly more accessible to startups, developers, and institutions lacking elite hardware.
The innovation arrives at a critical time when GPU shortages and high costs are creating an “AI divide,” especially in countries like India. According to IIT Madras director V. Kamakoti, this technology shows that “you don’t need a revolver to kill a mosquito”—a metaphor for achieving big results without heavy resources. Tech advisor William Raduchel, former chief strategy officer at Sun Microsystems, believes Ziroh’s method could have a profound market impact globally.
Ziroh’s technology has even caught the attention of chip giants Intel and AMD, who have tested its potential. As the AI industry looks for scalable, affordable solutions, Kompact AI stands out as a game-changing approach that could shape the future of computing.
