Deep tech start-up Ziroh Labs, in collaboration with the Indian Institute of Technology Madras (IIT Madras) and IIT-M Pravartak Technologies Foundation, on Wednesday, launched ‘Kompact AI,’ an AI platform that enables foundational models to be built and run using CPUs without requiring GPUs.

The three parties are also aligning to establish a Centre of AI Research (CoAIR) to solve India’s compute accessibility challenges. The CoAIR will focus on developing practical, efficient AI solutions focusing on CPU and edge device inferencing.

California-headquartered Ziroh Labs has designed Kompact AI with the aim of making AI’s benefits available to all — even those who cannot afford expensive GPU infra. While AI development is currently dominated by large general-purpose models designed for scale, Ziroh Labs wants to serve small- and medium-sized companies and start-ups with their platform.

It solves the problem of inference and fine-tuning for models with less than 50B parameters and allows developers to build, train and infer AI using CPUs commonly found in Cloud Data Centers and Edge Devices. Kompact AI features a Model Library, which consists of multiple foundational models such as DeepSeek, Qwen, and Llama optimised for CPU compatibility. These models range from text, speech, vision and multimodal models.

While AI inference on the Edge is not new and big tech majors like Qualcomm and Intel also have solutions towards this, Ziroh Labs said that Kompact AI’s superior performance sets them apart.

Using CPUs

“Anything that can be computed on a GPU can also be calculated on a CPU; what really matters is that we are able to offer 3x more quantitative performance than current systems without compromising model quality or accuracy,” Hrishikesh Dewan, Co-Founder, Ziroh Labs, said. He added they have a roadmap for taking their platform to start-ups and other smaller companies to run domain-specific models.

The demonstration of Kompact AI platform was done in the presence of V. Kamakoti, Director, IIT Madras; Whitfield Diffie, Fellow of the Royal Society and Turing Awardee; Scott McNealy, Founder of Sun Microsystems; William J Raduchel, Former CTO of AOL Time Warner and Chief Strategy Officer at Sun Microsystems; and S. Sadagopan, Former Director, IIIT-Bangalore, among others. All of them are also advisors of Ziroh Labs.

“This effort (by Ziroh Labs) is certainly a major step in arresting the possible AI divide between one who can afford the modern hyper scalar systems and one who cannot,” Kamakoti said. He also said that IIT-M will help Ziroh Labs take the platform to more start-ups and MSMEs, and also governmental agencies.

Published on April 9, 2025