IIT Madras-Backed AI4Bharat Launches India’s First Independent AI Evaluation Platform

The platform introduces a new layer to India’s AI ecosystem, where significant investments have been made in AI models, computing infrastructure and language technologies, while independent evaluation infrastructure has remained a critical missing component.

Voice of India aims to become a trusted reference point for AI evaluation in India by enabling governments, enterprises, researchers and AI developers to independently assess the performance of AI systems across Indian languages, accents, dialects and deployment environments.

Professor Mitesh Khapra of AI4Bharat at IIT Madras said India had a unique opportunity not only to build AI for the world but also to help define how the world measures it.

“Evaluation determines what gets built. With AI systems becoming more deeply integrated into society, rigorous evaluation becomes as important as model development itself. Through Voice of India, we hope to provide the research community, industry and policymakers with scientifically robust evaluation frameworks that reflect India’s linguistic diversity,” Khapra said.

Unlike approaches that rely only on performance metrics or benchmarks developed for Western contexts, the platform provides scientifically rigorous, India-specific evaluations that reflect how AI systems are used across the country.

The Voice of India AI Evaluation Platform has been established through a collaboration between AI4Bharat at IIT Madras and Josh Talks AI, with the ambition of supporting the wider AI ecosystem across government, enterprises, academia and industry.

The platform has been designed as an open, neutral and scientifically rigorous evaluation system that allows AI developers to benchmark their models while helping buyers make more informed deployment decisions.

As part of the launch, AI4Bharat and Josh Talks AI introduced a suite of benchmarks under the Voice of India framework, including a finance-focused benchmark to evaluate AI performance across key banking workflows.

“India has a once-in-a-generation opportunity to contribute something unique to global AI — not necessarily the biggest models, but the evaluation infrastructure that makes AI trustworthy for the world. The Voice of India AI Evaluation Platform is a step towards building that future,” said Supriya Paul, co-founder of Josh Talks AI.

The launch is significant as enterprises and government agencies deploying AI in India will, for the first time, be able to compare models through independent, deployment-specific evaluations rather than relying solely on vendor benchmarks.

The platform introduces a three-stage evaluation framework comprising constituent model evaluation, end-to-end deployment evaluation under real operating conditions, and periodic post-deployment audits.

This framework enables organisations to assess not only how AI systems perform in controlled environments but also how they perform for real users in real-world scenarios.