Mahalanobis in the era of Big Data and AI
Introduction:
- Even now, there is great sorrow over the passing of Professor P.C. Mahalanobis, an Indian scientist who popularised statistics. As a result of his lifelong courtship of statistics, unwavering and fearless leadership to advance a statistics and survey culture in India, the founding of the Indian Statistical Institute (dubbed “a mighty monument of his handicraft”), and his nurturing of a generation of exceptional academicians, Mahalanobis has left behind an enduring legacy.
The era of big data:
- Over the past 20 years, there has been a dramatic global change in the nature of data and statistics.
- With the advent of the Internet and almost everything contained within the Internet of Things, there has been a rush of data, most of it useless.
- There is currently a lot more data than what the technology can handle. Big Data is still believed to be in its infancy.
- Another crucial but related topic is how artificial intelligence (AI) is changing our lives and behaviours.
- Society is in a precarious situation. One may imagine Mahalanobis’ responses to the madness brought on by Big Data and the AI-driven revolution. Mahalanobis was a leading figure in statistics and a key figure in the early development of Indian democracy.
- Data has frequently appeared to be large in the past even if the technology at the time wasn’t capable of evaluating it.
- Mahalanobis also encountered a Big Data problem when his comprehensive surveys generated a large amount of data that needed to be analysed for effective planning.
- At his Indian Statistical Institute, Mahalanobis persuaded the government to purchase the first two digital computers for the country (and South Asia, as well), ushering in the era of computers in India in 1956 and 1958. These devices were capable of processing enormous volumes of data and challenging mathematical calculations.
A connection with the COVID 19 Situation:
- Mahalanobis had a great knack for utilising technology for the benefit of others, maybe as a result of his background in physics. He was “by training a physicist, by instinct a statistician, and by conviction an economist.”
- Therefore, it is reasonable to think that Mahalanobis would have understood the potential of AI to increase human productivity, including Big Data analysis, and perhaps in a way that is noticeably more effective than how AI is being utilised to accomplish that goal.
- As an example, the COVID-19 timeline was the subject of numerous disputed estimations, which was a recent Big Data error.
- One could argue that if Mahalanobis were still living today, the country’s COVID-19 response would have been much stronger.
- If he took the initiative, our data might be beyond doubt and the analyses might be much more accurate.
- And “Plan Man” from India may be the ideal choice for organising the creation of the best medical infrastructure for disaster preparedness in the future.
AI regulation:
- Thanks to Mahalanobis and the application of broad technical exercises with democratic participation, planning was substantially removed from politics roughly seven decades ago from the perspective of the newly independent nation. We are at a crossroads right now.
- India’s upcoming census will be carried out digitally. The dynamics of other polls are likewise expected to change under the new usual situation. Statistics is developing in this way. We would lose Mahalanobis’ inspirational leadership without the help of this new statistical perspective.
Conclusion:
- Since AI threatens to displace millions of jobs without creating replacements and is also contributing to the spread of false information, there is a big global effort to control it.
