The Indian State and Shared Digital Infrastructure
The state as an organisation
If there is one thing that unites Indian citizens, it is our shared experience of the complexity and, sometimes, obduracy of the state. Every one of us has experienced such deep and repeated frustrations in our interactions with it — dealing with labyrinths of paperwork and corruption just to acquire a driver’s license, to fix the persistent potholes in our roads, or to obtain all the various permits that are required before a business can begin to operate — and witnessed such lapses in areas like public health and education, that we consider ineptitude to be the rule, rather than the exception. It is a reality perennially acknowledged across Indian society, in the anecdotes we all exchange with our friends and our families, and in our literature, films, and music. The conclusion is undeniable and unsurprising — the state is simply not governing in the manner that citizens expect and require it to, under the social contract.
What is surprising is that these disappointments persist even after decades of democracy and in spite of a widespread consensus among citizens that there is a crisis of governance. It is even more surprising that these stories exist in an era of economic growth unparalleled in India’s modern history, and alongside stories of remarkable successes. The Indian state has, for instance, consistently, and generally smoothly, conducted elections for hundreds of millions of voters, it has succeeded in eradicating polio, and it has improved its collection of direct taxes through the successful rollout of the Tax Information Network. The contrast between achievements such as these and the poor quality of public services in health, education, and sanitation, is striking. But what is perhaps most surprising to me after my years of working in government and in policy, is that after every story of administrative dysfunction there is no systematic investigation, not by the press, politicians, civil society, nor the bureaucracy itself, into why these issues occurred. Even though, or perhaps because, such incidents are so commonplace, there is no dedicated effort to diagnosing the causes of failure and restructuring processes to ensure that the failure is not repeated. Nor is there a structured exploration into why the numerous efforts at administrative reform that have been initiated over the years have themselves persistently been mostly unsuccessful.
I have come to realise that this is because we typically tend to think of — and focus our reform efforts on — the state as an institution, rather than as an organisation. Douglas North (1990) famously described institutions as humanly devised constraints that structure interactions, setting the formal rules of the game. Understanding the Indian state in this manner, through an institutional lens, has obvious and intuitive value. But it cannot, by definition, take you very far in terms of translating the formal rules into the actual practices and processes that touch the lives of citizens, or understanding the various costs inherent in those interactions. In India, discussions at an institutional level are characterised by a violent agreement that “things must change” but fall short of recommending any concrete, practical actions that can be taken to change things. This is understandable because institutional reform is a colossal undertaking with no easy answers. Even to those inside the government, it can sometimes seem like there is too much to do in too many places to be able to even identify a meaningful starting point for an analysis, let alone begin solving the issues. I believe that improving governance in India therefore necessarily requires understanding the state as an organisation, thinking about its systems and processes — how it manages things like information, funds flow, recruitment, personnel management, and training — and the impact these have on public officials.
1 I owe to Devesh Kapur the terminology of ‘institutional’ versus ‘organisational’.
Take, for instance, the Right to Information Act of 2005 (RTI Act). It was at the time of its introduction, and remains, a landmark legislation — and one made possible by conceptualising the state as an institution. In the 15 years since, however, it has become clear that there are still unaddressed issues around the substantive aspects of that institutional provision. The institutional focus does not necessarily contend with what kind of information will be most effective at extracting accountability, nor with the processes that may need to be put into place to ensure that data — especially relevant, up-to-date transparent, and usable data — is proactively collected, so that it is readily available when a citizen exercises their right to demand it under the RTI Act. Once we move past the institutional focus, it becomes possible to at least begin to understand the means by which failures of accountability in particular, and governance more generally, can endure in a long-standing democracy — and in so doing, opens up the space for more targeted reform interventions.
The institutional approach can also be limiting in terms of the solutions that are considered to address India’s governance issues — most prominently when it comes to the use of digital technologies. Focusing on the state as an institutional monolith can encourage a narrow dichotomy between its use of digital technologies as either unambiguously good or bad — rather than a more thoughtful consideration of which aspects of the state’s operations could benefit from adopting digital technologies, how, and what kinds of digital technologies in particular would be most appropriate given institutional responsibilities. Looking at the state as an organisation — unpacking it into its constituent parts and how they interact — can enable us to ask, and attempt to sensibly answer, questions such as – what is the outcome the state hopes to achieve, who are the officials whose actions will collectively shape that outcome, and what resources and incentives do each of those officials have (Khemani, 2019)? It makes it possible to first identify the localised constraints that prevent an official from performing their role and then — depending on the diagnosis of the problem — deploy a fitting digital solution.
The Sea Change Podcast Ep3: More about experiences in government, and the growing pressure on the bureaucracy to deliver services better.
The problem of problems
In his renowned book The Checklist Manifesto, the writer and public health researcher Atul Gawande describes three different kinds of problems: the simple, the complicated, and the complex. Simple problems are like baking a cake from a mix; there is a recipe, and you only need to know a few basic techniques — there is an obvious relationship between cause and effect. Complicated problems are like sending a rocket to the moon. There is no straightforward recipe, success requires multiple teams of experts in various fields. Yet, ultimately, complicated problems can be broken down into smaller, simple problems. It is not self-evident and requires specialist knowledge, but there is a linear relationship between cause and effect. Once you learn how to send a rocket to the moon, you can repeat that process with other rockets, and eventually perfect it. Complex problems, however, are like raising a child. There is no formula at all. What worked with one child may not work with another, because every child is unique and there are innumerable, interacting factors that play a role. And no matter what you do, there is no way to predict what kind of adult that child will be — there is no linear relationship between cause and effect, and the outcomes will always remain uncertain.
Thinking about problems in this way is crucial because the nature of our response must necessarily differ depending on the nature of the problem. Ultimately, the answer to the question of how we go about reforming the Indian state is that it depends on whether the problem is simple or complicated, or complex.
It is my understanding that about 80% (if not more) of the problems that public officials deal with are simple or complicated — you can be reasonably sure that a particular stimulus will lead to a particular response. The difference within these, that is, which are simple, and which complicated, depends on factors such as the scale, sequencing, and geographical spread of the problem, which inform the extent of coordination that is necessary to address the concern. But with both, it is possible to understand beforehand what response a particular stimulus will lead to, and to break them down into a series of simpler tasks using digital tools or changed processes.
The remaining problems are complex, in that it is not possible to predict from the outset what response a particular stimulus will lead to. There are multiple actors interacting with each other in ‘many-to many’ interactions, such that the relationship between cause and effect is not at all straightforward (Snowden, 2015). When it comes to these complex problems, the sheer difficulty of mapping out the consequences of any policy decision in abstraction means that it is essentially unknowable until engaged and interacted with (Pritchett, 2019). No policy design can be correct a priori; the key to its success lies in how it is implemented by officials with the autonomy to iterate and tailor that policy to their specific context. Unlike simple and complicated problems, complex ones cannot be broken down into a series of simpler tasks. They require that the power to address the problems be devolved to those officials rooted in the context, dealing with the problems where and when they arise.
A crucial weakness in the Indian state is that it currently misclassifies problems. It can deem complicated problems as complex, so that officials are subject to a tremendous amount of cognitive overload and administrative burden in managing issues that actually lend themselves to straight-through processing and can be restructured (digitally and/or manually, depending on what is appropriate to the issue) to be dealt with as a matter of routine — or what can be referred to as an ‘if-this-then-that’ treatment.
The Indian state can also deem complex problems as simple or complicated, insisting on edicts from the upper echelons of government that privilege homogeneity and compliance, and failing to contend with the diversity and specificity of contexts that officials must engage with. This can rob officials of the discretion and autonomy that they need to adequately respond, leading to a state-organisation that is congenitally not designed for agility. By misclassifying problems thus, the state can deprive its officials of the necessary cognitive space and resources to actually tackle complex problems, in an iterative fashion, when they arise. This is true of every level of government because every public official – right from a frontline Anganwadi Worker to a Chief Secretary – must at some stage deal with such complex problems in their work.
Read more: Societal Platform Core Values
This issue is compounded by the fact that the distribution of talent in government, like any other organisation, is bell-shaped. Extremely high-quality officials are rare, and as long as we misclassify problems, we also continue to misallocate our most precious resources. — by expecting the vast majority of our officials to deal with complex problems regardless of their capabilities or intrinsic motivation, and by overburdening our most exceptional ones with issues that can be dealt with by restructuring routine processes.
The power of shared digital infrastructure
My experience suggests to me that there are three key ways in which the Indian state-organisation would benefit from the solutions made possible by shared digital infrastructure (SDI) — both as smaller digital Lego bricks that can be reused or recombined in diverse settings, as well as overarching unified societal platforms that distribute the ability to solve and crowdsource innovation.
Read More: Societal Platform Tech Principles
First, SDI can provide tools that enable us to introduce observability into the system, and thereby generate information that may help us diagnose the problem correctly in the first place.
Digital technologies which back information systems can improve our understanding of various constraints by facilitating the collection of granular, high-frequency, usable data.
In particular, I think these efforts can be guided by an effort to understand the means, motives, and opportunity of an individual public official working in the relevant domain in a particular context (the MMO framework). The state-organisation is — like any organisation, public or private — made up of people. Public officials, of course, operate within and are shaped by processes, systems, and practices — all of which together constitute the broader state-organisation. But the individual public official is at the very heart of the administrative machinery, and their performance depends on the interaction between their motives (that is, their intent to achieve organisational goals) and the constraints they face in doing so. These constraints can be at the individual level, where they might not have the means to deliver on their job role, or at the organisational level, where they might not have the opportunity required to perform (John et al., 2019).
This is not to miss the institutional forest for the individual trees, but instead find a path into the forest to begin to understand what the issues are. Concentrating on individual officers also pegs initial reform efforts onto something palpable, putting agency and responsibility back into a bureaucrat’s hands, rather than grasping at the often more elusive strands of institutional culture. The MMO framework can help reform champions collect information and identify the key constraints in a given context, and structure policy and programmatic responses accordingly.
For instance, an analysis of child malnutrition in a particular district in Bihar would start by looking at the means, motives, and opportunity of an Anganwadi Worker operating in that district. The responses would differ depending on the nature of the constraints identified by this MMO framework. Complex problems would require that the relevant Anganwadi Worker be given the appropriate amount of agency to address the issue at hand, while simple and complicated problems could be addressed by orchestrating an if-this-then-that solution and focusing on the fidelity of policy implementation.
By introducing observability, SDI can also generate information about the work that is being done using the means, motives, and opportunity presently available. What this can mean in the context of administrative reform is deploying products and tools that can track and produce logs, time-series data, and other metrics that enhance the ability of supervisors (and ideally citizens) to monitor what is going on (especially but not only when a formal complaint has been lodged). For instance, data about the number of muster rolls signed by the mukhiyas in a particular district can be a leading indicator of the kind of payments that might be coming up in the next fortnight. This will allow officials to budget accordingly rather than having to wait until the payment requests are generated. Additionally, metadata on information being generated remotely can be captured using telemetry, allowing recognition of patterns and insights into the timing, frequency, and location of data entry or events to identify anomalies and risks. Data collected in public hospitals on disease prevalence, for instance, could be used to predict how many dialysis machines hospitals might need in a year’s time, allowing for timely and well-thought-out procurement decisions.
Second, SDI can provide solutions to simple and complicated problems that lend themselves to straight through processing.
A good example of this is smart payments. Smart payments essentially execute themselves to produce the output if certain predefined conditions are met. They work on the basic principle of “if this, then that”. For instance, if “the officer directly responsible has certified that the supplier has delivered the product” and if “the person required to verify the quality of what has been supplied does so”, then payment is released — so long as it was clearly articulated beforehand that these were the only two criteria to initiate a payment.
Implementing smart payments can sharpen the locus of accountability, take away discretion that has no value (and greatly reduce an officer’s ability to seek bribes), and decrease administrative burden. The certifying officer, for her part, is compelled to ensure that the certification is made on the basis of valid and credible data rather than on the basis of a bribe, or due to negligence. There is no way of diffusing blame through the system if it is subsequently found that the certification was incorrect. Moreover, it frees her up to devote her time to the truly complex problems under her remit. The algorithmization of the if-this then-that relationship may only be a limited example of SDI, but I believe it is a powerful one.
I think another good example relates to the Annual Performance Assessment Report (APAR), the self appraisal system followed in the Indian bureaucracy. APAR has always struggled with getting officials to file their reports in time. They were often filed late and only in the run-up to promotions. However, the introduction of the digital platform Sparrow enabled the implementation of a lock-in period, beyond which reports could not be edited. This compelled officials to fill in their forms on time and ensured that individuals could be held accountable for inaction.
Third, by making abundant scarce pivotal resources, such as expertise, content, or assessment capability, SDI can enable officials to deploy solutions even in the absence of local capabilities for certain things.
When it comes to complex problems, I feel strongly that reform is best carried out by those who are close to the problem. Ultimately, SDI will be crucial in these situations because it will enable the state organisation to distribute the ability to solve problems to those best situated to solve them, even if they are at the periphery. They can then build upon it to create diverse context-specific innovations.
This is especially true when it comes to leveraging digital technology to bring in platforms that connect the solution side with the local demand, capacity, and customisation side — this has the potential to devise interoperable solutions that can then be locally customised. The point of the platform is not to solve the problem itself, but instead how to resolve the frictions that prevent people from developing solutions.
The government ’s recently announced National Programme for Civil Services Capacity Building (Mission Karmayogi) is an important example of this. By seeking to facilitate officials’ access to on-the-job training, Mission Karmayogi enables officials to develop the means to perform their jobs well. Operated under the Department of Personnel Training (DoPT), the Integrated Government Online Training (iGOT) platform provides an online marketplace for impactful courses that those with identified competency gaps can use. Competency building courses (provided by universities and training academies) will be listed on the platform, and once individual ministries, departments, and organisations have identified the competencies that their officials need to develop, they can leverage iGOT to compare prices and impact scores to identify the course and provider that best meets their training requirements.
The central government has already used a version of this platform, with some success, to offer online health training for officials dealing with the COVID pandemic, including courses on sample collection, quarantine and isolation, and ventilator management. In its fully realised form, by making a repository of training materials that can be accessed through a smartphone available, iGOT can equip community health workers to build their knowledge about the nutritional needs of infants or teach them communication methods to help convince unwilling parents to immunise their children. iGOT has the potential to impact the means of officials at all levels of government and, in turn, improve the capacity of the Indian state to effectively discharge its responsibilities to its citizens.
Of course, SDI is not a panacea, and it will not solve all of India’s governance problems. But it is a worthwhile approach wherever there is something that has to be delivered at scale, which requires uniform quality of distribution but diversity in its different aspects, and where technology can be an enabler. In this way, it can help make the Indian state fundamentally more agile and responsive to citizen needs. SDI will not teach India’s teachers how to educate their students — nurturing a child’s minds is an archetypal complex adaptive problem — but it will help us make sure that the teachers do actually show up to school, that they are not overburdened by administration when they do, and that they can access resources that will help them do their jobs better (think of the work that EkStep does, for instance).
The window of reform, and a call to action
Reform inside government is only truly possible when three things co-occur: political feasibility, financial viability, and administrative desirability. That is a rare window of opportunity to introduce reforms, and I believe it can only correctly be considered by a reform champion who is situated within the relevant context. This is why partnerships between governments and philanthropists/social entrepreneurs are essential.
Once this opportunity comes to be and it is successfully recognised as such, the amount of time available to enact reform is typically 18-24 months. Even though electoral terms are 5 years, many politicians take some time to settle into their portfolio and are compelled to prioritise their attention on re-election in the last third or so of their term. In any case, very few ministers finish a single term given the vagaries of cabinet reshuffles, party priorities, and coalition politics. They therefore have little appetite or ability to focus their energies on any reform that will take more than 18-24 months, a time pressure compounded by the typical length of time that bureaucrats spend in any particular role. There is an even greater time pressure on reform champions when you account for the fact that the basic foundations and building blocks of Indian administration are so fractured that before you can actually enact reform, you probably need to spend some time setting those basics up in the first place.
SDI is an important aspect of making sure that extremely desirable changes in the Indian state move from the realm of “too big to solve / too difficult to do / we can’t do it in 24 months” to a feasible proposition. I am therefore of the view that reform champions in government will benefit greatly from having ready access to SDI that they can then leverage and customise as necessary.
2 I owe this phrase to the practical wisdom of Anil Swarup IAS, retired. former Secretary to the Government of India.
It is in this regard, in particular, that I see a role for reform champions outside government. The significant impact that I believe eGov Foundation has had makes it a very good example. Their success in the area of urban reforms is, in my view, the result of their willingness to take risks. They did not have a single customer government when they decided to start on their journey to build the DIGIT Urban Stack. But they believed in their theory of change and went out on a limb (supported by thoughtful and patient philanthropy). When the administrative, political, and financial circumstances coalesced that a reform champion who wanted to bring about change was in a position to be able to do so — eGov was there, ready with SDI that they could use to hit the ground running. eGov made what would otherwise have been an impossible project to pursue, practical and easily possible within 18-24 months.
As an analogy, imagine if a car company’s marketing strategy was to send drawings of cars to all its dealerships and inviting customers to come view those — and then say, “please give me 95% of the cost of building this car as an advance, and I will then deliver this car to you in 5 years”. When philanthropists or people in the social impact sector more generally ask “where is the buy-in for this product?”, that’s essentially the same as asking “I have provided the drawing, where is my customer?”. Like cars, SDI has to be built out in advance so reform champions can test drive it the minute they decide they need a car.
It is time to build out the SDI we believe in, so that we are ready whenever the Indian state is.
Gawande, A. 2009. The Checklist Manifesto. Picador.
John, A., Newton-Lewis, T., & Srinivasan, S. 2019. Means, Motives and Opportunity: Determinants of community health worker performance. BMJ Global Health, 4(5), 1–5. https://doi.org/10.1136/bmjgh-2019-001790
Kapur, D. 2020. Why Does the Indian State Both Fail and Succeed? Journal of Economic Perspectives, 34(1), 31–54. https://doi.org/10.1257/jep.34.1.31
Khemani, Stuti. 2019. “What Is State Capacity?” World Bank Policy Research Working Paper (February). http://documents1.worldbank.org/curated/en/336421549909150048/pdf/WPS8734.pdf
Pritchett, Lant. 2019. “Book Talk on Deals and Development.” Presented at the ESID Conference, 2019, Manchester, United Kingdom, September.
North, Douglass, C. 1990. Institutions, Institutional Change and Economic Performance. Cambridge University Press.
Snowden, David J. 2015. Keynote: Embrace Complexity, Scale Agility by Dave Snowden. Bangalore, India: Agile India 2015. https://www.youtube.com/watch?list=PL9Z-JgiTsOYSGUvr_4HLBQs InqlXu4bC&v=lYlqhvzI_VQ&ab_ channel=ConfEngine