The Indian Administrative Service Meets Big Data - Milan Vaishnav & Saksham Khosla (Carnegie Endowment Research Paper)

India’s economy has grown rapidly in recent years, but the country’s bureaucratic quality is widely perceived to be either stagnant or in decline. While small, India’s elite civil service cadre, the Indian Administrative Service (IAS), occupies the nerve center of the Indian state. Unfortunately, the IAS is hamstrung by political interference, outdated personnel procedures, and a mixed record on policy implementation, and it is in need of urgent reform. The Indian government should reshape recruitment and promotion processes, improve performance-based assessment of individual officers, and adopt safeguards that promote accountability while protecting bureaucrats from political meddling.

http://carnegieendowment.org/2016/09/01/indian-administrative-service-meets-big-data-pub-64457

Key Insights Into the IAS

·         For officers early in their careers, exam scores and education are highly predictive of future success.

·         Older officers who enter the service as part of larger cadres face limited career prospects and are less effective at improving economic outcomes.

·         While initial characteristics heavily shape career trajectories, in the long term, there are clear rewards for officers who systematically invest in training or acquire specialized skills.

·         Individual bureaucrats can have strong, direct, and measurable impacts on tangible health, education, and poverty outcomes.

·         Surprisingly, officers with strong local ties—thought to be vulnerable to corruption—are often linked to improved public service delivery.

·         Political interference generates substantial inefficiency: the best officers do not always occupy important positions, while political loyalty offers bureaucrats an alternative path to career success.

·         Counterintuitively, greater political competition does not necessarily lead to better bureaucratic performance.

A Reform Agenda for the Civil Service

·         The central and state governments should pass and implement pending legislation that protects bureaucrats against politically motivated transfers and postings. Despite judicial prodding, most states have stalled on such moves.

·         The IAS should use data on civil servants’ abilities, education, and training when placing officers early in their careers. As officers gain experience, performance metrics can inform key decisions about promotion and allocation.

·         The government should consider the proposal that officers deemed unfit for further service at certain career benchmarks be compulsorily retired through a transparent and uniform system of performance review. While the present government has moved in this direction, this procedure should be institutionalized.

·         State and central governments should discuss whether state cadres should be given greater latitude to experiment with increasing the proportion of local IAS officers and track their relative performance.

·         Further research is needed to better understand the impact of local officers on development outcomes, to develop data on bureaucratic efficiency among officers in senior posts, and to systematically examine the workings of state-level bureaucracies.

INTRODUCTION

In the annals of global democracy, India holds an unusual status. Almost seventy years ago, at the time of winning its independence from the British Empire, the country instituted a system of universal franchise at an extremely low level of per capita income and when the vast majority of its population lacked even basic literacy. Over these seven decades, India has surprised many pessimists by sustaining democratic governance despite remaining a very poor country.

The considerable economic progress India has achieved is undeniable, particularly in the last few decades. Between 1990 and 2014, India averaged an annual rate of per capita economic growth of nearly 6.5 percent, reducing the share of its population living in extreme poverty from 50.3 percent as of 1987 to 21.3 percent by 2011 in the process.¹

In today’s global economy, marked by slumping growth rates and extreme volatility, India stands out as a relative bright spot. In the coming years, according to forecasts by the International Monetary Fund, India is expected to remain the fastest-growing major economy in the world, having finally displaced China as the occupant of this coveted designation.²

Yet while India’s short-term prognosis is quite favorable, there is nothing preordained about its future economic trajectory. Globally, there is a robust, positive relationship between the quality of government and economic progress. But India has experienced rapid growth in spite of below-par governance.

Indeed, the quality of India’s public-sector institutions in particular has struggled to keep pace with the country’s rapid economic advancement. As the adage goes, “India grows at night while the government sleeps.”³ Unless India is able to reform its administrative apparatus, sustained economic gains will prove elusive.

Those who have come into contact with the country’s bureaucracy have long criticized it for being cumbersome, slow, inefficient, and often venal. Indeed, its infirmities are so widely known that the Indian bureaucracy is the subject of unstinting pop culture mockery. From Ji Mantriji, an adaption of the BBC series Yes Minister that made light of political will meeting administrative intransigence, to Office Office, a long-running sitcom about a hapless common man stymied by a corrupt, labyrinthine state, the Indian administrative apparatus has not fared well in terms of popular perception.

Today, in 2016, there is a lingering view that corruption and politicization of the civil services have become more, not less, entrenched. According to a measure of government effectiveness developed by the World Bank that captures the quality of a country’s civil service, its independence from political pressure, and the quality of policy formulation and implementation, India’s performance is middling. Data from 2014 place India in the forty-fifth percentile globally, nearly a 10 percentage point decline from the country’s position in 1996, when these data were first collected.⁴

The Indian Administrative Service (IAS) is situated at the nerve center of this bureaucratic state. It has played a crucial and storied role in managing natural disasters, preserving law and order during episodes of political instability, and conducting free and fair elections.⁵ Unfortunately, the IAS faces a number of serious challenges—from diminishing human capital to political interference—that, if left unaddressed, will lead to further institutional decline. While a competent, functional IAS may not be a sufficient condition for improving key development and governance outcomes, it is likely a necessary one. Fortunately, a host of new, data-driven research sheds light on the conditions under which the IAS can become more efficient and effective in (a modified version of) its present structure.

CLEANING RUST FROM THE FRAME

While small in number, the influence of the IAS is outsize. It constitutes but a tiny fraction of all government bureaucrats, collectively (and, typically, pejoratively) referred to as babus in Indian parlance—there were 3.3 million individuals employed by the government of India (at all levels) in January 2014, but roughly only 4,800 serving IAS officers as of January 2015.⁶ Yet, perhaps no single bureaucratic entity has received more attention, from actors ranging from government commissions to op-ed columnists, than the IAS.⁷

Saksham Khosla

This group represents the crème de la crème of the Indian civil service. Dating back to the times of the British Raj, when it was known as the Indian Civil Service (ICS), the IAS has occupied the most pivotal administrative posts across India at every level, from administrative districts (analogous to U.S. counties) to states, all the way up to the central government in New Delhi.⁸

Over time, however, even sympathetic voices admit that this “steel frame,” as then British prime minister David Lloyd George termed the ICS in 1922, has deteriorated.⁹ An increasingly intransigent political executive has repeatedly abused its authority to transfer, suspend, and promote officers at will, damaging the morale of the service and brazenly politicizing its very essence. The quality of new hires is said to be falling as the best and brightest college graduates are unimpressed by uncompetitive public-sector wages, while those who do enter government service are often not allowed to develop domain expertise that can inform policymaking in an increasingly complex, interconnected world. “The overwhelming perception,” one commentator quipped, “is that corrupt bureaucrats are despised but thrive; the honest are respected but do not rise; and idealists end up in the boondocks.”¹⁰

These concerns about the role and relevance of the IAS are not restricted to think tank forums and newspaper columns. When then Indian prime minister Manmohan Singh delivered his inaugural address to the nation in 2004, he called the reform of administrative and public institutions—including refurbishing the IAS—an “immediate priority” for his government.¹¹ Although very little administrative reform was implemented during his government’s two terms in office, more than a decade later Singh’s successor, Prime Minister Narendra Modi, is echoing many of the same sentiments. Indeed, one week after Modi was sworn in as India’s fourteenth prime minister in May 2014, he summoned all 77 secretaries of central departments and ministries—most of whom are senior members of the IAS—to his official residence for a closed-door meeting. The session, the first in a decade, was a pep rally of sorts for senior IAS officers, an attempt to rejuvenate the upper echelons of a bureaucracy that had grown increasingly demoralized.¹²

Although there is no shortage of opinions on what ails the IAS or what fixes should be implemented, there has been a surprising paucity of hard data on its ranks and their performance. Bureaucratic activities in India are conducted concurrently at the district, state, and central levels with striking variation in the degree of efficiency at each level—not to mention wide variation across geographies. Any proposals for serious, sustainable administrative reform must pry open the black box of the bureaucracy. In particular, three questions stand out: What determines the career success of officers in the IAS? To what degree can individual officers influence tangible development outcomes in areas such as poverty, health, and education? And what impact does politics have on bureaucratic functioning?

A spate of recent research, combining unprecedented access to data on the career profiles of IAS officers with granular measurement of local development outcomes as well as electoral and political dynamics, sheds new light on these questions. This paper reviews the findings of these studies and discusses their implications for institutional reform. These studies are not well-suited to address existential questions regarding the potential role the IAS should play in a twenty-first-century India, but they do help provide answers to the three questions above.

The literature finds that bureaucrats’ initial endowment of human capital is highly predictive of future success in moving up in the ranks. While initial conditions heavily shape career trajectories, there are clear payoffs to officers who show improvement and acquire specialized skills during their careers.

Moving up the IAS ranks is a narrowly construed definition of success, however. The quality of individual bureaucrats can also have strong, direct, and measurable impacts on tangible development outcomes. One characteristic in particular that seems to matter is local embeddedness. Officers with local ties are associated with improved public goods outcomes—but only when propitious conditions exist that reduce the risk of corruption.

Bureaucrats do not function in a vacuum; political interference poses a constant threat to bureaucratic functioning. Research has shown that political loyalty—rather than professional qualifications—represents a viable path to professional mobility. However, the impact of politics is not uniformly negative. For instance, in areas where elections are less competitive (and, hence, incumbent politicians are more likely to be reelected) bureaucrats are better motivated to do their job. This is at odds with the prevailing wisdom that greater electoral competition incentivizes better bureaucratic performance.

The quality of individual bureaucrats can also have strong, direct, and measurable impacts on tangible development outcomes.

Taken together, this new empirical literature suggests several obvious recommendations for civil service reform. For starters, it is imperative that the central and various state governments institute key safeguards to protect against arbitrary, politically motivated transfers and postings of civil servants. Furthermore, the IAS should use data on civil servants’ abilities, education, and training to inform posting decisions early in their careers. On this, the research is unambiguous: there is valuable information that can predict the future effectiveness of civil servants, yet these data are rarely used by those in charge of making personnel decisions. The advent of big data also provides a natural opportunity to use metrics on officers’ performance in the field to inform promotion and retention decisions. Finally, although India’s founders chafed at the prospect that IAS officers should be too closely linked with their home states for fear of elite capture, this issue should be revisited for further consideration.

While these suggested alterations are relatively minor in nature, they are perhaps more consistent with what the political traffic in India can realistically bear. When it comes to the bureaucracy, even enacting minor reforms—much less sweeping change—can come with a hefty political price tag, given the power of public-sector unions.¹³

There are three caveats about the papers reviewed here and what they do and do not focus on. First, the sole preoccupation of this study—and that of the literature analyzed—is with the IAS, even though it is but one segment of the sprawling Indian civil service. This narrow focus is arguably a consequence of the IAS’s disproportionate influence over policy formulation and implementation. Unfortunately, this narrow focus precludes an examination of the various state-level civil service bureaucracies. The variation in bureaucratic performance across Indian states is crying out for further exploration; to date, there have been few studies on India that have concerned themselves with administrative dynamics at the subnational level. While this paper does not remedy this shortcoming, it does add a novel dimension to prior studies of the bureaucracy by surveying new literature that uses previously hard-to-access professional histories of individual IAS officers coupled with highly disaggregated political and economic data.

Second, there are many dimensions of IAS officers’ job descriptions that are worth scrutinizing. They maintain responsibility for multiple tasks—from regulation to law and order, and from election management to the administration of development schemes. This analysis is focused on this final domain—development and social service delivery. This approach is justifiable, not least because it is easier to identify and measure qualitatively meaningful outcomes in the development domain, relative to regulation or justice. Furthermore, hard data on development outcomes and the control IAS officers have over state-led interventions allow researchers to draw a connecting line from one to the other. Development and service delivery arguably represent the biggest growth areas for elite bureaucrats, given the rise of the welfare state in India and the concomitant proliferation of government-sponsored social-sector programs. And the IAS’s developmental mandate is the service’s most conspicuous area of underperformance.

Third, some of the research reviewed in this paper comprises unpublished work, and so the findings from these various studies are necessarily tentative. Nevertheless, given the complementarities in the initial conclusions of this growing literature, their results merit substantive consideration.

NEITHER INDIAN, NOR CIVIL, NOR A SERVICE?

The present-day dynamics of the IAS have colonial roots. The decision of independent India’s founding leaders to retain the basic structure of the ICS, the predecessor of the IAS, has meant that the elite civil services exhibit a significant degree of path dependency when it comes to their operational dynamics. However, the ICS was built to serve a very different political master at a very different time in history.

The ICS first came into existence through the Government of India Act of 1858.¹⁴ The ICS was created as an all-India service, with positions reserved at every level of government: in administrative districts, for collectors (about 50 percent of all officers); in provincial headquarters (roughly 25 percent); and in the central government (another 10 percent).¹⁵

In its design, the ICS—not surprisingly—imitated Britain’s bureaucratic setup, known informally as the Whitehall or Westminster model, in which senior civil servants advise cabinet-rank ministers on policy formulation.¹⁶The so-called steel frame of the British Raj was a small organization administering a massive country; the ICS numbered 1,032 officials at its peak in 1931 out of an overall bureaucracy of about 1 million officials ruling over an undivided India totaling approximately 350 million people.¹⁷ ICS officers in the prewar period were among the best-paid bureaucrats in the world; in 1935, an ICS secretary to the government of India earned 6,666 rupees, while the U.S. secretary of the treasury earned just half as much.¹⁸

Upon achieving independence in 1947, India’s founding leadership retained the ICS with little alteration—aside from a change in name—a decision met with some controversy. A segment of public opinion viewed ICS officers as unsympathetic facilitators of imperial rule. Indeed, Indians were allowed to sit for the service’s entrance exam beginning only in 1922; prior to that date, no native Indians were represented in the service’s ranks.¹⁹Those who did successfully join the service once the rules were changed were often treated with suspicion and called “brown sahibs” by their fellow countrymen. Furthermore, in a federal India, many state chief ministers feared that a central administrative structure, as embodied by the ICS, would interfere with decentralized forms of authority.

Notwithstanding these concerns, the founding Congress Party leadership decided to retain the ICS structure because party leaders had little experience with alternative models and were cognizant of the potentially large disruption scrapping the service would entail. While certain alterations might have been necessary, they reasoned it would be wiser to proceed gradually. Furthermore, despite the scorn Indians may have heaped on the ICS prior to independence, many prominent elites associated with the independence movement were impressed by the way the civil service had largely maintained order in the tumultuous decade prior to 1947. As one scholar put it, “even Indian nationalists and their newspapers considered [the ICS] impartial, high-minded, conscientious, and incorruptible.”²⁰

The ICS was far from politically neutral during the Raj era, in the sense that it was deeply invested in the continuation of the status quo and was opposed to the nationalist Congress Party. But it was arguably neutral in the sense of subordination. That is, members of the ICS had a highly professionalized, technocratic self-image, carrying out the wishes of their superiors while subordinating their personal views on policy.²¹ Many nationalist leaders believed that the service would continue to be loyal in the wake of independence, but this time grounded in a democratic context and beholden to India’s indigenous popular leadership.²²

To proponents of continuity, the value of maintaining an all-India civil service was premised on three additional underlying beliefs: that such officers would have a national, rather than parochial, outlook; that an elite bureaucratic corps would attract the best nationwide talent; and that such a group would possess an ingrained sense of independence and impartiality.²³

One of the most persuasive voices in this camp was that of Sardar Vallabhbhai Patel, India’s first home minister, who campaigned vigorously for administrative continuity. Speaking at a provincial premiers’ conference in 1946 to decide the future of the All India Services, Patel stated that ICS officers were “useful instruments” that would “also serve as a liaison between the Provinces and the Government of India and introduce [a] certain amount of brashness and vigor in the administration both of the Centre and the Provinces.”²⁴ The ICS and IAS would play a critical role, therefore, in holding together India’s highly divided federal polity.

RULES OF THE ROAD

A deeper understanding of the internal processes driving the IAS’s policymaking function is crucial for identifying opportunities for organizational reform. The IAS possesses many of the classic features of a professional bureaucracy. This mandarin-style service has several important characteristics: meritocratic recruitment via a competitive examination; a distinct (albeit rigid) set of allocation and assignment procedures, especially in the early stages of an employee’s career; and predictable, long-term career incentives that reward seniority.

Organization and Recruitment

The term civil service in India is an umbrella category for several discrete organs. The IAS, along with the Indian Forest Service and the Indian Police Service (IPS), comprise the All India Services.²⁵ These organs serve both the state and the central governments and, hence, are said to be under the dual control of both tiers. This premise of dual control was underpinned by the belief held by India’s founders that the All India Services would need to act as a bridge between the center and the states, without being overly beholden to either.²⁶ While the central government largely controls recruitment and advancement, IAS officers belong to state cadres. Within these cadres, officers are one of two types: approximately half spend most of their careers in the service of their respective state governments, while the other half receive postings with the central government in New Delhi.²⁷

The IAS possesses many of the classic features of a professional bureaucracy.

Entry into the IAS is highly competitive. The Union Public Service Commission (UPSC), an independent constitutional body, recruits officers to the All India Services and the Central Civil Service through a multistep examination process.²⁸ Anywhere between 200,000 and 400,000 individuals annually sit for the Civil Services (Preliminary) Examination, a number pared down to approximately 10,000 for the Civil Services (Main) Examination and interview. Fewer than 1,000 candidates make the final cut; these successful few are known as direct recruits. Of these, only the top 100 or so qualify for the IAS, depending on vacancies; the remaining candidates are eligible for entry into the other All India and Central Civil Services.²⁹

Once admitted, IAS officers receive initial training at the Lal Bahadur Shastri National Academy of Administration in the state of Uttarakhand. This training comprises a year of classroom instruction on the machinery of government, followed by another year of district-level training to expose trainees to field realities. Based on their record of performance, state civil servants can also be promoted into the IAS on the recommendations of the Staff Selection Commission attached to the Department of Personnel and Training.³⁰

Allocation to State Cadres

After graduation, IAS officers are assigned to a state cadre through a quasi-random allocation process. The cadre allocation rule takes into consideration officers’ rankings as determined by the entrance exam, vacancies in each state, and a rotating roster of states organized alphabetically. For instance, individuals who perform better on the entrance exam are more likely to be assigned to their state of origin. At any given time, however, only one-third of any given cadre may comprise officers serving in their home state. Because officers spend the majority of their careers in their respective state cadres, allocation rules are explicitly geared toward ensuring that all states receive a uniform quality of talent.

Predictable Career Ladder

IAS officers move up in the bureaucratic hierarchy through clearly defined promotion waves (see table 1). Promotions for junior positions are based on years of service, while appointments to higher-level posts are contingent on screening by a committee of senior civil servants (and, thus, ostensibly involve an element of merit-based selection). Performance evaluation is conducted through a performance appraisal report written by an officer’s superiors.³¹

A critical juncture in every officer’s career is the process of empanelment. Exceptionally competent officers are placed on a panel by a special committee of secretaries entrusted with evaluating their service records; from this panel they are available for promotion as vacancies arise. Successfully empaneled officers are eligible to serve in the most senior and prestigious positions in government.

Two factors remain constant throughout the careers of IAS officers: first, from their earliest days on the job, they are entrusted with substantial responsibilities and authority over a large population; and second, career progression is driven by seniority, not performance. After completing their initial two-year training period, officers begin their careers as subdivisional magistrates, assisting superior officers in district government. After four to five years in their cadre (where they may be promoted to the post of an additional chief magistrate or chief development officer), officers are usually assigned to the post of district magistrate, a district’s chief executive. District magistrates oversee revenue collection, law enforcement, and crisis administration, making them among the most powerful bureaucrats in the country. They also are responsible for supervising all infrastructure development projects and working with district-level agencies to implement centrally sponsored schemes like the Pradhan Mantri Gram Sadak Yojana, an all-India rural roads program, or the National Rural Employment Guarantee Scheme, a federal workfare program and the largest social-sector scheme in the world. On account of their wide-ranging powers, district magistrates can be described as the “king-pin” in a district’s affairs, as one analyst put it.³²

Typically after nine years of service, officers become eligible for positions with the state government or the central government as part of a ministry’s junior staff.³³ At the sixteen-year mark, officers are eligible for the rank of joint secretary to the government of India. At the state-government level, officers become eligible for the highly prestigious post of secretary, which allows them to manage various state-level departments. Finally, retirement is fixed for all IAS officers at sixty years of age.

FLAILING STATE

Nearly seven decades following independence, India’s steel frame is exhibiting considerable signs of strain. Even insiders agree that the apex civil service is not functioning anywhere close to its highest capacity. Commenting on a new report by a political consultancy that rated the Indian bureaucracy as the most inefficient in Asia, leading political scientist Pratap Bhanu Mehta wrote, “the bureaucracy confuses ends with means, rules with outcomes, control with efficiency.”³⁴ The IAS of today is hampered by several concomitant issues: a decline in the quality of recruits, political interference, perverse incentives for career advancement, a lack of specialized expertise, and a perception of widespread corruption. These infirmities have compromised the ability of the IAS to fulfill its mandate.

Declining Human Capital

One reason for the IAS’s waning reputation is the supposedly diminishing quality of its recruits. Despite an incredibly competitive entrance examination—in 2016, 180 candidates were selected from a pool of 465,882 applicants (a success rate of 0.038 percent)—the government is finding it hard to lure young talent away from increasingly attractive private-sector opportunities (see table 2 for data on all UPSC-conducted exams).³⁵

According to a recent study, successful candidates are getting older, are increasingly less likely to hold a postgraduate degree, and take an average of four attempts to pass the entrance exam. The combination of rising average age and lack of advanced academic qualifications implies that many candidates spend a majority of their twenties preparing for and taking entrance examinations for the elite civil services.³⁶

Beyond the declining quality of new entrants, poor remuneration and severe pay compression—a reduction in the ratio of the highest government salary to the lowest—have had adverse effects on the morale and social prestige associated with a civil service career (see table 3).³⁷ One former IAS officer who joined the service in the mid-1980s notes that secretaries to the government of India earned as much money as their predecessors did fifty years earlier, in the mid-1930s. Once among the best-paid civil servants in the world, IAS officers slid toward the opposite end of the spectrum over subsequent decades.³⁸

Diminished Independence

A deeply pervasive culture of political interference has confounded efforts to combat the perceived diminishing quality of human capital in the bureaucracy. According to a 2010 survey of civil servants, only 24 percent believed that postings to sought-after stations were merit based. More broadly, nearly one in two respondents thought undue outside pressure was a significant problem.³⁹

Short average tenure in posts—as low as six months in India’s most populous state, Uttar Pradesh—and a growing number of posts of varying importance, duties, and pay effectively enable elected officials to use lateral transfers to punish officers.⁴⁰ The career of Ashok Khemka, an IAS officer who joined the Haryana cadre in 1991, is one famous case in point: for exposing endemic corruption across various state-government departments, he has been transferred 47 times in twenty-four years.⁴¹ For example, Khemka was transferred from Haryana’s transportation department to the ostensibly less important archaeology and museums department after making policy decisions that were in opposition to the interests of the politically important so-called transport lobby.⁴² Due to the looming prospect of being transferred, bureaucrats are susceptible to political pressure in the execution of their daily responsibilities.⁴³

Poor Incentives for Advancement

Many observers—including many current and former officers—have questioned whether the rules governing advancement in the IAS are allowing the best and the brightest to move up in the ranks. For starters, the bias toward seniority in filling key posts reduces the ability of high-performing officers to swiftly obtain promotions, while protecting poorly performing officers who have more years of service under their belts. The empanelment process, through which officers are selected for service in the central government, is highly opaque and can be influenced by the judgments of politicians, who might wish to derail officers who cross them.⁴⁴

Lack of Specialization

In addition, some experts have questioned whether the IAS can continue to exist as a generalist service in a world that is increasingly complex and where domain knowledge has become more valuable. The frequent rotation that officers experience in the service means that they are constantly developing new skills and new expertise but very rarely stay in one field or sector long enough to become genuine experts.⁴⁵

Malfeasance

Taken together, several of the factors listed above are major drivers of malfeasance in the service. Endemic political interference can lead to rent-seeking behavior even for honest officers, who might feel forced to comply with questionable demands from superiors for fear of being punished. Furthermore, uncompetitive public-sector salaries (not to mention years of foregone wages as candidates devote an increasing amount of time to passing the civil services exam) encourage officers to make extra money while in office.

In the 2010 survey mentioned previously, 78 percent of IAS respondents believed some or most officers used influence to secure coveted positions, while 62 percent thought some or most officers indulged in nepotism.⁴⁶ A former director of the Central Bureau of Investigation (CBI), commenting on a recent spate of investigations and arrests of senior IAS officers, bemoaned “the escalation of corruption from the bottom of the bureaucratic hierarchy to its higher echelons.”⁴⁷ According to a statement released by the Modi government, between January 2012 and April 2015 the CBI opened as many as 74 cases against IAS, IPS, and Indian Revenue Service officers for allegedly violating the Prevention of Corruption Act.⁴⁸

OPENING THE BLACK BOX

A reform agenda for the IAS must seek to resolve the perverse incentive structures that riddle the top functionaries of the Indian state. For the first time, thanks in part to advances in the collection and analysis of big data, scholars have unprecedented access to detailed information on the career profiles of IAS officers. This information, when combined with fine-grained data on development indicators and electoral and political dynamics, provides significant new insights on bureaucratic performance.

A reform agenda for the IAS must seek to resolve the perverse incentive structures that riddle the top functionaries of the Indian state.

The growing empirical literature on the effectiveness of the IAS comprises three broad lines of questioning. First, what determines the upward mobility of IAS officers in the service, thereby shaping career outcomes? Second, what impact can individual bureaucrats have on actual development outcomes? Third, how do politicians and bureaucrats interact while in power, and how does this impact development on the ground? (See table 4 for a summary of the studies and their important attributes.)⁴⁹

Determinants of Upward Mobility

The first line of inquiry examines the determinants of career success in the IAS. The term success here refers strictly to the career advancement of individual officers, as opposed to their impact on tangible development outcomes.

A key predictor of future professional success is an IAS candidate’s entrance exam performance and post-entry training scores. Combining cross-sectional data on subjective assessments of IAS officers from a wide range of societal stakeholders with detailed information about postings and pay scales of more than 5,600 IAS officers throughout their careers, Marianne Bertrand and her fellow researchers examined how predetermined characteristics of officers at the recruitment stage—age, caste, and exam scores (including both the entrance- and post-entry training exams), among others—correlate with officers’ perceived effectiveness. To establish an objective measure of performance, the researchers asked a diverse group of stakeholders—from state civil servants to politicians and journalists—to rate officers on five dimensions: effectiveness, probity, ability to withstand political pressure, responsiveness to the interests of poor citizens, and an overall summary rating.⁵⁰

There is a highly robust, positive correlation between officers’ scores on the IAS entrance exam and both their future investments in professional training and subjective performance ratings. Interestingly, stakeholders more positively assessed those officers who demonstrated the most improvement in their training compared with their baseline performance on the IAS exam.

It was also the case that officers who were older and entered the IAS as part of a large cohort exhibited lower effectiveness, according to the study’s subjective measures. Age serves as an impediment because older officers will be too old by the time jobs at the highest pay scale open up, at which point the competition for coveted jobs will be even more intense than usual. Larger cohort sizes also make upward mobility more difficult because they imply greater competition for promotions. The interaction between the two characteristics appears especially problematic: older officers in larger cohorts are significantly more likely to face delays in promotions and to be the subjects of official suspensions.⁵¹

Some of these findings are similar to those in a 2013 study by John-Paul Ferguson and Sharique Hasan, who used the records of more than 3,000 IAS officers to examine the impact of specialization on achieving early- and late-career milestones like postings to the central government and empanelment. Specialization was defined as the number of months spent working in a specific domain such as defense, finance, or transportation. Controlling for a host of individual-level characteristics (such as age, education, gender, and tenure) as well as political factors (like changes in party control in each state), junior-level officers with an above-average specialization score (defined as one standard deviation above the mean) were 36 percent more likely to receive a coveted posting with the central government in New Delhi. At earlier stages of their careers, officers are rewarded for their specializations because they signal ability and future potential.

However, there is no systemic match between accumulated experience and postings officers receive; in other words, specializing in a field does not raise the likelihood of working in that field at the center (finance is one notable exception).⁵² An officer’s prior educational performance—whether he or she graduated in the first division of an undergraduate class and possesses multiple academic degrees—remains a robust predictor of earning a posting with the central government in New Delhi.

With regard to empanelment, a late-career milestone, there was a positive and statistically significant relationship between accumulated experience and post-empanelment job offers. Officers with an above-average level of specialization were 43 percent more likely to become joint secretaries—a senior position with direct oversight of a specific governmental department. At this later stage of officers’ careers, however, specialization matters not for signaling reasons but because of domain-specific skill accumulation. As the authors wrote, “late in a career, more specialization is rewarded because it reflects specific skills.”⁵³ When it comes to being empaneled, as with winning early-career postings to the central government, educational performance was also linked with higher success rates.⁵⁴

These two distinct mechanisms—signaling and accumulating skills—are plausibly connected. If an officer is rewarded early on in his or her career for specialization, even if it has little to do with any specific domain knowledge, that officer has incentives to double down on specialization—which is rewarded for its intrinsic value at a later stage.

These findings suggest that the oft-heard notion that early-career officers have no incentive to acquire knowledge or improve skills in a given domain or area of expertise is not entirely accurate; those who do acquire and cultivate specific domain knowledge are rewarded for doing so.⁵⁵ On this point, an officer’s performance on the civil service entrance exam (a proxy for quality) is highly predictive of his or her future career potential. Officers of higher initial ability, as determined by their performance on the entrance exam, are more likely to invest in training and professional development (especially foreign training) over the course of their careers and, in turn, are more likely to be recommended for empanelment down the road.⁵⁶

Bureaucratic Influence on Development Outcomes

The second line of inquiry relates to the tangible impact individual IAS officers can have on development outcomes in their areas of operation.

An efficient bureaucracy matters for economic performance. For every IAS officer in their sample, Bertrand and her colleagues calculated a “predicted effectiveness” score using a combination of individual and organizational-level characteristics.⁵⁷ This comprehensive measure of predicted effectiveness of senior IAS officers was positively associated with per capita state-level gross domestic product (GDP) and industrial growth. Predicted effectiveness was also positively associated with higher total annual public revenue. Interestingly, higher revenue was not driven by improved taxation; rather, it was the result of increases in nontax revenue sources (such as dividends and profits from public-sector enterprises) and grants comprising major funding schemes from the central government—all activities supervised by senior IAS officers.

The service’s arcane bureaucratic rules also can have material impacts. A one standard deviation increase in the average age at entry was associated with a 10.6 percent lower state-level GDP per capita; the impact increased by another 4 percent if the cohort size increased by one standard deviation.⁵⁸

One of the biggest debates in the comparative thinking on bureaucracy is the virtue of embeddedness, or the strength of local ties.⁵⁹ Proponents argue that bureaucrats must be locally embedded (typically, native to a given area) if they are to be truly effective. After all, local officers are more likely than those from other parts of the country to be close to the population they serve and able to use their knowledge of language and culture to work well with local stakeholders. On the flip side, detractors argue that officials who are too closely intertwined with the local community only fulfill the policy priorities of elites or exclude the broader community from key public goods and services.

Data suggests bureaucrats with strong local ties to their communities often outperform outsiders when it comes to delivering public goods. In a 2015 paper, Rikhil Bhavnani and Alexander Lee used data on nearly 4,800 serving IAS officers (as of March 2007) to examine whether locally embedded bureaucrats—those IAS officers serving in their home state (known as their state of domicile)—enhanced service delivery between 1991 and 2001, as measured by the proportion of villages in a district with high schools.⁶⁰ A one standard deviation increase in the proportion of local IAS officers was linked to a 4.6 percent increase in the proportion of villages with public high schools. IAS officers’ early career postings in their cadre states are largely apolitical, which means that the analysis did not have to account for unobserved forces driving personnel assignment.⁶¹ The researchers studied access to public high schools, rather than elementary schools, due to concerns of ceiling effects: most villages had access to elementary schools in 1991, and there was little incentive for the government to keep building more of them. Interestingly, embeddedness has no discernible impact on the provision of roads and phones, responsibility over which lies not with the district administration but with parastatal organizations, which are publicly owned but privately managed entities in charge of providing public goods and services.

Bureaucrats with strong local ties to their communities often outperform outsiders when it comes to delivering public goods .

However, the story does not end there; the authors also tested for variation in the impact of embeddedness. It is still possible that there are areas where typical mechanisms of local accountability are ineffective and, hence, bureaucrats are more likely to be susceptible to elite capture. The data suggests that embeddedness was associated with more high school construction—but only in districts with high literacy and large vernacular newspaper circulation (and, hence, greater accountability). The presence of these two factors allows the local populace to better monitor government actions. In districts with low newspaper circulation and literacy, the converse is true: embeddedness had no impact on high school construction. It stands to reason that in the latter environment, where the local populace cannot effectively hold officers accountable, the threat of corruption looms much larger. Interestingly, the positive impacts of embeddedness go beyond facility with the local language or local political connections, suggesting deeper—possibly cultural—advantages.⁶²

Finally, individual IAS officers have a moderately large positive impact on district-level economic outcomes. In a 2015 study, Jonas Hjort, Gautam Rao, and Elizabeth Santorella adopted methodologies developed in education literature (for instance, to quantify the value added of teachers on individual student learning outcomes) and in the field of labor economics (intended to measure worker impact) and applied them to the study of district collectors in India. Based on this value-added methodology, an individual IAS officer could explain up to 2 percent of variation in the outcomes of investment projects in his or her district and roughly 0.4 percent of variation in nighttime luminosity (which is often used as a proxy for local economic activity).⁶³ These are very sizable effects.

Because the scholars also had details on the individual characteristics of district collectors, they could unpack the correlates of better bureaucratic performance. District collectors with better past educational performance (that is, first-class honors in their highest completed degree) were more likely to deliver better outcomes. Similarly, IAS officers who could speak a state’s official language also exhibited better performance, on average.

Politician-Bureaucrat Dynamics

The third and final pillar of new research on the IAS disaggregates the impact of elected officials on the bureaucracy.

The most visible and lamentable aspect of political interference in the civil service has been the phenomenon of punitive transfers.

The most visible and lamentable aspect of political interference in the civil service has been the phenomenon of punitive transfers. In a 2012 article, Lakshmi Iyer and Anandi Mani used the career histories of 2,800 IAS officers—combined with data on political changes, proxy measures of bureaucrat ability, and a measure of the perceived importance of different IAS posts—to show how politicians use frequent reassignments to pressure bureaucrats. There appear to be two major sources of bureaucratic inefficiency. First, because politicians seek to exercise a degree of control over civil servants, important bureaucratic positions are not necessarily filled by the most qualified officers available (as judged by their initial ability). Second, junior IAS officers systematically underinvest in skill acquisition because loyalty to powerful politicians, as opposed to merit-based advancement, offers an alternative path to career success.

The extent of what is often referred to as the Transfer-Posting Raj is extraordinary. The probability that an IAS officer would be transferred in a given year was 53 percent, and this is increased by 10 percent when a state elects a new chief minister. The average tenure of an IAS officer in any given post was a mere sixteen months, which stands in contrast to recommendations of various expert committees that have argued for fixed tenures as long as five years. Bureaucrats ranking among the top twenty in their cohort were 2.2 percentage points less likely to be transferred after the election of a new chief minister (and significantly more likely to be empaneled later in their career). Being of the same caste as the core constituency of the chief minister’s political party increased an officer’s probability of obtaining an important post by 6.6 percentage points.⁶⁴

Taken together, this evidence outlines two divergent paths to moving up in the bureaucratic hierarchy: an officer can either invest in expertise or leverage his or her caste affinity to secure important positions. Does one path lead to more success overall? There is no evidence to suggest this is the case: the average importance of posts held by officers through their career varies little with initial ranking, irrespective of which track they choose.⁶⁵

With regard to the impact on economic development, in places where the probability of an officer being transferred increased by 10 percentage points, poverty rates exhibited a much slower pace of decline than in other districts—suggesting lasting damage to policy outcomes. These results, the authors emphasized, should be treated as suggestive because there could have been some unobserved factor(s) influencing both transfers and development outcomes.⁶⁶

Yet another study, authored by Anusha Nath in 2015, focused exclusively on the impact of political competition on a bureaucrat’s ability to implement development activities. The author argues that electoral competition has a counterintuitive impact on bureaucratic outcomes. Whereas a good deal of theory predicts that electoral uncertainty leads to better governance outcomes because politicians are worried about losing reelection bids if they do not perform, Nath posits the opposite: bureaucrats are more incentivized to do their job when it is almost certain that the political incumbent will be brought back to power.⁶⁷

Nath’s analysis compiled professional histories of all IAS officers serving as district collectors between 1999 and 2009, data on the implementation of projects executed using constituency development funds allocated to members of parliament (MPs), and official election returns.⁶⁸ Nath’s primary measure of bureaucratic performance was the time it took district collectors to sanction projects MPs propose to be built with money from their discretionary funds. Although MPs can propose small public works projects and use earmarked funds to finance their construction, it is the district administration—led by the district collector—that has to undertake the work. This gives collectors an important degree of power; they can speed up (or slow down) the pace of development projects—at least to a certain extent—based on their preferences.

In constituencies where incumbents were prohibited from standing for reelection (because their seats had been reserved for ethnic minorities by an independent redistricting, or delimitation, agency), the average time it took for a district collector to sanction an MP’s proposed project increased by 13 percent. The agency’s decision to change the reservation status of a given parliamentary constituency in the following election occurred midway through MPs’ terms, which makes it a reasonable exogenous shock.⁶⁹

Conversely, in constituencies that are party strongholds (that is, where reelection for a politician belonging to the incumbent party is virtually guaranteed based on its track record over the past four election cycles), the district administration approved projects 11 percent faster than average. Additionally, district collectors were more effective in implementing projects when they were eligible for promotion and when the incumbent politician was likely to remain in power.⁷⁰ In short, where there is greater political certainty, the bureaucracy performs better.

This finding closely tracks Iyer and Mani’s insight that bureaucratic transfers exhibit a spike in the aftermath of political turnover. As electoral pressure diminishes, a virtuous cycle is initiated whereby politicians incentivize bureaucrats with future postings and civil servants exert more effort into approving development projects. This is not merely a result of politicians selecting better-performing bureaucrats to begin with; because electoral and administrative boundaries do not perfectly overlap, Nath was able to measure how district officers responded differentially to multiple politicians overlapping with his or her given district.

This work begs the question: when do politicians want to put effort into incentivizing bureaucrats? This puzzle awaits further research, but a forthcoming paper by Saad Gulzar and Benjamin Pasquale offers one plausible narrative. The authors used an original data set of nearly 500,000 villages where the National Rural Employment Guarantee Scheme (NREGS) operates to compare officers supervised by a single politician with those supervised by several politicians. Specifically, the authors compared NREGS outcomes (the number of villagers who worked in NREGS and the average number of days worked) in villages whose block administration is split across two politicians with other villages whose block administration is not split.

The study found that split blocks employed fewer individuals in NREGS, who in turn received fewer workdays than their counterparts in unsplit blocks. Further, these results were driven by a specific free-rider problem: if a politician faces a higher marginal cost of effort, as happens when split blocks in his or her constituency are shared with politicians from the same party, development outcomes worsen. The same is true when the marginal benefit of a politician’s effort increases, for instance when the political importance of an area grows.

Politicians are therefore incentivized to motivate bureaucrats only when the benefits are internalized. As the authors suggested: “Politicians realize that large development programs offer them an important opportunity to earn favor with voters. Development program designs that help politicians claim credit will strengthen democratic accountability and improve service delivery.”⁷¹

MARGINAL REVOLUTION

Given the concerns dogging the IAS, calls for reform are all too commonplace, especially in New Delhi. And there is no shortage of ideas about how best to proceed. Reform ideas literally run the gamut.

Some analysts have called for doing away with the IAS entirely. For instance, journalist Mihir Sharma has argued for abolishing the IAS on the grounds that an unaccountable and misinformed bureaucracy based on the Whitehall model simply cannot administer a twenty-first-century state.⁷²

While there might be merit to scrapping the system and beginning with a clean slate, as opposed to pursuing a strategy of gradual updating and renewal, public institutions are notoriously sticky and path dependent. Furthermore, replacing local institutions with idealized versions of Western best practices is extremely risky, especially when such reform fails to address underlying social inequalities.⁷³ As one former IAS officer put it, tearing down and replacing a structure that connects villages to districts, districts to states, and—finally—states to the capital of India is no easy task.⁷⁴

This resistance to change is perhaps why many experts have suggested keeping the service intact but introducing a series of updates to its recruitment and overall operations. Many of these alterations can be found, in some form or fashion, in the various reports of the Second Administrative Reforms Commission, a major government-led initiative launched in 2005 to prepare a blueprint for overhauling the Indian bureaucracy. The commission was the latest in a long string of expert panels, dating back to the 1947 Secretariat Reorganization Committee, established by the government to propose civil service reforms.⁷⁵ Recognizing that “inefficiency, corruption and delays have become, in public perception, the hallmarks of public administration in India,” the commission released fifteen reports on various facets of governance, including undue political interference, inadequate accountability mechanisms, and capacity building.⁷⁶

Regarding recruitment, the commission recommended significantly lowering the permissible age of entry into the civil services and establishing national institutes of public administration that would cultivate a new pool of aspiring civil service applicants. In an attempt to engineer a shift away from seniority-based career progression, the commission also suggested that all promotions be based on successful completion of mandatory training.

Finally, to strengthen accountability mechanisms, the commission recommended a system of two intensive reviews at the fourteen- and twenty-year marks to determine continuance in public service, as well as a new civil service reform bill that would fix a minimum tenure for senior posts and establish safeguards against arbitrary dismissal.⁷⁷

The obstacles to even modest reform of this type, such as opening up senior management positions in the IAS to individuals from the private sector, are immense.⁷⁸ For instance, proposals to allow for lateral entry into the IAS have drawn withering criticism from current and retired civil servants, who have argued that infusing external talent into high-profile posts is likely to both affect incumbent morale and distort the incentives of new entrants.⁷⁹After initially raising hopes that it would resist opposition to infusing public service with more lateral entrants, the Modi administration has apparently relented. In December 2015, Minister of State for Personnel, Public Grievances, and Pensions Jitendra Singh clarified that the present government has no plans to pursue lateral entry into the IAS.⁸⁰ If the past is any guide, future governments will also move incrementally, if at all, on civil service reform given stiff resistance from incumbent IAS officers.⁸¹

In that spirit, the government would be wise to consider three broad areas in which to undertake incremental policy shifts: enacting legislation to prevent arbitrary transfers of personnel, making data-driven decisions on allocation and retention, and reexamining the potential benefits of increasing the number of local officers in state cadres.

Thwarting Political Interference

Political interference remains one of the biggest obstacles to bureaucratic effectiveness. Perhaps for the first time, researchers have drawn clear, quantifiable links between the pervasive abuse of the transfers and postings of civil servants and development outcomes.

One step the present government could take to rectify this situation is to prioritize action on a series of draft bills that place constraints on politicians’ ability to arbitrarily transfer bureaucrats. This pending legislation includes the Public Services Bill (2007), the Civil Services Bill (2009), and the Civil Services Standards, Performance, and Accountability Bill (2010), all of which have been languishing. In recent years, the only notable act of civil service reform has come not from parliament but from the judiciary; in 2013, the Supreme Court of India directed both the central and the state governments to establish civil service boards to manage the tenure, transfer, and posting of all officers in the All India Services.⁸²Unfortunately, the order has been widely perceived as toothless, because very few states have heeded the call to fix a minimum tenure of two years for civil servants.⁸³

Another idea, which has been mooted and deserves consideration, is to develop a stability index for key posts for which the average length of tenures must remain above a certain predetermined average (say, two years). This approach should allow for flexibility; while there might be good reasons for an individual officer to be transferred, on average such moves should be the exception rather than the rule.⁸⁴

Since coming to power in May 2014, the Modi government has taken steps to curb politicized transfers, although its moves have received mixed reviews. Some commentators have praised the new process instituted by the Prime Minister’s Office (PMO), whereby senior bureaucrats run background checks on all officers seeking postings to the central government with two criteria in mind: honesty and efficiency.⁸⁵ Critics, however, argue that centralizing power in the PMO does not bode well for an effective administrative machinery and point to frequent reshuffles at the joint-secretary level and falling numbers of officers willing to work at the center as evidence of this weakness.⁸⁶

Increasing Career Incentives

A second potential area for reform is the manner in which existing processes of recruitment and seniority-based career progression can introduce inefficiencies into the bureaucracy. The empirical finding that an individual officer’s initial score on the Civil Services (Main) Examination is highly predictive of future success appears to be fairly robust. Beyond initial exam scores, postrecruitment training (including improvement in training performance relative to an officer’s starting point) is also positively correlated with perceived effectiveness.⁸⁷ What this means is that there is useful information available about each civil servant’s general ability even before he or she enters the service after the probation period. Yet, these valuable data points are not systematically used in future decisions regarding retention or assignment to sector-specific positions.⁸⁸

Organizational features of the service that dictate career progression, such as those having to do with the rigid age windows around entry and exit and seniority-based promotions, can also have a measurable (often negative) impact on bureaucratic effectiveness. The older an officer is when entering the IAS and the larger his or her cohort, the less effective that officer is likely to be in the future. Furthermore, the assignment of senior officers at the joint-secretary level ought to ensure a strong match between the posting and specific skills that have been accumulated over time.

The recommendations of the Second Administrative Reforms Commission on selection to key leadership positions in the civil services are especially germane. Recognizing that the current system of empanelment suffers from a lack of transparency, the commission argued for a system of performance appraisal that privileged domain competence over subjective annual performance appraisal reports and made domain expertise a criterion for senior management positions of a technical nature. Additionally, the commission made the case for greater competition for positions at the joint-secretary level and above (in both state governments and the government of India) by opening them up to candidates from all senior administrative services, such as the Indian Economic Service, the Indian Revenue Service, and the Indian Information Service.

The body also favored opening up additional secretary (one rank above joint secretary) positions to qualified individuals from the private sector.⁸⁹The Modi government has taken a welcome step in this direction by restructuring the empanelment process. Previously, an expert committee would aggregate an officer’s annual personal appraisal reports (where outstanding grades were typical) for the preceding sixteen years—a system predicated on negative disqualification, or searching for reasons to drop candidates, rather than on considered selection based on affirmative criteria.⁹⁰ By introducing a comprehensive evaluation that ranks officers on their functional skills, domain expertise, behavioral competence, and integrity, the center seeks to eliminate ambiguity from the empanelment process and explicitly tie high job performance to moving up the career ladder.

Given that older officers entering the bureaucracy are perceived as less effective by internal and external stakeholders like civil society members, businesspeople, politicians, and other civil servants, reducing the maximum age of entry into the IAS is a relatively easy reform the government could introduce.⁹¹ Although the Second Administrative Reforms Commission recommended limiting the permissible age, the Department of Personnel and Training moved in the opposite direction, increasing the age limit for aspiring candidates in 2014.⁹² The agency made the switch despite the fact that two-thirds of all civil servants the government surveyed agreed that the maximum age of entry should be decreased.⁹³ Of late, however, both the government and the bureaucracy seem to have reached a consensus on the importance of lowering the upper-age limit. According to media reports, the Modi administration is likely to accept the recommendation of an August 2016 UPSC panel report to implement a phased reduction in the age limit for general, able-bodied candidates from thirty-two to twenty-seven years.⁹⁴ This small step not only improves the IAS’s human capital pipeline but also paves the way for organizational reform in the future.

There is also a case for reducing the overall number of IAS positions. Over time, the number of authorized positions has ballooned, often creating redundancies or multiple layers of bureaucracy, which further encumber decisionmaking (see table 5). Reducing the size of the individual cadres would also decrease the number of promotions, a step that is needed to ensure only the best officers reach the upper levels of the IAS ranks.⁹⁵ As one commentator has noted, the pyramid structure of promotions looks more like a cylinder because “75 percent of officers become joint secretaries and 40 percent reach the level of additional secretary.”⁹⁶

This begs the question of how to improve the process around promotion decisions. Just as data can help inform the initial assignment of officers in the service, they can also be of use in latter stages of officers’ careers. However, the likely benefits of shrinking the size of cadres must be balanced against the costs of creating disincentives for talented young people to join the service. After all, instituting an up-or-out system could adversely affect job security and stability.

Stepping back, it is outcomes that should drive government policymaking. The advent of big data, especially on concrete outcomes that can be traced to a specific officer’s time in a given post, opens up wide-ranging possibilities for performance-based evaluation and promotion. Seniority is a blunt instrument for deciding who gets promoted and who does not, especially when fine-grained data are now readily available. The choice of the word outcomes here must be emphasized; governments all over the world, including India’s, typically track expenditure (such as education funds spent) and defined outputs (number of teachers trained or school buildings constructed), but few have made the jump to outcomes (reading skills of a third grader) that more closely get at issues of quality.⁹⁷

Data-driven performance metrics could not only be used for promotions, but they also could help guide salary and remuneration decisions. There is a growing literature about performance-based pay for public-sector workers, and while the jury is still out about the effectiveness of such schemes, limited experimentation is certainly worth pursuing.⁹⁸ To be clear: data need not be the only criterion on which officers are judged. However, data could be one critical component.

There is the final pesky issue of what to do with perennially underperforming officers. While the government can adopt smarter methods for ensuring that the best officers are selected, promoted, and placed in the right jobs, it must also find creative ways of dealing with poor performers. The Second Administrative Reforms Commission recommended that within the framework of a new civil services law, the government institute a new policy whereby all officers who are deemed unfit for service at the time of their twenty-year review be forcibly retired.

Neither the Singh government nor the Modi administration embraced this suggestion, but the latter has taken new steps to crack down on poor performers. According to media reports, the Department of Personnel and Training has begun systematically reviewing the performance of central officers who have either completed thirty years of service or reached fifty years of age. Those officers who receive negative reviews, the reports suggest, are to receive a notice that their services will be terminated within three months.⁹⁹ At the end of 2015, the Modi government disclosed that it had dismissed or compulsorily retired thirteen bureaucrats for unsatisfactory performance.¹⁰⁰ This process of dismissing officers who are negatively rated at predictable career benchmarks should be institutionalized so that it does not rest on the preferences of any one government but becomes a transparently enforced and embedded rule.

Allocating Officers to Home States

A third reform is somewhat counterintuitive: there might be unexpected benefits from allocating a higher percentage of junior officers to their states of domicile, or home states.¹⁰¹ This proposal flies in the face of some of the original arguments to the contrary made by India’s founders. In a speech to the Constituent Assembly in 1949, then home minister Sardar Vallabhbhai Patel highlighted the IAS’s role in encouraging center-state harmony, claiming that “you will not have a united India, if you have [not] a good all-India service which has independence to speak out its mind.”¹⁰² In subsequent decades, political scientists have echoed these claims, contending that the current structure of the IAS plays “a key role in generating all-India loyalties.”¹⁰³ The notion of altering the insider-outsider ratio goes against conventional wisdom among senior bureaucrats, many of whom contend that local officers are susceptible to capture by their personal network, while outsider officers with no such stakes tend to perform better.

The architects of the IAS may have been right to distrust bureaucrats with strong local ties. Yet it is possible that the widespread prevalence of accountability mechanisms in contemporary India—in the form of growing social, television, and print media circulation and rising literacy levels—may minimize the threat of capture by vested interests. While a change to the national cadre allocation policy is unwarranted at this stage, reform-minded state cadres could experiment with increasing the number of local IAS officers and closely tracking their impact on development outcomes relative to other bureaucrats.

AVENUES FOR FUTURE RESEARCH

While the primary research findings, taken in aggregate, suggest potential reforms to policies on recruitment, training, career advancement, and transfers, important gaps remain. First, additional research is needed to further test the hypothesis that local officers generate better development outcomes than outsiders. The evidence to date suggests that embedded IAS officers have a tangible, positive impact in areas where strong accountability mechanisms are present. However, a concern many bureaucrats share about increasing the proportion of domiciled officers is that they lack the broad national outlook possessed by officers assigned to states other than those to which they belong. Scholars should explore whether the positive relationship between insider status and development outcomes holds as officers are promoted to posts in state governments or to postings in New Delhi.

Second, woefully little is known about bureaucratic efficiency at the most senior levels of management. This is arguably when productivity matters the most, because senior civil servants are in charge of state- or central-government departments toward the end of their careers. A slew of news reports have documented the phenomenon of retired IAS officers being appointed to official bodies and administrative tribunals.¹⁰⁴ This trend has led some scholars to voice concern about a “sinecure state,” in which senior IAS officers modulate their performance in their final years of service in the hopes of winning a plum postretirement assignment.¹⁰⁵

This conjecture opens up several opportunities for future research. For example, is there a systematic increase in senior bureaucrats assuming postretirement postings? What impact, if any, does this behavior have on bureaucratic efficiency during officers’ final months and years in office? The answers to these questions are of significant importance in determining whether new rules regarding postretirement government employment should be contemplated.

Third, little is understood about the workings of the state-level bureaucracies, the variation among them, and their impact on development and governance. There has been little systematic research into these issues, despite the fact that states are today the prime venues for political competition, economic policymaking, and governance writ large. Researchers do have hunches worth exploring. The conventional wisdom is that the quality of bureaucrats from the state services is lower than in the IAS. In the words of political scientist Devesh Kapur, “if there are questions about the competencies, integrity and political pressures on the IAS, these are likely to be considerably greater in the case of the PCSes.”¹⁰⁶ But there is likely to be considerable variation across states. Scholar Atul Kohli has remarked that the quality of state-level bureaucracy in southern India has generally been superior to that delivered in the north. Kohli qualifies this statement, writing, “I hesitate in asserting this ‘fact’ because, to the best of my knowledge, it has not been documented by scholarly research; comparison of state level bureaucracies across India is crying out for further research.”¹⁰⁷ Comparative analyses of state-level bureaucracies—not to mention an examination of the interaction between the IAS and the state civil services—are ripe for deeper exploration.

CONCLUSION

The challenges facing the Indian state in the twenty-first century are immense. The country’s fundamentals—a young and growing workforce, a virtually unprecedented urban transition, and a domestic marketplace with seemingly infinite potential—should positively influence its quest to fulfill this promise and sustain high rates of growth. However, India does possess one significant Achilles’ heel: the quality of its public-sector institutions.

Any serious reform program for civil administration must address the infirmities of the core bureaucracy. Although the IAS represents a small share of the overall administrative apparatus, given its control over executive positions at all levels—local, state, and national—it is a critical component. For the first time, thanks to a new body of literature that leverages big data with cutting-edge statistical methodologies, there is rigorous evidence to help inform reform discussions. While the solutions implied by the data are not revolutionary, they have the virtue of being based on solid evidence.

As the obstacles facing India’s transition to a middle-income economy grow in size and complexity, the country’s policymakers cannot let institutional lethargy get in the way of efficient policy implementation. A modern Indian state requires an administrative apparatus that encourages and recognizes productive high performers, ensures political buy-in within the policymaking process, and values genuine innovations in service delivery over an unquestioning adherence to hierarchy and procedure.

NOTES

¹ Growth figures, measured in constant prices, are sourced from the Reserve Bank of India. Poverty data, using a poverty cutoff of $1.90 per day, comes from the World Bank. For an explanation of the latter, see Rukmini S., “Poverty Is Falling Fast in India, but We Still Measure It Terribly,” Hindu, October 28, 2015, http://www.thehindu.com/data/poverty-is-falling-fast-in-india-but-we-still-measure-it-terribly/article7810119.ece.

2 International Monetary Fund, World Economic Outlook: Too Slow for Too Long (Washington, D.C.: International Monetary Fund, 2016).

³ Gurcharan Das, India Grows at Night (London: Penguin UK, 2013). Even a shock like the landmark economic reforms of 1991 did little to fundamentally alter the nature of India’s slothful state. As economist Lant Pritchett has argued, market liberalization efforts of the early 1990s were “administrative capability saving” reforms in which the state simply stopped trying to perform extraneous functions. See Lant Pritchett, “Is India a Flailing State? Detours on the Four Lane Highway to Modernization,” Harvard Kennedy School Working Paper RWP09-013, May 13, 2009.

⁴ The government effectiveness indicator combines the views of a large number of survey respondents—enterprises, citizens, and experts—across countries over the period 1996–2014. It is based on over 30 individual data sources produced by a variety of survey institutes, think tanks, nongovernmental organizations, international organizations, and private-sector firms. As a benchmark, in 2014 high-income countries of the Organization for Economic Cooperation and Development ranked in the eighty-eighth percentile on the government effectiveness indicator, on average. For more detail, see World Bank, “Worldwide Governance Indicators,” http://info.worldbank.org/governance/wgi/index.aspx#home.

5 The outstanding service of officers like C.D. Deshmukh, an ICS officer and the first Indian governor of the Reserve Bank of India, and S.R. Shankaran, a 1958-batch IAS officer in the Andhra Pradesh cadre who fought against bonded labor and atrocities against Dalits, have become the stuff of legend. Several reports by international development organizations on successful innovations in service delivery have recognized the pivotal role IAS officers have played in implementing the Indian government’s key development and economic priorities over the past seven decades. See World Bank, Reforming Public Services in India: Drawing Lessons From Success (Washington, D.C.: World Bank, 2006); also see NITI Aayog and the United Nations Development Program, Social Sector Service Delivery: Good Practices Resource Book (New Delhi: Government of India, 2015).

⁶ The data on the total number of public-sector officials in India are current as of January 1, 2014. See Government of India, Report of the Seventh Central Pay Commission (New Delhi: Government of India, 2015), 23. The data on IAS officers are current of January 1, 2015. See Government of India, National Informatics Center, “Cadre Strength of Indian Administrative Service,” http://civillist.ias.nic.in/YrCurr/PDF/AppendixA.pdf.

⁷ The widespread use of the term babu is a perennial grievance among IAS officers. As of mid-2016, several tweets by the IAS (Central) Association admonished national newspapers and television channels for their use of the term. A number of Facebook posts by serving officers with large social media followings also expressed disappointment.

⁸ Throughout this paper, we use the term central rather than union government to refer to the government in New Delhi—even though the latter is the officially recognized term. However, central government is the commonly accepted designation outside of India.

⁹ The origins of the oft-quoted “steel frame” reference can be found in a speech then British prime minister David Lloyd George delivered to the British Parliament in 1922 on the subject of the Indian Civil Service: “If you take that steel frame out, the fabric will collapse. . . . There is one institution we will not cripple, there is one institution we will not deprive of its functions or of its privileges, and that is that institution which built up the British Raj—the British Civil Service in India.” See House of Commons Debates, vol. 157, columns 1495–1525, August 2, 1922, http://hansard.millbanksystems.com/commons/1922/aug/02/civil-service-india.

¹⁰ Uttam Sengupta, “Shaking Up the Frame,” Outlook, June 16, 2014, http://www.outlookindia.com/magazine/story/shaking-up-the-frame/290983.

¹¹ Manmohan Singh, “Prime Minister’s Address to the Nation,” June 24, 2004, http://archivepmo.nic.in/drmanmohansingh/speech-details.php?nodeid=1.

¹² Sandeep Unnithan and Kumar Anshuman, “Yes, Prime Minister,” India Today, July 3, 2014, http://indiatoday.intoday.in/story/narendra-modi-yes-prime-minister/1/369923.html. In his previous role as chief minister of the state of Gujarat, Modi had cultivated a reputation for relying heavily on the state’s elite civil servants, rather than ministerial colleagues, to implement his flagship programs. The gathering held in the first week of his tenure as prime minister suggested that Modi planned to steal a page from his Gujarat playbook to guide his governing approach in New Delhi.

¹³ The IAS has a staff association that regularly advocates on behalf of officers’ interests. For example, in advance of the report of the Seventh Central Pay Commission, which sets central government salaries, the association vociferously argued against the notion of establishing pay parity between the IAS and other government services. See Remya Nair, Mayank Aggarwal, and Yogendra Kalavalapalli, “IAS Officers Get Pay Commission Jitters,” Mint, October 30, 2015, http://www.livemint.com/Politics/70YQ66cahp5KID17OyOZWI/IAS-officers-get-pay-commission-jitters.html.

¹⁴ Jawaharlal Nehru, a central figure in India’s freedom struggle, wrote a series of letters to his daughter Indira while imprisoned by British authorities from 1930 to 1933. In one of the 196 letters, India’s future first prime minister wrote: “Someone—I think it was Voltaire—defined this ‘Holy Roman Empire’ as something which was neither holy, nor Roman, nor an empire. Just as someone else once defined the Indian Civil Service, with which we are unfortunately still afflicted in this country, as neither Indian, nor civil, nor a service.” Jawaharlal Nehru, Glimpses of World History (New Delhi: Penguin, 2004).

¹⁵ The remaining 15 percent were judicial officers, working permanently as judges in districts or as high court justices. David C. Potter, India’s Political Administrators: 1919-1983 (New York: Oxford University Press, 1986), 21.

¹⁶ In comparison, political appointees fill the deputy-secretary, undersecretary, assistant-secretary, and (approximately half of all) deputy-assistant-secretary positions in the U.S. bureaucracy. See Edward Page, “Has the Whitehall Model Survived?” International Review of Administrative Sciences 76, no. 3 (September 2010): 407–423.

¹⁷ Population data are taken from the 1931 census. For more details, see Government of India, Ministry of Home Affairs, “Census Reports 1931,” http://www.censusindia.gov.in/Census_And_You/old_report/Census_1931n.aspx.

¹⁸ Anirudh Krishna, “Continuity and Change: The Indian Administrative Service 30 Years Ago and Today,” Commonwealth and Comparative Politics, no. 4 (November 2010): 434.

¹⁹ Prior to 1922, any Indian seeking to enter the ICS would have to travel to London to sit the annual competitive examination. See Potter, India’s Political Administrators, 83.

²⁰ David Gilmour, “The Ruling Caste,” Asian Affairs 37, no. 3 (2006): 312–319.

²¹ Arudra Burra, “The Indian Civil Service and the Nationalist Movement: Neutrality, Politics and Continuity,” Commonwealth & Comparative Politics 48, no. 4 (November 2010): 404–432.

²² As Arudra Burra argues, “for [Sardar] Patel, the fact that the ICS was a loyal civil service to the Raj was precisely [italics in original] what made Indian ICS officers useful to the new state. Their loyalty was proof that their allegiance was to the state irrespective of its political colour.” See Burra, “The Indian Civil Service and the Nationalist Movement,” 427. This is not to say that India’s postindependence leaders did not harbor skepticism about the true nature of the ICS. Prior to independence, Jawaharlal Nehru accused the ICS of having “built up a caste which is rigid and exclusive. Even the Indian members of the service do not really belong to that caste.” See Bidyut Chakrabarty, “Jawaharlal Nehru and Administrative Reconstruction of India: A Mere Limitation of the Past or a Creative Initiative?” South Asia: Journal of South Asia Studies, no. 1 (April 2006): 83.

²³ Beryl A. Radin, “The Indian Administrative Service (IAS) in the 21st Century: Living in an Intergovernmental Environment,” International Journal of Public Administration, no. 12–14 (December 2007): 1,527.

²⁴ Quoted in S.R. Maheshwari, Indian Administration (New Delhi: Orient Longman, 1984), 211.

²⁵ Additionally, there are the Central Civil Services and the State Civil Services. The former belong exclusively to the central government and consist mainly of technical organizations like the Indian Postal Service and the Indian Foreign Service. The latter account for the bulk of the bureaucracy at the subnational level. State civil servants typically work under the IAS in the states, whose officers occupy the most consequential positions in government.

²⁶ K.P. Krishnan and T.V. Somanathan, “Civil Service: An Institutional Perspective,” in Public Institutions in India: Performance and Design, eds. by Devesh Kapur and Pratap Bhanu Mehta (New Delhi: Oxford University Press, 2005).

²⁷ After gaining acceptance to the IAS, officers are assigned to a cadre for their entire career and in accordance with a complicated set of allocation rules that is intended to ensure the even distribution of talent across states. Each Indian state corresponds to a cadre, although there are three joint cadres for groups of smaller states: Assam and Meghalaya; Manipur and Tripura; and Arunachal Pradesh, Goa, Mizoram, and the Union Territories. For details, see Government of India; Ministry of Personnel, Public Grievances, and Pensions; Department of Personnel and Training, Cadre Allocation Policy for the All India Services, Office Memorandum No. 13011/22/2005-AIS (I) (New Delhi: Government of India, April 10, 2008).

²⁸ The Central Civil Services are classified further into Group A, B, C, and D services. The UPSC conducts the recruitment process for the All India Services and Group A and B services. The Staff Selection Commission recruits entry-level officers to Group C and D services, while individual State Public Service Commissions conduct the hiring process for state civil servants.

²⁹ The IAS has instituted several reservation-based affirmative action policies for members of the Scheduled Castes (SCs), the Scheduled Tribes (STs), and the Other Backward Classes (OBCs) communities. According to the 2011 census, SCs and STs comprised 16.6 and 8.6 percent of India’s population, respectively. While estimates for OBCs vary (the last publicly available caste census dates back to 1931), recent data from the sixty-sixth round of the National Sample Survey (2009–2010) indicates that OBCs comprise approximately 41 percent of the population. The UPSC has mandated quotas of 15 percent, 7.5 percent, and 27 percent respectively for members of these three groups. See Government of India, Ministry of Personnel, Public Grievances, and Pensions, Department of Personnel and Training, “Brochure on Reservation for Scheduled Castes, Scheduled Tribes and Other Backward Classes in Services,” Office Memorandum No.A36011/1/2013-Estt(Res) (New Delhi: Government of India, January 23, 2014); and R. Ravikanth Reddy, “UPSC Notifies Civil Services Exam,” Hindu, May 24, 2015, http://www.thehindu.com/news/national/upsc-notifies-civil-services-exam/article7239777.ece. The policy of reservation has made the IAS a more representative body and more in sync with the Indian populace at large; yet, there is currently no systematic evidence of its impact (positive or negative) on performance. One recent study looked at the impact of reservations on productivity among Indian railway employees. While the study did not explicitly look at the IAS, the results were suggestive. The authors found no evidence that reservations reduced productive efficiency. See Ashwini Deshpande and Thomas E. Weisskopf, “Does Affirmative Action Reduce Productivity? A Case Study of the Indian Railways,” World Development 64 (December 2014): 169–180.

³⁰ These officers, called promotees, undergo an eight-week training program at the Lal Bahadur Shastri National Academy of Administration and various administrative training institutes across India.

³¹ The performance appraisal report (PAR) system replaced the controversial annual confidential report (ACR) system after the Second Administrative Reforms Commission criticized the latter as representing a supervisor’s subjective opinion. The PAR takes into account a variety of indicators, such as personal attributes, functional competence, and work output to arrive at an overall grade between 1 and 10 for every officer. Government of India, Prime Minister’s Office, Press Information Bureau, “Evaluating IAS Officers: PAR to Replace ACR,” May 7, 2005, http://pib.nic.in/newsite/erelcontent.aspx?relid=9096.

³² Akhileshwar Prasad Singh, “The Changing Role of Collector and District Magistrate,” Indian Journal of Political Science 55, no. 2 (April–June 1994): 167. In Upamanyu Chatterjee’s fictionalized account of the IAS, one character describes the power of district collectors as the ability to “play God over 17,000 square kilometers.” Upamanyu Chatterjee, English, August (New Delhi: Penguin, 1988): 38–39.

³³ Roughly two-thirds of all IAS officers typically receive postings to the central government. See John-Paul Ferguson and Sharique Hasan, “Specialization and Career Dynamics Evidence From the Indian Administrative Service,” Administrative Science Quarterly 58, no. 2 (June 2013): 6.

³⁴ Pratap Bhanu Mehta, “Our Bureaucracy, Our Selves,” Indian Express, June 5, 2009. The Hong Kong–based Political and Economic Risk Consultancy surveyed about 1,200 investors across Asia and labeled India’s bureaucracy the least efficient, calling civil servants “a power centre in their own right at both the national and state levels, and . . . extremely resistant to reform that affects them or the way they go about their duties.” See “Singapore Bureaucracy Best in Asia, India Worst – Survey,” Reuters, June 3, 2009, http://in.reuters.com/article/idINIndia-40062020090603.

³⁵ Government of India, Ministry of Personnel, Public Grievances, and Pensions, Press Information Bureau, “Civil Services Examination, 2015 – Result Declared,” May 10, 2016, http://pib.nic.in/newsite/PrintRelease.aspx?relid=145168. In comparison, the acceptance rate for elite Indian institutions of higher learning like the Indian Institutes of Technology and Indian Institutes of Management was 16.5 percent and 17.1 percent, respectively. Roshan Kishore, “What It Takes to Crack the Civil Services Entrance,” Mint, July 24, 2015, http://www.livemint.com/Opinion/Z1xMAkTfdWjbq9UBWVQs0L/What-it-takes-to-crack-the-civil-services-entrance.html.

³⁶ K.P. Krishnan and T.V. Somanathan, “The Civil Service,” in Rethinking Public Institutions in India, eds. by Devesh Kapur, Pratap Bhanu Mehta, and Milan Vaishnav (New Delhi: Oxford University Press, forthcoming).

³⁷ This ratio was 36.4:1 in 1947 but only 11.4:1 in 2008. See Government of India, Seventh Central Pay Commission, 67; also see K.P. Krishnan and T.V. Somanathan, “The Civil Service.” The Seventh Central Pay Commission also documents that a general helper, the lowest-ranked employee in the government, now makes 22,579 rupees, more than double his counterpart in the private sector. For top management positions, however, the pay ratio in the public sector continues to lag considerably: an analysis of 50 major firms listed on the National Stock Exchange of India found that top management were paid 170 times the salary of the average staffer. See N. Sundaresha Subramanian, “Nifty Firm Directors Earn 170 Times Their Staff,” Business Standard, November 26, 2015, http://www.business-standard.com/article/companies/nifty-firm-directors-earn-170-times-their-staff-115112600039_1.html.

³⁸ Krishna, “Continuity and Change,” 434. The decline in monetary compensation over time is a point that is heavily contested. While poorly paid in salary terms, IAS officers are still eligible for perquisites like household help, vehicles, housing, and land—although the latter two function akin to stock options and can take a long time to vest.

³⁹ The survey covered 18,432 officers belonging to the ten select services, including all three All India Services. Government of India, Ministry of Personnel, Public Grievances, and Pensions, Department of Administrative Reforms and Public Grievances, Civil Services Survey: A Report (New Delhi: Government of India, 2010).

⁴⁰ Virendra Nath Bhatt, “A Hellhole for Civil Servants,” Tehelka, September 23, 2013, http://www.tehelka.com/2013/09/a-hellhole-for-civil-servants/.

⁴¹ “Haryana Transfers Ashok Khemka Again,” Economic Times, April 7, 2016, http://economictimes.indiatimes.com/news/politics-and-nation/haryana-transfers-ashok-khemka-again/articleshow/51733822.cms.

⁴² Sukhbir Siwach, “Ashok Khemka, Whistleblower IAS Officer, Transferred Again,” Times of India, April 2, 2015, http://timesofindia.indiatimes.com/india/Ashok-Khemka-whistleblower-IAS-officer-transferred-again/articleshow/46777082.cms.

⁴³ In its landmark judgment on the need to free bureaucrats from political interference, the Supreme Court of India stated that “civil servants are not having stability of tenure, particularly in the State Governments where transfers and postings are made frequently, at the whims and fancies of the executive head for political and other considerations and not in public interest.” T.S.R. Subramanian vs. Union of India, Supreme Court of India, Writ Petition (Civil) No. 82, October 31, 2013.

⁴⁴ Shantanu Nandan Sharma, “Not Happy With Your Performance Appraisal? Join the Government,” Economic Times, April 13, 2013, http://economictimes.indiatimes.com/jobs/not-happy-with-your-performance-appraisal-join-the-government/articleshow/msid-19520037,curpg-2.cms?from=mdr; also see “Bureaucrats Fleeing From Modi’s Delhi: A ‘Control Freak’ PM Is Not the Only Reason,” Firstpost, July 10, 2015, http://www.firstpost.com/politics/bureacrats-fleeing-from-modis-delhi-a-control-freak-pm-is-not-the-only-reason-2336420.html.

⁴⁵ Thus, while many IAS officers lament arbitrary transfers and brief tenures in a given post, many perceive benefits from regular rotations across domains because this offers a diversity of experience. One cynical interpretation is that this also makes accountability much more diffuse. See Prabhu Ghate, “Reforming the Civil Service: Meeting Crucial Need for Expertise,” Economic and Political Weekly 33, no.7 (February 14–20, 1998): 359–365.

⁴⁶ Department of Administrative Reforms and Public Grievances, Civil Services Survey, 89–90.

⁴⁷ R.K. Raghavan, “The Stained Steel Frame,” Hindu, January 28, 2016, http://www.thehindu.com/opinion/lead/corruption-in-civil-services-the-stained-steel-frame/article8159067.ece.

⁴⁸ The response of the minister of state in the Ministry of Personnel, Public Grievances, and Pensions, Jitendra Singh, to an unstarred question in the Rajya Sabha, April 23, 2015.

⁴⁹ See Marianne Bertrand, Robin Burgess, Arunish Chawla, and Guo Xu, “Determinants and Consequences of Bureaucrat Effectiveness: Evidence From the Indian Administrative Service,” International Growth Center Working Paper, September 24, 2015, http://www.theigc.org/wp-content/uploads/2015/08/ias_draft_v12.pdf (accessed October 26, 2015); Rikhil R. Bhavnani and Alexander Lee, “Local Embeddedness and Bureaucratic Performance: Evidence From India,” Working Paper, Department of Political Science, University of Wisconsin-Madison, August 26, 2015, https://faculty.polisci.wisc.edu/bhavnani/wp-content/uploads/2015/09/BhavnaniLeeEmbeddedness.pdf (accessed October 23, 2015); John-Paul Ferguson and Sharique Hasan, “Specialization and Career Dynamics: Evidence From the Indian Administrative Service,” Administrative Science Quarterly 58, no. 2 (June 2013): 233–256; Jonas Hjort, Gautam Rao, and Elizabeth Santorella, “Bureaucrat Value-Added and Local Economic Outcomes,” Working Paper, Department of Economics, Harvard University, November 24, 2015 (on file with authors); Lakshmi Iyer and Anandi Mani, “Traveling Agents: Political Change and Bureaucratic Turnover in India,” Review of Economics and Statistics 94, no. 3 (August 2012): 723–739; Anusha Nath, “Bureaucrats and Politicians: Electoral Competition and Dynamic Incentives,” IED Working Paper 269, Boston University, October 6, 2015, https://www.dropbox.com/s/a5n7jldtqw6gza9/AnushaNath_BureacratsAndPoliticians.pdf?dl=0 (accessed October 26, 2015). Some of the findings the authors attribute to Iyer and Mani are contained in a previous version of their 2012 article. See Lakshmi Iyer and Anandi Mani, “Traveling Agents: Political Change and Bureaucratic Turnover in India,” Working Paper, Harvard Business School, November 2009, http://www.hbs.edu/faculty/Publication%20Files/09-006_161d8937-9cd3-4709-b3ba-53cdc9588cfc.pdf (accessed October 13, 2015). It is important to note that of the six papers under consideration, as of mid-2016 only two—Ferguson and Hasan (2013) and Iyer and Mani (2012)—have been published in peer-reviewed publications. Hence, one should treat the research findings of this new literature with caution. Nonetheless, the fact that key results converge across papers allows one to formulate some initial conclusions.

⁵⁰ As a composite indicator of bureaucratic efficiency that takes the views of fellow officers and stakeholders into account, this index measure arguably provides more precise information about a given IAS officer’s abilities than the current system of performance appraisal. For example, according to a media report, an official review of the executive records of 1,089 serving IAS officers carried out by fourteen state cadres found only two officers unfit for continuation in service. This is despite the fact that according to the government’s admission, formal complaints against IAS officers have risen in recent years, from 246 in 2013–2014 to 333 in 2014–2015 and to 342 in 2015–2016. See Subhomoy Bhattacharjee, “Only 2 of 1,089 IAS Officers Inept: DoPT,” Business Standard, April 22, 2016, http://www.business-standard.com/article/economy-policy/only-2-of-1-089-ias-officers-inept-dopt-116042200066_1.html; also see Press Trust of India, “Rise in Complaints Against IAS, IPS Officers: Government,” Business Standard, May 4, 2016, http://www.business-standard.com/article/pti-stories/rise-in-complaints-against-ias-ips-officers-govt-116050400617_1.html.

⁵¹ Bertrand, Burgess, Chawla, and Guo, “Determinants and Consequences of Bureaucrat Effectiveness.”

⁵² While Ferguson and Hasan can rule out any systematic relationship between specialization in any one particular field and the rate of promotion, they cannot fully disentangle the precise mechanism that connects specialization and career success. They list at least two possibilities: that specialization reduces the uncertainty about an officer’s ability or that it allows officers to forge better working relations with superiors, which results in more positive evaluations.

⁵³ Ferguson and Hasan, “Specialization and Career Dynamics,” 19.

⁵⁴ For instance, the authors found that officers who graduated from college in the first division are 66 percent more likely to be empaneled than others. Officers who possess a larger number of academic degrees also receive a positive bump, although smaller in magnitude.

⁵⁵ One example of the traditional contention on specialization can be found in Naresh C. Saxena, “Improving Programme Delivery,” Seminar 541 (September 2004).

⁵⁶ Iyer and Mani, “Traveling Agents.”

⁵⁷ The authors consider four sets of background attributes: individual characteristics (gender, caste, urban or rural background, and age), education (science, technology, engineering, and math degrees and academic distinction), work experience (prior jobs in the public, private, and research sectors), and scores on the entrance and training exams. The organizational determinants under consideration include age at entry, cohort size, and their interaction.

⁵⁸ It is not uncommon for aspiring candidates to take multiple attempts to clear the preliminary and main entrance exams. While the correlation depicted by Bertrand and her fellow researchers do not consider the potentially confounding effect of the number of exam attempts, it is possible that a candidate’s average age at entry could obscure several years of preparation and failed attempts preceding entry and, in turn, a candidate’s innate unsuitability to work in the IAS.

⁵⁹ Peter B. Evans, Embedded Autonomy: States and Industrial Transformation (Princeton, NJ: Princeton University Press, 1995).

⁶⁰ Bhavnani and Lee use access to public healthcare centers as an additional outcome measure and obtain similar results.

⁶¹ While the cadre allocation policy fixes the total number of insider officers in a state, this study measured the impact of local officers as compared with outsiders.

⁶² It is possible that embeddedness generates positive effects if local officers wish to improve their districts or if they possess knowledge of local customs and connections that improves their performance. However, the positive results in this paper were driven by the presence of accountability mechanisms, not local knowledge as such.

⁶³ The data Hjort and his fellow researchers employed measures nighttime light intensity based on high-resolution images captured by satellites at night. This type of luminosity measure has become a common metric of economic activity. See J. Vernon Henderson, Adam Storeygard, and David N Weil, “Measuring Economic Growth From Outer Space,” American Economic Review 102, no. 2 (April 2012): 994–1028.

⁶⁴ This latter finding on caste loyalty relied on a subset of data from just two northern Indian states, Uttar Pradesh and Uttarakhand. Given that senior civil service positions are typically filled on the basis of seniority, it is possible that the top-ranking officers in a cohort know that they have a higher probability of reaching top positions and, perhaps, have a better incentive to behave well with politicians.

⁶⁵ Here, the researchers looked at the average importance of an officer’s posts across the entirety of their career. See Iyer and Mani, “Traveling Agents.”

⁶⁶ Ibid. Furthermore, the researchers were unable to uncover any statistically significant impact of transfers on either immunization coverage or completion of road projects—two alternative indicators of district-level performance.

⁶⁷ For a theoretical account of why political competition is good for governance, see Pranab Bardhan and Tsung-Tao Yang, “Political Competition in Economic Perspective,” BREAD Working Paper No. 78 (2004). For a leading empirical account of this relationship, see Timothy Besley, Torsten Persson, and Daniel M. Strum, “Political Competition, Policy and Growth: Theory and Evidence from the US,” Review of Economic Studies 77, no.4 (2010): 1329–1352.

⁶⁸ Each MP is given a fixed amount of funding each year to implement public works projects in their constituency. In 2015, the amount was approximately $735,000. While the MP recommends projects, the district administration is responsible for implementation. The scheme is known as the Members of Parliament Local Area Development Scheme.

⁶⁹ The exogeneity of such a shock might be disputed on the following grounds: shortly after new census data are released, officials begin speculating which constituencies will gain or lose reservation after a new delimitation. Therefore, it is plausible that bureaucrats might know in advance, if not with absolute certainty, the changes in store for the politicians in whose constituencies they work.

⁷⁰ Nath found no systematic bias in the types of projects that district officials sanction faster. This addresses the possible concern that district officials are motivated by rent-seeking incentives associated with certain lucrative types of projects. Furthermore, because bureaucratic promotions occur at different times, both before and after elections, Nath could examine the differences in the timing of promotions. Bureaucrats who were up for promotion following elections changed their behavior in party strongholds or in places where the incumbent politician could punish the bureaucrat if he or she does not perform.

⁷¹ Saad Gulzar and Benjamin Pasquale, “The Political Economy of Oversight: Evidence From India’s Employment Guarantee,” forthcoming, American Political Science Review.

⁷² Mihir S. Sharma, “End the IAS,” Business Standard, June 5, 2015, http://www.business-standard.com/article/opinion/mihir-s-sharma-end-the-ias-115060501417_1.html.

⁷³ For an archetypal example of the literature on institutional reform, see Peter Evans, “Development as Institutional Change: The Pitfalls of Monocropping and the Potentials of Deliberation,” Studies in Comparative Institutional Development 38, no. 4 (December 2004): 30–52.

⁷⁴ Krishna, “Continuity and Change,” 442.

⁷⁵ A lengthy list of similar administrative commissions and expert task forces can be found in Bibek Debroy, “Dismantling the Steel Frame,” Seminar 594 (February 2009).

⁷⁶ Government of India, Ministry of Personnel, Public Grievances, and Pensions, Department of Administrative Reforms and Public Grievances, Refurbishing of Personnel Administration— Scaling New Heights, Second Administrative Reforms Commission, Tenth Report (New Delhi: Government of India, 2006), 1.

⁷⁷ All told, the commission’s various reports have recommended as many as 1,200 reform measures to improve bureaucratic efficiency, 600 of which had been implemented as of July 2013, according to the government. Government of India, Ministry of Personnel, Public Grievances, and Pensions, Department of Administrative Reforms and Public Grievances, “Implementation of Recommendations of Administrative Reforms Commission,” Presentation to Officers of the Indian Administrative Service (Phase IV) 1992–1998 Batch, July 4, 2013, http://darpg.gov.in/darpgwebsite_cms/document/file/sample_presentation_arc_2.pdf; Prajapati Trivedi, “Administrative Reforms Must for Nation’s Long-Term Growth,” Business Today, December 27, 2014, http://www.businesstoday.in/magazine/cover-story/prajapati-trivedi-on-need-of-economic-business-reforms/story/213481.html.

⁷⁸ Manish Sabharwal, “A New Kind of Babu,” Indian Express, April 1, 2015, http://indianexpress.com/article/opinion/columns/a-new-kind-of-babu/; Arvind Panagariya, “Bringing Competition to the Top Civil Services” Yojana, August 2005.

⁷⁹ Gulzar Natarajan, “Lateral Entry, Blind Alley,” Indian Express, April 13, 2015, http://indianexpress.com/article/opinion/columns/lateral-entry-blind-alley/.

⁸⁰ Government of India, Ministry of Personnel, Public Grievances, and Pensions, Press Information Bureau, “Lateral Entry in IAS,” December 10, 2015, http://pib.nic.in/newsite/PrintRelease.aspx?relid=132978.

⁸¹ Only 43 percent of all IAS officers agree with the idea of merit-based lateral entry into the higher echelons of the civil service, compared with 56 percent for all other services. See Department of Administrative Reforms and Public Grievances, Civil Services Survey, 36. The Second Administrative Reforms Commission too noted that most IAS associations opposed lateral entry from the private sector, although some were in favor of allowing civil servants to work in private-sector organizations for brief periods (three years or less). See Department of Administrative Reforms and Public Grievances, Ministry of Personnel, Public Grievances, and Pensions, Government of India, Refurbishing of Personnel Administration—Scaling New Heights, 63.

⁸² T.S.R. Subramanian vs. Union of India.

⁸³ The court order applies to all state and central employees of the All India Services. See also Akshat Kaushal, “SC Judgment Is Neither Novel Nor Landmark,” Rediff, November 5, 2013, http://www.rediff.com/news/interview/sc-judgment-is-neither-novel-nor-landmark/20131105.htm.

⁸⁴ Naresh C. Saxena, “Administration and the People: Higher Bureaucracy Needs Radical Reforms,” Planning Commission (2010).

⁸⁵ Uday Mahurkar, “Transfer-Posting Raj Ends,” Indian Today, November 25, 2015, http://indiatoday.intoday.in/story/transfer-posting-raj-ends/1/531599.html. While honesty and efficiency are worthy attributes in bureaucrats, screening officers seeking central postings for moral rectitude does not necessarily address the systemic issue of politicized transfers.

⁸⁶ A.K. Bhattacharya, “Looking for Logic in a Reshuffle,” Business Standard, February 7, 2016, http://www.business-standard.com/article/opinion/a-k-bhattacharya-looking-for-logic-in-a-reshuffle-116020700699_1.html; Gopal Pillai, “Yes Minister, the Fault Is Entirely Yours,” Wire, April 9, 2015, http://thewire.in/9952/yes-minister-the-fault-is-entirely-yours/.

⁸⁷ The quantitative impact of an officer’s entry exam score and demonstrated improvement as captured by an officer’s post-entry training exam score on perceived effectiveness is similar, lending credence to the view that intrinsic motivation plays a significant role in predicting future success. It is the authors’ understanding that in some states, initial ability is taken into account when assignments are determined, if not officially. For instance, one veteran IAS officer reported that the topper of the Tamil Nadu batch is given preferential consideration for the prestigious job of deputy secretary for the budget. In West Bengal, the batch topper is often posted as deputy secretary to the chief secretary of the state government, another strategically important post.

⁸⁸ As a counterargument, one could argue that IAS officers who underperform might be systematically excluded from the professional development or career opportunities that would allow them to improve their performance. However, it is not clear that this is the case because virtually all officers are able to apply for long-term foreign training at least once in their careers (after seven years of service and before they turn forty-five years old).

⁸⁹ Government of India, Refurbishing of Personnel Administration—Scaling New Heights, 211–212.

⁹⁰ Uday Mahurkar, “India’s Top Babus Face New Modi Test,” India Today, June 23, 2016, http://indiatoday.intoday.in/story/narendra-modi-civil-servants-empanelment-process/1/698816.html.

⁹¹ The finding on the perceived effectiveness of IAS officers comes from Bertrand, Burgess, Chawla, and Guo, “Determinants and Consequences of Bureaucrat Effectiveness.”

⁹² The Second Administrative Reforms Commission recommended that the eligibility age be set between twenty-one and twenty-five years for candidates from the general caste category, between twenty-one and twenty-eight years for OBCs, and between twenty-one and twenty-nine years for SC and ST candidates and for the physically challenged. At present, general-category applicants are limited to six attempts and a maximum age limit of thirty-two years, while OBC applicants can take nine attempts until the age of thirty-five, and SC and ST candidates have an unlimited number of attempts up to thirty-seven years of age. For more information, see “No Change in Age Limit, Attempts for This Year: DoPT,” Hindu, November 19, 2014, http://www.thehindu.com/news/national/no-change-in-age-limit-attempts-for-this-year-dopt/article6612165.ece.

⁹³ Department of Administrative Reforms and Public Grievances, Civil Services Survey, 35.

⁹⁴ Aloke Tikku,“UPSC Panel Wants Lower Eligibility Age, Govt Says Let’s Build Consensus,” Hindustan Times, August 12, 2016, http://www.hindustantimes.com/india-news/upsc-panel-wants-lower-eligibility-age-govt-says-let-s-build-consensus/story-QDfHTkuAlOBdentPQNp3iM.html.

⁹⁵ Saxena argues that 25 to 50 percent of officers between the ages of fifty-two and fifty-five should be retired, as is the practice in the Indian Army. Reducing the number of IAS positions would open up space for the elimination of unnecessary posts and the infusion of new talent via lateral entry. See Saxena, “Administration and the People.”

⁹⁶ Sabharwal, “A New Kind of Babu.”

⁹⁷ The move to evaluating civil servants on the basis of outcomes rather than outputs was a prominent feature of the Second Administrative Reforms Commission. See M. Veerappa Moily, “Transforming Our System of Governance,” Seminar 594 (February 2009).

⁹⁸ A recent review of the literature on performance-based pay finds that such mechanisms are more successful when they involve frontline functionaries of the state and when the incentives of government and citizens align. Where performance-based pay can create problems is for state authorities tasked with ensuring compliance, which can lead to corruption and tension between citizens and the government. See Frederico Finan, Benjamin A. Olken, and Rohini Pande, “The Personnel Economics of the State,” National Bureau of Economic Research Working Paper No. 21825 (December 2015).

⁹⁹ Siddhartha Rai, “Government Employees to Get Reviewed at 50, Says DoPT,” Business Today, September 17, 2015, http://www.businesstoday.in/current/policy/dopt-headed-by-the-pm-narendra-modi-to-review-and-retire-non-performing-bureaucrats/story/223833.html.

¹⁰⁰ Bharti Jain, “Modi Govt Dismissed 13 Officers, Penalized 45 for Inefficiency,” Times of India, December 17, 2015, http://timesofindia.indiatimes.com/india/Modi-govt-dismissed-13-officers-penalized-45-for-inefficiency/articleshow/50212073.cms.

¹⁰¹ The insider-outsider ratio is fixed at 1:2 but can be lower if insider vacancies remain unfilled. See Government of India, Ministry of Personnel, Public Grievances, and Pensions, Department of Personnel and Training, “Cadre Allocation Policy for the All India Services-IAS/IPS/IFS – Reg,” Office Memorandum No. 13011/22/2005-AIS (I), April 10, 2008.

¹⁰² Constituent Assembly Debates, Volume 10, October 10, 1949, Part I, www.indiankanoon.org/doc/735670/.

¹⁰³ Ashutosh Varshney, “How Has Indian Federalism Done?” Studies in Indian Politics 1, no. 1 (2013): 47.

¹⁰⁴ See Sumit Khanna “Retirement Temporary, Benefits Are Constant for Babus,” DNA India, August 20, 2012, http://www.dnaindia.com/india/report-retirement-temporary-benefits-are-constant-for-babus-1730393; Jayant Sriram, “Revenge of the Babus: Liberalisation Has Expanded the Power of the Bureaucracy, Creating a Permanent Establishment That Never Retires,” India Today, September 27, 2013, http://indiatoday.intoday.in/story/revenge-of-the-babus-liberalisation-expanded-power-of-bureaucracy/1/312039.html; and “New Custodians of Our Right to Information: 9 Ex-Babus, Law Prof in CIC,” Hindustan Times, February 20, 2016, http://www.hindustantimes.com/india/new-custodians-of-our-right-to-information-9-ex-babus-law-prof-in-cic/story-670LjRG9cCxpl8Di2Wd3vN.html.

¹⁰⁵ See Navroz K. Dubash, “New Regulatory Institutions in Infrastructure: From De-Politicization to Creative Politics,” in Rethinking Public Institutions in India, eds. by Devesh Kapur, Pratap Bhanu Mehta, and Milan Vaishnav (New Delhi: Oxford University Press, forthcoming).

¹⁰⁶ Devesh Kapur, “The Other Steel Frame,” Business Standard, August 18, 2013, http://www.business-standard.com/article/opinion/the-other-steel-frame-113081800640_1.html.

¹⁰⁷ Atul Kohli, “State and Redistributive Development in India,” in Growth, Inequality and Social Development in India: Is Inclusive Growth Possible?, ed. by R. Nagaraj (London: Palgrave Macmillan UK, 2012).

A NEW PUBLIC POLICY FOR A NEW INDIA by Shiv Visvanathan

What makes public policy exciting and potentially inventive is the contested nature of the public sphere. It is anchored in a diversity of perspectives which challenges the dominance of one subject.

India is a country full of paradoxes. The elite in the country are forward-looking; they emphasise the need for reskilling but they conduct all this with backward-looking institutions. An acute observer once said: “we want to be [a] knowledge economy without reflecting on the demands of [a] knowledge society. As a result, we lack the institutions to be systematically innovative and our policies seem short run and erratic. We are being outfought and out-thought in the realm of knowledge and policy, while confusing rhetorical victories for real time gains.”

In fact, our new regime talks of the demise of the Planning Commission as a feather in its cap. It conflates the existence of the Planning Commission with the ideology of the previous regime and treats it merely as a policy issue. Today, our medical and our environmental policies, for example, are in a shambles, and yet there are no relevant spaces to debate them. We are a tangled, regulatory society without being “socialist” in terms of justice, which we set out to be.

          Confusion over distinctions

Let’s face it. Our knowledge society does not differentiate between information and knowledge. Knowledge is embodied, epistemic, and has tacit elements. By confusing information and knowledge, we lack critical thinking, the metalanguages and the heuristics that go into the definition of knowledge. Central to such confusion is Sam Pitroda’s Knowledge Commission report of 2006 which equated the knowledge revolution to the information revolution and confused technology with epistemology.

In fact, the elite in India do not realise that of the four revolutions of the 20th century, in Quantum physics, Genetics, in Linguistics and in Knowledge, the last two bypassed us. The profound Linguistic revolution had no impact in India despite the fact that an exceptional linguist like Ferdinand de Saussure was a professor of Sanskrit at Geneva in the same period. While the footprints of the Quantum revolution appeared in India well after World War II, the knowledge revolution led by Gregory Bateson, Thomas Kuhn and Claude Levi Strauss never excited us.

Contemporary India, in that sense, was never sensitive to the genealogies of knowledge. We boasted of the Planning Commission and the Knowledge Commission, of the D.S. Kothari Commission but saw education and knowledge in instrumental terms. To add to our problems, we misread the managerial revolution and the debates on governance and democracy. We revamped a few commerce departments and believed that we had reinvented management. But our Indian Institutes of Management (IIM) had little research sensitivity. We consumed knowledge but we rarely added creatively to the stockpile. India became a consumer of knowledge rather than a translator or an inventor of knowledge systems.

          Knowledge and power

This background is necessary to understand the new relations between knowledge and power. Linking the two is the field called policy. It also creates two kinds of intellectuals, the policy intellectual and the public intellectual.

The distinction is critical. The policy intellectual serves as an extension of the state. He/she is more a product of think tanks, of groups which strictly cater to policy interests of the state or of corporations. A public intellectual is a figure who provides a wide-ranging critique of policy, and looks more creatively at the relation between knowledge and power. A knowledge society needs both sets of intellectuals. The late Sukhamoy Chakravarty, the economist, was a great policy intellectual. Ashis Nandy, Rajni Kothari and U.R. Ananthamurthy belong to the category of public intellectuals. The policy intellectual usually takes his expertise for granted. The public intellectual questions the nature of expertise, probing deeper into the ethics and genealogy of ideas. In the post-liberalisation period, India has had more policy than public intellectuals with think tanks like the Centre for Policy Research and the Observer Research Foundation dominating the scene.

The think tanks and their attempts to formulate policy raise the whole question of the relation between knowledge and the public sphere. Policy formulation has not really articulated the views of the public sphere. In fact, the first challenges to policy came from the social movements, and from civil society which identified policy and experts as mere extensions to the state. The movements that grew around the Bhopal gas tragedy, the Narmada dam; the narratives of displacement and dispossession raised deep questions about policy and expertise, and about the public consumption of policy. Governance is now seen no longer as a statist exercise and the question of governmentality involves civil society articulating new epistemologies, notions of citizenship, ideas about the democratisation of knowledge and the assessment of public policy impacts. Governance has become tied to democracy, with the public sphere becoming crucial and public policy a critical field.

          Field of the future


Public policy is not its impoverished, mechanistic cousin, Public Administration. Jawaharlal Nehru started the Indian Institute of Public Administration on the basis of the Paul Appleby report. Public policy became that empty space between management and public administration. It had a different texture and different requirements. Management schools in India have never succeeded in establishing a successful school of public policy as all efforts have become annexes of departments of economics.

Public administration is more a monument to the bureaucratic ego in India than to administrative reflexivity. As experiments, public policy has never succeeded, and yet today is a fast growing field with new departments at various institutions and universities. So far, it is a case of necessity not generating adequate inventiveness in our institutions. Yet, public policy is one of the fields of the future, linking as it does, new notions of empowerment in democracy with new ideas of knowledge in policy.

What makes public policy exciting, protean and potentially inventive is the contested nature of the public sphere. It is anchored in a diversity of perspectives which challenges the dominance of one subject. For example, economics, which was almost a canonical discipline, now realises that it confronts a new commons of social sciences which sees its sense of measure as inadequate to understand freedom or suffering. The new developments in feminism, cultural studies, future studies and science studies have added an increasing plurality to the fields of knowledge. Today, the relation between the ‘expert’ and the ‘citizen’ has changed and new forms of knowledge have to be considered. One sees this particularly in the development of ecological policy.



Nature which was once taken for granted or seen as passive in the realm of knowledge is now becoming a part of the social contract. The problems of climate change, and the energy crisis have revealed that science and economics are inadequate to answer questions related to ecology. Revolutions in ecology show that panarchy, complexity and risk had created a non-Promethean science where policy is merely prudent and precautionary. The subject of ethics has made a big return into the making of these disciplines. A subject-wise understanding in terms of the old hierarchies of knowledge is inadequate for policy. We are looking for new modes of knowledge which are intercultural, interdisciplinary and holistic. The emphasis is now on emergence rather than certainty.

          New demands of democracy

These revolutions in knowledge have been catalysed by the new demands of democracy. Democracy is no more a passive exercise of citizenship reduced only to the exercise of periodic elections. Today, democracy is more proactive. The citizen knows more and demands more. She is ready to challenge the dominance of the expert. She senses that her active role is required to sustain a society. The public sphere today is more dynamic and contested.

One senses the excitement and the choices before India in the issues confronting us. In the 1950s, India treated nuclear energy as sacrosanct. Today, the fishermen of Kudankulam in Tamil Nadu, and the tribals and villagers in Maharashtra, Rajasthan and Gujarat are challenging the location of nuclear plants and even the feasibility of nuclear energy.

One sees similar debate on the future of biotechnology, for example on the need for Genetically Modified (GM) crops. For the first time, one saw an Environment Minister invite all stakeholders to a debate when in 2010, Mr. Jairam Ramesh of the Congress called for public consultations on the release of Bt brinjal. It was wonderful to watch the public sphere debating public policy on biotechnology.

The recent debates around growth, development and the fate of forests and the future of mining have also raised issues that public policy must answer. The new generation has to ask itself whether nature has rights: for example does a mountain have legal standing? When a tribal says that when a mountain dies, a myth dies, how does one translate his language into the dialects of policy? Recently, there was a report on the death of a waterfall. How does one analyse the death of a ‘myth’ through costs and benefits? Is a waterfall only about cusecs of water?

Similarly, the city raises its own seedbed of questions around the informal economy, the future of waste, issues of violence — all of which confront the policymaker. Ethics, science, suffering and philosophy cannot be ignored in any debate today. A student has to reach into the best of the academe to answer the new challenges to citizenship. One has to dream of futures in realistic terms going beyond the simplicity of smart cities to ask what urban space and urban imagination are.

Today, at a time when the university is in crisis, and the relevance of academics is in question, subjects such as public policy can revitalise the university, intensify the debates around intellectual life and show that the life of the public mind has new challenges. A subject like public policy is an invitation to construct a feasible future. It will be interesting to see how many Indians accept its challenge and construct the dream of a different India.

(Shiv Visvanathan is a professor at Jindal School of Government and Public Policy.)

Article courtesy: http://www.thehindu.com/opinion/lead/a-new-public-policy-for-a-new-india/article7070831.ece

New Publication mentioning reference of this Blog - PUBLIC ADMINISTRATION:APPROACHES & APPLICATIONS

Unmatchable rejoice! This blog now finds place in the PEARSON published textbook put together by IGNOU and other renowned Indian Universities' faculties (Page 123). 
It is a wonderful feeling that cannot be expressed completely in words. There is more to come for sure. Keep the love, wishes and blessings coming.


PUBLIC ADMINISTRATION:APPROACHES & APPLICATIONS:
This volume is written to fulfill the demand for a comprehensive textbook on public administration. It covers almost all the topics in public administration taught at the undergraduate level of Indian universities. However, because of its multi-disciplinary focus and orientation, this book will also be useful for the postgraduates as well as civil services aspirants. For the benefit of the readers, a chapter on post-new public management and another one on historical antecedents of civil society have been added. The objective of this volume is to provide an insight into the key concepts and issues of public administration, especially in the changing neo-liberal context.

Link: http://www.pearsoned.co.in/web/books/9789332555075_Public-Administration_Alka-Dhameja.aspx

SITUATING SOCIAL SCIENCES IN THE PHILOSOPHICAL DEBATE ON RESEARCH METHODOLOGY WITH A FOCUS ON PUBLIC ADMINISTRATION - LAVANYA SURESH

This article contextualises and locates social sciences in the

wider debate of research methodology. It goes on to narrow

down its focus to the discipline of Public Administration by

tracing its evolution and reaches the conclusion that a Kuhnian

historiography of a scientific discipline has characterised the

growth of theory in this discipline since its inception.

INTRODUCTION:

THE METHODOLOGY behind social science research work invariably

depends on the philosophical orientation one subscribes to. There are two

philosophical schools of thought dealt with here, the scientific school that is

geared towards “generalisability beyond spatio-temporal context” (Mukherji

2000:14) and the Hermeneutics approach which involves empathic

interpretation of reality, both have their relative strengths and weaknesses.

Scientific Method

The scientific school of thought gained from the contributions of Karl

Popper, Thomas Kuhn and Imre Lakatos. Following are the summarised

versions of the philosophies and what each thought of others work.

Popper’s Falsification

The central aspect of Popper’s theory is that he sees science as a set

of distinct unconnected theories (DiCicco and Levy 1999) that may be

overturned at anytime. This he terms as “fallibilism”. (Walker 2010:438)

The strength of a theory lies in its resilience to withstand falsification rather

that in verifiability (Mukherji 2000). His concern was not with paradigm

shift as in Kuhn but on identifying anomalies so as to falsify existing theory.

Hence refutation is the mark of progress, wherein, dominant and competing

theories are pitted against each other leading to the development of science.

The two vital principals of enquiry are thus “avoiding narrow specialisation”

and maintaining a “highly critical approach.” (Wallcer 2010:439).

To summarise, Popper criticises Kuhn calling dominant paradigms

and incommensurability as myths (Walker 2010) and established an open

society characterised by multiplicity of methodologies and theories, which

ultimately are conjectures that need to be critically examined and then tested.

The results may allow for falsification which will either set aside theories

that are inaccurate or elevate those that can be empirically tested.

The limitations of this theory are pointed out by Kuhn, who criticises

falsification stating that, if any and every failure were to fit the grounds for

the rejection of a theory then all theories would be rejected. (Kuhn 1970)

Kuhn’s Revolutionary Theories of Scientific Development

Kuhn refutes the assumption that development of science is a cumulative

process and states instead that science develops by successive revolutions

from one paradigm to another. (Kuhn 1970). The interim periods are

characterised by what Kuhn calls as ‘Normal Science’ during which time

all research and all scientists are guided by a dominant paradigm, that leads

to narrow and directed research (Kuhn 1970; Walker 2010).

Paradigms may be “understood in terms of its life-cycle” (Walker

2010:435) marked by phases. The pre-paradigm phase is characterised by

debate on legitimacy of methods, problems and standards of solutions, this

actually helps define the paradigm. Once the paradigm is assimilated, the

phase of normal science appears and all decent disappears. Problems that

do not conform to the paradigm, that is anomalies, are usually ignored. The

concerns of legitimacy, etc., once more come to the forefront just before

the scientific revolution (Kuhn 1970), when the dominant paradigm is

first challenged. This phase is called as the period of crisis at which time

anomalies mount and an awareness of the same is brought about. There

shall be extensive studies into the anomalies that lead to discoveries (Wade

1977) and an alternative paradigm is proposed that ultimately brings on

the revolution.

The scientific revolution is therefore a “destructive-constructive

paradigm change” (Kuhn 1970:66) that leads to the development of

science. The alternative paradigm, however faces a consequent struggle

for acceptance among the defenders of the old paradigm. This stand off is

ultimately solved by “non-rational factors” (Wade 1977:144) like persuasion.

The idea of “incommensurability” (Wade 1977:144) between competing

paradigms that Kuhn emphasises at this point is central to his theory, as

a new paradigm does not build on an older paradigm, it only supplants it.

Lakatos ‘s Methodology of Scientific Research Programmes (MSRP)

Lakatos’s MSRP closely resembles Kuhn’s theory of paradigmms even

though he criticised him. To quote Lakatos, “Where Kuhn sees paradigm;

I also see rational research programmes”. (Walker 2010:436). Lakatos’s

theoretical framework describes the development of science in terms of

progressive development or degeneration (Walker 2010; DiCicco and Levy

1999). According to Lakatos, a science comprises a number of distinct and

competing series of research programmes (DiCicco and Levy 1999). Within

these research programmes he identifies certain core entities; they are as

follows (DiCicco and Levy 1999):

(i) Hard core assumptions:They are assumptions that are

‘irrefutable’ and not subject to empirical testing. Researchers

utilise the same to formulate auxiliary hypothesis;

(ii) Auxiliary Hypothesis: This is a protective belt around the hard

core assumptions, drawn from it and are subject to empirical

testing;

(iii) Positive Heuristic: Research in the programme is guided

by positive heuristic, which is “a partially articulated set of

suggestions or hints” (DiCicco and Levy l999: 686).

(iv) Negative Heuristic: Are those that “delineate the types of

variables and/or models that ought to be shunned by researchers

within a research programme because they deviate from the

assumptions of the hard core” (DiCicco and Levyl999:686).

Lakatos, unlike Kuhn, focuses on the evaluation of the progressive

nature of science. (Walker 2010). His criterion for scientific development is

seen in the light of ‘problemshifts’. Those that are consistent with hard core

assumptions are termed as intraprogram’ problemshifts and those that violate

the hard core assumptions are termed as ‘interprogram’ problemshifts and

generally initiate new research (DiCicco and Levy 1999). Hence scientific

progression has three criteria:

(i) The alternate theory must include all “unrefuted” facts of the

previous theory– “Theory of Subsumption” (Walker 2010:438)

(ii) It must predict a novel fact;

(iii) The theory should have additional corroborative evidence over

the previous theory.

Degenerating or ad hoc research programmes fail to fulfil the above

criteria. Hence like Kuhn, Lakatos also looks at the efficient growth of

scientific knowledge. (Walker 2010). However, a limitation demonstrated

by DiCicco and Levy is that MSRP fails to elaborate on the progressive

or degenerating nature of individual projects in a research programme.

(DiCicco and Levy 1999).

Relevance of Kuhn, Lakatos and Popper to Social Sciences

In discussing research methodologies social sciences often look to

Kuhn, Lakatos and Popper for meta-theoretical guides. Kuhn and Lakatos

themselves have, however, been very critical of this application of their

work. Nevertheless, Kuhn’s “paradigm mentality based on normal science

and incommensurability has been widely employed, if not internalised, by

political scientists.” (Walker 2010:436).

Lakatos is referred to by DiCicco and Levy as the, “...the metatheorist

of choice” (DiCicco and Levy 1999:676; Walker 2010). Scholars in many

fields, from international relations to economics have used MSRP. Unlike

Kuhn and Lakatos, Popper applied his ideas directly to social sciences.

(Walker 2010) and his theory of falsification has been seen as being equally

pertinent to social and as it is to natural sciences.

Hermeneutics

The hermeneutics approach has widely been applied to social science

studies. Wilhelm Dilthey and Hans-Georg Gadamar are two profound

thinkers within this school of thoughts.

Dilthey’s Hermeneutics as the Foundation of Geisteswissenschaften

Dilthey was of the opinion that hermeneutics is the foundation of

any discipline that interprets expressions of man’s life or in other words

Geisteswissenschaften (humanities). He objected to the adopting of natural

sciences methods to the study of man and instead wished to establish the

epistemology of Hermeneutics or a method to study man and understand

him (Palmer 1969).

Understanding is re-experiencing the thoughts of the author. The bases

of his theory of understanding are the concepts of utility of life, expression,

and historicality.

• Life is the complex fusion of feeling and will that is experienced

and needs to be understood in terms of the context of the past or

history”. Life must be understood in the experience of life itself

(Palmer 1969:102).

• An expression is the expression of the inner life of man by

way of art, language, etc., “in which the spirit of man has been

objectified” (Palmer 1969:112).

• Historicality or history can tell man what his nature is today,

though this nature is not fixed.

This gives rise to the hermeneutic circle in which understanding is

grasped from the reciprocal relation or dialogue between the whole (context)

and its parts (text), with regard to the lived experience of the interpreter or

his historicality. Or in other words, the interpretation depends on the situation

in which the interpreter himself stands and hence changes with time.

Gadamar’s Philosophical Hermeneutics

Hermeneutics to Gadamar is a philosophical process wherein

understanding is ontological or a process in man in a culture and history

and is marked by both universality and historicity. This understanding is

not reached methodologically as in Dilthey, but through a dialectic process

between tradition and one’s own self-understanding or prejudgements

(Palmer 1969).

Therefore, understanding functions through a relation of past, present

and future. Interpretations are based on not only what one experiences at

present, but on the tradition of interpretation that existed in the past and the

possibility it opens for the future. Language is the medium through which

history speaks. Hermeneutics then is human understanding that is historical,

linguistic and dialectical. Understanding is not “an act of man but an event

in man” (Palmer 1969:216).

A Case for a Kuhunian Approach to Public Administration

The publication of Kuhn’s The Structure of Scientific Revolutions

in 1962 provides a “definite hallmark for identifying paradigm shifts or

revolutions” (Paine 2002) in fields of study. It enables us to gain a whole

new outlook to the development of a subject and tempts one to adopt the

same to social sciences as well. However, this application has its challenges;

Kuhn himself points to the gap between natural science and social science

(Kuhn, 1970). He “characterises the social sciences by their fundamental

‘disagreement’ over the ‘nature of legitimate scientific problems and

methods” (Walker 2010:433). Nevertheless, the auther believes that such

an application is not only possible but also advantageous as it allows for

the identification of the definite revolutionary ideas that have given new

life to a discipline over the ages. To illustrate, the author has chosen one

branch of social science–Public Administration.

Public administration has developed through a“constellation of facts,

theories and methods” (Kuhn 1970:1) that have been brought about by a

“piecemeal process” (Kuhn 1970:1). Be it the contributions of Woodrow

Wilson, Herbert Simon or Dwight Waldo, each has led the discipline in

different directions.

Kuhn’s idea is explained by paradigms that are dominated by a theory.

These paradigms lead to the establishment of normal science; which is a

period wherein the dominant theory acts as the basis of research for the rest

of the scientific community (Kuhn 1970). In public administration, Woodrow

Wilson’s concept of dichotomy set the tone for the early studies in the field

(Henry 2004) and characterised the Paradigm of Political/Administrative

Dichotomy (Henry 2004)1887-1926 (Avasthi and Maheshwari 2005), Henry

Fayol’s Industrial and General Management underpinned the development

that occurred during The Principles of Administration Paradigm 1927-1937

(Henry 2004), Elton Mayo’s Hawthorne Experiment influenced the Human

Relations (Bhattacharya 2004) Paradigm 1920’s-1930’s, Herbert Simon’s

Administrative Behaviour defined the Behavioural (Bhattacharya 2004)

Paradigm 1938-1947 (Avasthi and Maheshwari 2005), and so on.

When a paradigm is established it attracts most of the next generation

researchers, who set out to further articulate it. These works based on the

paradigm do not overtly disagree with the fundamentals established by

the dominant theory (Kuhn 1970), as can been seen in the first paradigm

of Public Administration. All works, from Frank Goodnow’s Politics and

Administration 1900 right up to Leonard D. White’s Introductions to the

Study of Public Administration 1926, held true to Woodrow Wilson’s

concept of dichotomy (Henry 2004). This phase of ‘normal science’ Kuhn

states also allows for detailed and in-depth study, often developing models

and principles that facilitate the paradigm (Kuhn 1970) as was done by

Luther H. Gulick and Lyndall Urwick in the Papers on the Science of

Administration, Mooney and Reiley in Principles of Organisation and W.F.

Willoughby in Principles of Public Administration (Avasthi and Maheshwari

2005) in the Paradigm of Principles of Administration. They determined

its significant facts, matched the facts to theory and applied the same to

the problems of the time, which are the characteristics that mark Kuhn’s

literature of normal science (Kuhn 1970).

Ultimately though, the aim of normal science is puzzle solving (Kuhn

1970) or in other words, researches set out to solve the problems that

confront the paradigm. As was done by the thinkers within the Paradigm of

Principles of Administration, who set out to use the “network of concepts,

methodologies, (and) theories” (Kuhn 1970:42) of the paradigm to solve

the problem of effectiveness and efficiency that plague the times (Avasthi

and Maheshwari 2005).

But normal science does not stay the same. It has an in-built mechanism

to bring about change, for as it expands the number of novelties or anomalies

tend to increase (Kuhn 1970). Initially anomalies are ignored; as Mary

Parker Follett’s Creative Experience 1924 was during the dominance of the

Paradigm of Principles. (It was only with the coming of the next paradigm,

the Human Relations approach, that her work was recognised.) But, when

the current paradigm persistently fails to explain all puzzles the anomalies

can no longer be ignored.

These anomalies are results that “violate the paradigm-induced

expectations” (Kuhn 1970:52-53) and it brings in a period of crisis, where

in extensive studies into the anomalies occurs. This ultimately leads to the

scientific revolution. Elton Mayo’s Hawthorne experiments are a notable

example of the same. The experiments began as an attempt to prove the

efficacy of the principles that marked the Paradigm of Principles but the

results demonstrated the influence of the social and psychological factors instead, shaking the foundation of the principles school of thought

(Bhattacharya 2004). Such cases are termed as discoveries by Kuhn and

lead to revolutions. The Hawthorn studies led to the establishment of the

human relations approach which ultimately became the next paradigm.

Hence, initially only what is expected is observed, gradually an awareness

of an anomaly occurs; this awareness opens up a period of adjustment till

that time that the anomaly becomes the basis of the next paradigm and is

anticipated within it. It is then that the discovery is complete (Kuhn 1970).

After the discovery of a new paradigm and its assimilation, the

previous paradigm is discarded. But “The decision to reject one paradigm is

always simultaneously the decision to accept another” (Kuhn 1970:77) i.e.,

a previous paradigm is declared as invalid only if an alternative paradigm

is available to take its place. Hence there is no competition of theories

but a replacement of the old by an “incompatible” (Kuhn 1970:95) new

paradigm. This was illustrated in the development of public administration,

when the discipline was redefined by Herbert Simon (Bhattacharya 2004)

during the behavioural paradigm. His principle thesis was that there are no

such things as principles of administration (Avasthi and Maheshwari 2005)

and he called the principles as “no more than proverbs” (Bhattacharya

2004: 13). He provided an alternative positivistic approach in dealing with

administrative challenges, the substantial focus shifting towards ‘decision

making’ (Bhattacharya 2004). These changes are the revolutions that

characterise the development of the field. In Kuhn’s view one sees this

development as a cumulative process only when a person “writes history

backwards” (Kuhn 1970:138). Hence, advancement in a field of study is a

succession of paradigm bound periods, punctuated by revolutionary breaks

(Kuhn 1970).

CONCLUSION

Therefore, as illustrated above, Kuhn’s scientific revolutions can

quite successfully be applied to social science. Social scientists in the past

have looked to Kuhn for methodological guides to develop the discipline

(Walker 2010) and will continue to do so in the future. As David Truman

in his presidential address to the American Political Science Association

(APSA) stated, the application of Kuhn’s concepts of paradigm is a very

helpful means to regenerate the discipline (Walker 20,0:433). Although

predominantly philosophical in its emphasis paradigm does not totally

negate sociological and psychological aspects in science, especially so for

it concerns the numbers of the community of scientists. Hence it does not

leave a small window open for humaneneutics.

ABOUT THE AUTHOR:

Lavanya Suresh – PhD Scholar, CPIGD, Institute for Social and

Economic Change (ISEC), Bangalore.

CONTENT COURTESY - IJPA JAN- MARCH 2015