This paper should be cited as: Watson, D. 2005. Capacity building for decentralised education service delivery in Ethiopiaand Pakistan: a comparative analysis. (ECDPM Discussion Paper 57I). Maastricht: ECDPM. http://www.ecdpm.org/dp57I

David Watson
Accredited Governance Consultant
Department for International Development (DfID)

In association with
Lissane Yohannes (Ethiopia) and Adnan Qader Khan (Pakistan)

The UK Department for International Development (DFID) financed the field work for this paper

A case study prepared for the project ‘Capacity, Change and Performance’

Study of Capacity, Change and Performance: Notes on the methodology

The lack of capacity in low-income countries is one of the main constraints to achieving the Millennium Development Goals. Even practitioners confess to having only a limited understanding of how capacity actually develops. In 2002, the chair of Govnet, the Network on Governance and Capacity Development of the OECD, asked the European Centre for Development Policy Management (ECDPM) in Maastricht, the Netherlands to undertake a study of how organisations and systems, mainly in developing countries, have succeeded in building their capacity and improving performance. The resulting study focuses on the endogenous process of capacity development – the process of change from the perspective of those undergoing the change. The study examines the factors that encourage it, how it differs from one context to another, and why efforts to develop capacity have been more successful in some contexts than in others.

The study consists of about 20 field cases carried out according to a methodological framework with seven components, as follows:

• Capacity: What are the components of capacity, i.e. the capabilities, in a given context?
• Endogenous change and adaptation: How do processes of change take place within an organisation or system?
• Performance: What has the organisation or system accomplished or is it now able to deliver? The focus here is on assessing the effectiveness of the process of capacity development rather than on impact, which will be apparent only in the long term.
• External context: How has the external context – the historical, cultural, political and institutional environment, and the constraints and opportunities they create – influenced the capacity and performance of the organisation or system?
• Stakeholders: What has been the influence of stakeholders such as beneficiaries, suppliers and supporters, and their different interests, expectations, modes of behaviour, resources, interrelationships and intensity of involvement?
• External interventions: How have outsiders influenced the process of change?
• Internal features and key resources: What are the patterns of internal features such as formal and informal roles, structures, resources, culture, strategies and values, and what influence have they had at both the organisational and multi-organisational levels?

The outputs of the study will include about 20 case study reports, an annotated review of the literature, a set of assessment tools, and various thematic papers to stimulate new thinking and practices about capacity development. The synthesis report summarising the results of the case studies will be published in 2005.

The results of the study, interim reports and an elaborated methodology can be consulted at www.capacity.org or www.ecdpm.org. For further information, please contact Ms Heather Baser (hb@ecdpm.org).



Contents

Acronyms
Introduction
Comparison of the case studies in Ethiopia and Pakistan

Acronyms

ADB Asian Development Bank
DAC Development Assistance Committee (OECD)
DfID Department for International Development (UK)
ECDPM European Centre for Development Policy Management
NGO non-governmental organisation
OECD Organisation for Economic Cooperation and Development
PMU project management unit
PRSP Poverty Reduction Strategy Paper
SWAp sector-wide approach
TA technical assistance
USAID United States Agency for International Development

Ethiopia
DfIDE Department for International Development Ethiopia
EPRDF Ethiopian Peoples' Revolutionary Democratic Front
ESDP Education Sector Development Programme
MCB Ministry of Capacity Building
MoE Ministry of Education
PSCAP Public Sector Capacity Building Programme
PTA parent-teacher association
RCBB Regional Capacity Building Bureau
VSAT very small aperture terminal (satellite receiver)

Pakistan
CCB citizen community board
CIET Community Information Empowerment and Training
DCO District Coordination Officer
DSP Decentralisation Support Programme
DfIDP Department for International Development Pakistan
DTCE Devolution Trust for Community Empowerment
MCB Ministry of Capacity Building
MoE Ministry of Education
MPAs Members of the Provincial Assembly
NCHD National Commission for Human Development
NRB National Reconstruction Bureau
NRSP National Rural Support Programme
PESRP Punjab Education Sector Reform Programme
PMIU Project Monitoring and Implementation Unit (of PESRP)
WB World Bank

1 Introduction

This report compares the findings of the case studies of capacity building for decentralised education service delivery in Ethiopia and Pakistan (published as ECDPM Discussion Papers 57H and 57G, respectively), in accordance with the Terms of Reference. The report

• summarises the main features of the two cases in terms of the national contexts, and features of the capacity building experiences in the two countries;
• compares the main features of the contexts and capacity building experiences with a view to drawing conclusions about the apparent significance of various aspects of these experiences (Figure 1 compares the government structures in the two countries); and
• distils conclusions about what factors appear to matter most in the relationship between capacity, change and performance, and determine the feasibility of building effective capacity for devolved education service delivery over time.

The paper is not a substitute for reading the full texts of the two cases. It is written in note form, without much of the evidence used to back up statements and conclusions in the case study papers themselves.

The largely tabular structure of the report compares the two countries against a variety of features. The right-hand column draws conclusions, and assesses the significance of the feature concerned in relation to capacity, change and performance.

The following paragraphs attempt to pull together the main strands of the 'story' in order to highlight the main similarities, contrasts and - to the extent permitted by the data - factors that may explain the capacity building experiences in the two countries.

National contexts: some performance parameters; investment in education

Both Ethiopia and Pakistan are poor, categorised as low-income countries according to the World Bank, but Pakistan has four times the per capita GNP of Ethiopia (and twice the purchasing power parity). But Ethiopia gives significantly higher priority to education, especially if measured by the proportion of GDP so allocated. Ethiopia's absolute and relative poverty accounts for it having less than half the primary school completion rate of Pakistan. Ethiopia has made better progress in achieving equity of access for girls, and this is reflected in proportionately higher level of literacy among women than in Pakistan.

The private sector is a major player in the overall education system capacity in Pakistan, having boomed in response to demand (even from poorer households) for better quality education than the state sector provides. The private sector is not currently significant in Ethiopia, but it is growing player in secondary education, especially in urban areas. The key performance factors cited as reasons for the popularity of private schools in Pakistan is much lower teacher absenteeism (even though the teachers are less well qualified, and worse paid than those employed in state schools); better infrastructure standards, and a higher proportion of co-educational schools and women teachers (explained by the fact that the lower pay attracts fewer men).

Political background and patterns and extent of 'devolution'

The analysis of the respective histories and the emergence of political players and power blocs is revealing. These factors are significant determinants of capacity development potential, and the 'environment' in which capacity - of local government in general, and as a delivery vehicle for education in particular - can be built. Coincidentally, both countries experienced a 'shot-in-the-arm' regarding devolution at the same time (2001), but for different reasons, and with differing degrees of ownership at the crucial regional/provincial level. Up to then, neither country had empowered local governments.

After 2001, Ethiopian local governments were given much more autonomy over (albeit inadequate) financial resources and personnel. In comparison, Pakistan's devolution was, in the words of a major study of the policy, 'incomplete'.

Accountability: the key to local service 'capacity'?

Both countries have put great store in enhancing community accountability as a means of pressuring local governments to deliver better services. Contexts and strategies differ fundamentally.

In Pakistan the President and the National Reconstruction Bureau (NRB) aspire to use the Devolution Trust for Community Empowerment to help form and mobilise citizen community boards (CCBs), which, empowered by law and access to financial resources (25% of local development budgets: by decree), will put pressure on local governments for better services, and indeed launch service capacity-enhancing initiatives. The National Commission for Human Development (NCHD) will gradually establish human development 'incubators' and establish partnerships among actors in the public, private and civil society sectors.

This strategy is to be implemented top-down in spite of the bureaucracy, rather than through or with it. The NRB has been suspicious of the vested interests of bureaucrats, particularly at the provincial level, and has been openly critical of what it sees as attempts to subvert devolution in general and these accountability-enhancing measures in particular. There are deep problems with representative democratic institutions in Pakistan. Service monitoring committees of district councils are still largely dysfunctional, and are caught up in issues of political patronage that permeate all levels of Pakistan polity. Figure 1 depicts the 'shallower' quantitative permeation of representative demo-cracy in the two countries: while there are approximately 1000 constituents for every elected union councillor in Pakistan, there are only 200 in Ethiopia.

In Ethiopia, the governing political party is (now) united since a 'renewal' in 2001. It is dominant nationwide. It (now) reflects a policy consensus, and its influence permeates all levels of government. It has spawned several parallel strategy initiatives: full fiscal devolution (in the better-off and larger regional states); a capacity building plan and corresponding administrative structure to implement it, which sees education as fundamental for nation-building (and which is to meet needs in private and civil society sectors as well as the state sector) and an education policy and sector-wide approach (SWAp), backed by a consortium of donors. The Education Sector Development Programme (ESDP) provides for education 'watchdog' institutions at local (woreda) and even kebele levels (average population 8000), established to monitor progress with state management and investment in education. 'Ownership' from the community level upwards is reflected in sometimes extraordinary levels of community contributions (in cash and in kind) for education infrastructure and materials. There is no better foundation for tight local accountability: communities are poor and, these contributions are a heavy burden. This parental investment gives rise to vocal and potentially critical local stakeholders.

Capacity building: definitions, history and experience

Histories of capacity building differ. In Pakistan, a colonially imposed formula for the bureaucracy persists, largely unreformed. Its elite, generalist, status- cadre- and grade-conscious culture (now much less well remunerated than in the past) has not taken kindly to changes in local government power structures at district level. Chief administrators (DCOs) are accountable for the first time to an (albeit indirectly) elected Nazim (mayor).

Capacity building in Pakistan has been traditionally 'formal, career development promotion-oriented training'. The many donor-supported initiatives (summarised in Appendix 5 of the Pakistan report) have tended to support traditional formal approaches, using off-site training as the usual vehicle. The results have not been encouraging. This lack of success and poor impact on performance is in part related to the methodologies used in these programmes. A more powerful explanatory factor is the non-supportive context in which trainees work, and the lack of performance incentives.

In Ethiopia, chronic under-investment in education in the past and - especially since abolition of fees expanded enrolments - the overloaded teacher workforce have meant that the education system has risen from a low base. The high political priority accorded to education (and the fact that it seen as integral - subordinate even - to the national capacity building strategy) has meant that the 'spotlight' has been on the key players at the regional state and woreda levels. They are empowered in terms of autonomy over resource use (although they are very short of resources) and have unambiguous responsibilities to deliver better education to a public that backs the education of their children with their own cash. These players 'have nowhere to hide' in accountability terms. They innovate and try to do their best as a result. The indigenous gemgema system of individual performance assessment and review - introduced by one of the main liberation groups during the civil war against the Derg - is applied rigorously against this background. It appears to be more pervasive, effective and potentially intimidating than formal civil service performance appraisal systems.

Responses of local governments to the new education service delivery challenges

Comprehensive, convincing evidence is currently lacking in both countries. Neither country has objectively researched information on how and why local government performance is responding to the pressures and changes taking place. The indications are that more, favourable, and faster change is taking place in Ethiopia. The Ministry of Capacity Building plans a study of local realities and responses since 2001. Pakistan has undertaken an important baseline survey of the opinions of all district populations about the services they receive. The results will be invaluable for tracking progress with both 'demand' and 'supply' sides of the strategy for delivering better services through devolution.

Learning from capacity building experience

Neither country appears to be learning systematically from their experiences of capacity building. This is surprising, particularly in Ethiopia, where a dedicated Ministry of Capacity Building has been established. It has more positive examples of progress in practical capacity building than does Pakistan, despite a shorter history and far fewer examples of capacity building activities.

Ethiopia poses the more propitious environment for gradual build-up of local capacities - and the capacities of capacity-providing institutions - than Pakistan. To some extent this depends on recognition at federal level that innovation in capacity building approaches should be encouraged. Paradoxically, the capacity challenges appear most formidable (and under-recognised) at that level in Ethiopia.

In Pakistan, not only is the local environment for ser-vice delivery more fraught and complex in terms of perverse incentives, but also capacity building institutions are caught up in the formal bureaucratic approach to 'training as capacity building'. The mechanisms by which they are financed in the public sector offer no incentives for innovation, responsiveness to local realities, or collaboration with private or civil society sectors. Pakistan therefore faces a real capacity building dilemma. The needs are clearly huge, but the socio-economic constraints on the establishment of institutions in the vanguard of transformation of service delivery - citizen community boards (CCBs) - are ill-understood and apparently underestimated. Long experience of the National Rural Support Programme in trying to form and support community organisations is not encouraging. There are few agencies that are either equipped, or - with the style of top-down strategy planned - empowered, to tackle the demands for CCB mobilisation creatively.

Implications for donors

The implication of both cases is that donors can and should do more to ensure that national capacity building strategies evolve in the light of comparative international experience. In Ethiopia, DfID sponsored a comparative study of experiences in local government capacity building, which was bypassed due to time constraints. In Pakistan, even after so much effort and expenditure devoted to capacity building in the past, the field is 'wide open' to discussion, demonstration and analysis of constructive and positive examples from elsewhere. The present ECDPM study and case studies will provide an important resource in this regard.

Epilogue, April 2005

At the request of DfID Pakistan (DfIDP), between December 2004 and January 2005 the authors compiled a supplementary policy paper based on the Pakistan/Punjab report for consideration by Punjab provincial government.¹ The paper focused on two positive case studies of capacity building in Punjab: the Project Monitoring and Implementation Unit (PMIU) of the Punjab Education Sector Reform Programme (PESRP), and the Policy and Strategy Unit of the Strengthening Decentralised Local Government in Faisalabad project.

Comparison of the case studies in Ethiopia and Pakistan :

Feature compared	ETHIOPIA Amhara regional state	PAKISTAN Punjab province	Analysis or conclusion regarding significance
Basic data & key education performance indicators2	Population: country 67 million, Amhara 17 million GNP per capita 2002: $100 / ppp $720 Aid as % of central government expenditure 2001: 39.3% % literacy 2001: aggregate 40%, f/m,3 67% % GDP on education 1998-2000: 4.8% % budget on education 13.8% Primary school completion rate 2001: 24% Primary school enrolment rate f/m 2001: 77%3 Public/private schools (Amhara) 12,471/491	Population: country 145 million, Punjab 70 million GNP per capita 2002: $410 / ppp $1940 Aid as % of central government expenditure 2001: 16.2% % literacy 2001: aggregate 44%; f/m, 49% % GDP on education 1998-2000: 1.8% % budget on education: 7.8% Primary school completion rate 2001: 59% Primary school enrolment rate, f/m 2001: 74%3 Public/private schools (Punjab): 63,000/18,400 (under-counted: Management Association estimate = 55,000)	Both countries are 'low-income countries', with Human Development Index rankings: Ethiopia 169, Pakistan 144 (out of 175 countries). Pakistan is richer, more indebted, and less aid dependent, but spends significantly less of its GDP per capita and budget on education, and its literacy rate is only slightly higher than that in Ethiopia (and comparatively worse for women; women in Pakistan are half as likely as men to be literate). It has a significantly better primary education completion rate, but proportionately fewer girls are enrolled than boys.
Political context and background (especially regarding devolution)	The country was not colonised for a significant period (except for five years during the Italian invasion during World War II). The current government is led by freedom fighters who displaced a dictatorial military regime in 1992 after a bloody civil war from 1975 to 1991. The 1995 Constitution provides for a federal structure: federal and regional state constitutions, and elected assemblies/councils. In retrospect, 1995-2001 was a period of deconcentration below regional level: the regions were dominant players. After a crisis in the ruling party EPRDF, 'renewal' in 2001 put greater emphasis on political and fiscal devolution to the woreda and kebele levels4 to improve service delivery. The politburo 'purge' left the EPRDF united and stronger. This, and the placement of key officials at federal, regional and woreda levels (including the Ministry of Capacity Building and its bureaux) meant that policy and priorities for capacity building could be taken forward consistently at all four levels of government.	Colonial British administration up to partition from India in 1947. The 1973 Constitution recognises only federal and provincial levels. After independence, successive governments never effectively devolved power or significant resources to local governments; any reforms were introduced in top-down fashion. The military government that took over in late 1999 introduced constitutional amendments, including limitations on the powers of provincial governments to amend local government ordinances without presidential approval for a fixed period. A devolution plan was introduced by the President through the NRB in August 2001 for community empowerment, bottom-up accountability, and improved service delivery. The role of the province was imprecisely defined in this plan. Emphasis was instead placed on new bodies outside government, including citizen community boards, whose roles were to apply pressure on (local) governments, and identify and develop projects and initiatives, which qualify for funding from the 25% of local development budgets reserved for that purpose.	History matters for capacity development, in terms of national 'stock' and 'ownership', policy towards capacity building, and abilities to absorb change. While both countries experienced a 'shot in the arm' with devolution and capacity-building imperatives at about the same time, the political 'ownership' of the national goals towards which policy was working differed: Ethiopia's was more widely shared (through the EPRDF's party discipline and communication chain) and apparently consensual than in Pakistan. It has appreciably 'deeper' representative democracy structures than Pakistan: the ratio of the population to each locally elected representative is 1:200 in Ethiopia, compared with 1:1000 in Pakistan. In the latter case, suspicions of the recentralising motives of the bureaucracy persist. The approach adopted in Pakistan could be characterised as introducing devolution in spite of the bureaucracy, not through it. There was considerable opposition to the devolution plan on the part of vested interests within the political and bureaucratic establishment, particularly at provincial level, which many thought stood to gain nothing in terms of power or resources from the plan. since 2001 more appears to have changed operationally at local level in Ethiopia than in Pakistan.
Key institutions introduced as part of devolution policy and strategy	• Ministry of Capacity Building (and its regional state and woreda level offices); • education and training boards (at woreda and kebele levels)	• National Reconstruction Bureau (spearheading devolution) • National Commission for Human Development (promoting human development incubators) • Devolution Trust for Community Empowerment (promoting citizen community boards) • elected union councils • school councils	Both countries embarked on devolution at about the same time (2001) although for different reasons. New institutions were introduced in both cases. Ethiopia's new institutions were both within government (at all levels), focused on promoting accountability, and outside government (education and training boards). Pakistan's institutional innovations were driven from the centre, principally via the NRB, NCHD and DTCE, with the aim of raising awareness, empowerment and mobilisation to enhance accountability of public bodies. Over 6000 elected union councils were a crucial aspect of this local empowerment thrust.
Main changes brought about by devolution	• Unconditional grants to woredas in four main regions (and Benishangul-Gumuz). • Responsibilities for planning and managing basic services assigned to woredas. • Formalisation of kebele level of government with elected bodies, and distinct role and institutions • Reserved seats for women at regional and sub-regional levels (approximately 25%)	• Establishment of popularly elected union councils (6000+) • Reserved seats for women and vulnerable groups (33%) • indirectly elected heads of district councils (Nazims) • Reduction in powers of district coordinating officer and subordination to Nazims.	Both countries' reforms recognised that government needed to be more accountable to the public if services were to be more responsive. In both cases, women were to be actively supported. Only Ethiopia's reforms provided discretionary financial resources to local government, and local authority over staff. Pakistan's local governments remained restricted in terms of authority over staff, and discretionary finance.
Notions of capacity and capacity building	The current capacity building strategy evolved after considerable dialogue with a variety of national and international stakeholders. It has three (ideally mutually supportive) elements: human capacity, systems and procedures. The new system of unconditional block grants has made an important contribution to development of capacities at local level. Ethiopia has an indigenous system of appraisal of an individual's contribution to the organisation's goals, and working relationships with colleagues (gemgema).	No formal definition or apparent discussion of what capacity or 'capacity building' is. The NRB chair saw it as a process of popular empowerment to provide the ability and confidence to hold government to account. Otherwise, particularly in the public sector, it is seen as (often off-the-job) 'training' to provide knowledge and skills to perform jobs better. There are some indications that a reduced appetite for and cynicism are setting in regarding the over-supply of 'training'.	Ethiopia focused explicitly on the 'capacity' issue as an element of its development strategy and, at its own initiative, sought international assistance to support it. In Pakistan, traditions of formal training within the civil service, and the history of many donor-supported interventions, has led to the belief (verging on faith) that 'training' is a good thing. This has contributed to an almost ritualistic approach to training.
Capacity challenges and roles at various levels	There are indications of capacity constraints, functional overload and detachment from local realities at the federal level, particularly in the Ministry of Education (MoE). The Ministry of Capacity Building (MCB) apparently exhibits these features to a lesser degree. At the regional level, and recently at woreda level, although capacity constraints are still real, pressures of political accountability are keenly felt, and appear to be acting as a spur to performance within (pressing) resource constraints. The political consensus (that devolution must be made to work, and local accountability structures have an important role to play in this) is a helpful feature of this context. Although we have witnessed determined attempts to make devolution work, and heard a variety of descriptions of how the various actors at local level are responding to the policy, there is little objective, rigorous comparative evidence of how levels of local government are responding (and performing) under devolution. The MCB plans such a study. There is little detailed information available to analysts (government or donor) at the centre of magnitudes of financial resource flows for education services to, and their use by, local governments.	Education was organised at provincial and local levels even before devolution. But there is still a (diminishing) tendency for 'vertical programmes' to be run from federal and provincial levels, and a (continuing) tendency for provincial interference in personnel and financial management matters of districts, both of which undermine the spirit of devolution. The role of the federal and provincial levels have not been officially reappraised since devolution was introduced in 2001. District-level functions in education are poorly defined. The ECDPM case study proposed a more comprehensive set of district functions for official consideration. Although the capacity burdens are at district and union levels, learning from experience is taking place slowly, against a backdrop of federal suspicion of bureaucratic obstruction of devolution and popular empowerment. An official response is awaited to a major study of progress in devolution by the World Bank, Asian Development Bank and DfID. This study did not focus on capacity issues per se, but sought to analyse the institutional, resource management, political and bureaucratic factors that are impeding performance in the devolution policy context.	Devolution has led to the biggest changes at local government level. Local government reactions to these challenges are not documented in Ethiopia, but better studied in Pakistan. In both countries, the implications for change and performance improvement at the federal (ministry) level are not being pursued with as much attention as they merit. There is a need for learning and sharing of experience on capacity-related issues at local level in both countries. The policy imperative in Pakistan is to establish and support bodies outside local government structures (CCBs) to put pressures on it to perform, in a hitherto inauspicious environment for local accountability and responsiveness. In Ethiopia the major challenge is to relieve the chronic overload on the primary education system capacity through provision of more financial resources (to permit deployment of more teachers). In better-off regions, communities demonstrate a sometimes remarkable willingness to contribute to the education infrastructure and even materials.
Examples and approaches to capacity building adopted so far	The 'capacity' for education services rests predominantly in the state; the private sector is comparatively a minor player. There have been a few major (bilaterally supported) capacity building efforts. One imminent public sector capacity building initiative is related directly and indirectly to devolved education services. Several analyses of capacities and performance in the sector have been undertaken, most (but not all) with donor support. While there are no impact analyses of these programmes, there appears to be appreciation of the way they have contributed to capacities at regional and sub-regional levels, and some evidence of positive outcomes on performance (especially on financial management). There was no evidence that the results of, and lessons emerging from these initiatives are being shared. A DfID-supported inventory of capacity building challenges, experiences and capacities in Ethiopia and elsewhere was undertaken, but it was not given the exposure planned. It appears that the 'learning from experience' function of the Ministry of Capacity Building is yet to be performed: there is no evidence yet that the agency is acting as a 'clearing house' for capacity building experiences. Its regional offices have supported capacity building of accountability-enhancing bodies.	Pakistan has a long history, and many examples, of capacity building initiatives within the state sector, usually involving major training components, often with donor support. The civil service has a well-developed network of formal training establishments. Needs analyses appeared to ignore fundamental operational, management-ownership, systemic, regulatory, authority-over-resource, 'political' and incentive obstacles to application of knowledge and skills derived from training. Conclusions of impact analyses of training or capacity building have generally been negative. Several programme evaluations identified obstacles to the effective translation of training programmes into improved job and organisational performance, or indicated that there was no observable evidence of behaviour change. One positive example (the training of enumerators for the service delivery survey) was not itself a primary capacity building initiative, but is nevertheless symbolic of what can be achieved when operational factors are favourable, and implementers are not constrained by prevailing conditions in the public services. There was no evidence of these evaluations being shared, or their conclusions being discussed. In only one case was a decisive management action taken as a result of an evaluation (non-continuation of the in-service teacher training programme). While the Devolution Support Programme is attempting to develop a 'demand-led market' for training, there are no indications of how applicants (including district governments) will be informed of the negative results of earlier efforts.	The emphasis on building capacity of (education) service delivery systems is a relatively recent phenomenon in both countries. Neither country has yet analysed its experience with capacity building, or examined its apparent impact on organisational or individual performance. The private sector in Pakistan is a major component of aggregate education sector capacity. There is little evidence of constructive state regulation and support of this capacity. Pakistan has more experience with training (labelled there usually as capacity building) efforts than Ethiopia, and these initiatives are well documented. Pakistan has introduced the notion of service delivery surveys. Through the application of systematic training, capacity in Punjab and elsewhere has been established to repeat these surveys. This offers the prospect of regular monitoring of trends in public opinion, and more evidence-based dialogue between (political and bureaucrat) policy makers, service providers and users. User surveys are planned in Ethiopia as part of the public sector capacity building programme. The WB/ADB/DfID devolution study analysed factors that are impeding the implementation of the policy, and cast light on numerous systemic obstacles. Only one distillation of previous capacity building and/or training activities has been attempted, in Ethiopia. This also addressed how best to address the capacity implications of devolution. Even this was not shared adequately or discussed as widely as intended. In both countries, there is potential for much more discussion of what capacity building is, and what lessons can be drawn from earlier experiences.
Significance of factors affecting capacity, change and performance	Factors that have led to some 'successes' and have been appreciated, from the experiences of several bilaterally supported local government/ education service capacity building initiatives appear to have been: • consultant/trainers' long experience and knowledge of local realities, and customising new or modified systems to those realities; • involvement of regional offices in programme preparation; • follow-up of off-job training in the workplace by trainers to facilitate application of new systems or skills, and 'troubleshooting' problems; • the timeliness of these initiatives, in relation to pressures for accountability from the top as well as the bottom of the delivery system; • approaches are practical and relate directly to tasks and outputs that participants have to tackle at work; • groups of colleagues are invited to be trainees and then work as teams.	Positive factors that led to the effective preparation and building of capacities of CIET's survey enumerators included: • clear specification of the tasks to be undertaken based on pre-tested survey instruments; • trainees qualified to undertake the training and roles; • only those who performed well were selected for field teams; • practical, task-based simulation training sessions were held under supervision and with feedback, in off- and on-site conditions, until tasks were demonstrably mastered; • staff who did not meet defined performance criteria in the field were dismissed and replaced; • close supervision. The WB/ADB/DfID study of devolution cast light on how the influences on capacity, change and performance have so far appeared to operate negatively in Pakistan. All of the factors in the ECDPM model appear to be significant.	There are positive examples of building capacities in both countries, but in neither case have the factors that led to success been effectively or explicitly distilled and 'publicised'. Nor have the implications for future (larger-scale) efforts been discussed and internalised by the key players. The WB/ADB/DfID study in Pakistan clarified the factors it identified as fundamental problems. They help to explain why, without urgent and critical consideration, conventional 'training' approaches to capacity building will continue to produce disappointing results in terms of organisational and individual performance.
External context	Several factors provide a generally supportive context for capacity development at local and regional levels in the medium term, including: • the clarity of regional states' autonomous status provided by the constitution; • dominant party political consensus on the importance of education in nation and democracy building; • long-standing education policy • unconditional fiscal devolution; and • strong local accountability HIV/AIDS is an increasingly significant negative influence on capacity development and maintenance at all levels. One of the most pressing external factors affecting capacity and performance in the education system is the poverty of the country in terms of financial resources, in particular the dearth of resources for financing education services. Quantitative pressures on teachers, and the poor state of infrastructure are severe constraints. As 'supplier' of future generations' core professional capacities, the higher education system needs an overhaul. The costs of higher education are high; its efficiency, accountability and responsiveness poor.	The external environment for capacity development to enhance performance at local level for service delivery is in general adverse in Pakistan. The bureaucracy still exhibits characteristics forged during colonial times: it is an elite, with a bias towards generalists, beset with a cadre system that is counter-productive; 'bottom-heavy' (excessive numbers of unskilled officers), and (therefore) prone to 'political' pressure for patronage-based employment, and to political interference in staff administration. In the past low priority has been accorded to education development at federal level (regardless of which political party was dominant). The recent demonstration of political preferences at the local level is not favourable to education: short-pay-off projects in water or electrical infrastructure tend to be preferred. Public awareness and expectations of, and amount of information about, education service delivery are modest. Socio-economic, socio-political, literacy and gender disparities are pronounced. Recent experience of the Rural Support Programme indicates that this has affected the capacity of local communities to forge social partnerships for a common purpose.	Like Ethiopia, Pakistan has no history of effective devolution of political and fiscal powers to elected local governments. In both cases, therefore, the capacity challenges of a devolved education service delivery are significant. However, the balance of prevailing external contextual factors implies that Ethiopia is in a better position than Pakistan to make strides in building capacities, and in improving system performance at the local level. The factors that constrain progress in Pakistan are inherently political, bureaucratic and socio-cultural. This means that they are less amenable to resources- or technically oriented inputs than those that constrain Ethiopia. The strategy employed by the current Pakistan government - community empowerment through the establishment of CCBs - has to be implemented in a socio-economic and local context where past experience indicates that newly forged social groupings are fragile and take time and effort to sustain.
External interventions	All major recent capacity building initiatives, respective needs analyses, and programme design exercises have been undertaken with the help of donors. Notwithstanding a gradual tendency for donors to adopt budget support modalities, 'projects' survive in significant numbers in the Ministry of Education. The Public Sector Capacity Building Programme (PSCAP) and Education Sector Development Programme (ESDP) are intended to address a history in the 1990s of ad hoc, poorly coordinated 'gap-filling' exercises in which donors pursued their own priorities and fads, and of the passive, laissez-faire attitude of the government. Donor preferences for SWAps (represented by the ESDP) carry risks of duplication of capacity building efforts in generic functions of local governments (e.g. financial management). These risks appear to be mitigated in Ethiopia by positive capacity building progress in this financial management function through the Decentralisation Support Activity, a USAID-supported programme under the Civil Service Reform Programme. The ESDP is a joint donor-government programme with an elaborate design and joint review mission mechanisms. 'Tools' to support capacities for programme implementation are being developed with external agency support (e.g. the ESDP project implementation manual). Even within elaborately constructed 'SWAp' vehicles (such as PSCAP) donor coordination and harmonisation - especially regarding TA operations - is elusive. Bilateral agencies tend to retain control over TA resources and are apparently reluctant to pool them. The protracted preparation process (18 months) of the PSCAP artificially raised expectations of the Regional Capacity Building Bureaux (RCBBs), but never engaged effectively with key service sectors (education and health), nor did it take on board the important questions regarding capacity development strategy posed by a DfID-sponsored international comparative study of local government capacity building. Appendix 5 of the Ethiopia report sketches the problems, delays and frustrations associated with the interplay among donor regulations and internal policy conflicts, limited government management capacities, and stilted coordination between donors and 'their' projects and consultants.	There have been many externally supported initiatives, some of them very large (e.g. the Social Action Programme), to support investment in capacity development to improve service delivery systems. (Capacity) needs analyses do not appear to take full cognisance of the factors that constrain public sector capacities and performance. A dichotomy is becoming apparent between the government and externally supported approaches to capacity development. Most government resources are directed to efforts to build community organisations (CCBs) or local partnerships between government, civil society and the private sector (National Commission for Human Development incubators). External agencies have supported public sector administration capacity development in the main. There is a growing awareness among donors of the apparently limited effectiveness of capacity development efforts in the past, and is leading to some innovative approaches. For example, • In the Decentralisation Support Programme (DSP), the ADB emphasises stimulation of demand for support in building capacities. It is using a capacity-oriented programme as a vehicle for stimulating policy dialogue between stakeholders at different levels of the state system. • DfID is supporting a 'third-party' evaluation of the in-service teacher training programme in Punjab. It is also supporting an experimental 'organisational development' approach to LG development in Faisalabad, designed to avoid the usual problems with TA, and to encourage facilitation of indigenous capacities and knowledge, rather than its substitution by external 'products'. • The (World Bank-led) Devolution in Pakistan study is unusual in that it was spontaneously requested by government, and was rigorously conducted 'in-house' by the staff of three cooperation agencies (not consultants). It was therefore more thorough than those agencies would have permitted consultants to be. The study is beginning to have positive spin-offs in terms of raising awareness of devolution policy and service delivery within the three agencies. • The President has just established a high-level group to review progress in devolution. Ostensibly, the Devolution report will be considered by that group.	The governance-related influences of historically and externally induced systems (particularly the bureaucracy) have been strong in Pakistan, but relatively weak in Ethiopia. This is despite the latter's relatively higher current dependency on donor support. Both countries illustrate some of the characteristics of donor programmes and their foibles, which are not conducive to effective capacity development: • A drive for comprehensiveness and complexity. • Pressure for measurable results in a limited time frame. • Consequent succumbing to the temptation to use 'PMU' approaches (project management units outside and in parallel to counterpart state agencies). The results at least from the Punjab case are not all negative, however. But in Ethiopia the project modality still aggravates severe capacity constraints at the centre. • The lack of appreciation of the extent to which 'external' political, historical, and systemic/context factors may constrain the responsiveness of individual sectors or organisations in a state bureaucracy, and their willingness to change and adapt. • The inability or unwillingness to terminate or interrupt support when significant mutually agreed terms are not met by government. • Past interruptions in donor support, for reasons other than poor progress or weak compliance in joint development programmes, which have disrupted ongoing capacity development programmes and affected mutual trust between donors and governments. • Apparent difficulties in pursuing policy dialogue on the basis of comparative international experience (for reasons which are unclear). • The lack of consensus within donor agencies about the merits, desirability and feasibility of devolution as a means towards the end of improved service delivery. On the positive side, donors have a crucial role to play in supporting objective evaluations and impact assessments of past capacity building efforts. It is unfortunate that this has not yet happened in Ethiopia, and that in Pakistan, the results - with one exception - appear to have been inadequately disseminated, discussed and acted upon.
Stakeholders	Roles of the two main protagonists: federal and regional state governments are constitutionally defined. Lower-level government roles in education services are defined clearly in MoE guidelines. Changes in the role of federal government agencies/ ministries, and restructuring due to devolution (and capacity implications) appear to have been little discussed (up to now). The MoE appears to have major capacity problems, which may or may not be addressed by current internally commissioned organisational analysis activity. Most MCB efforts appear to have been devoted to PSCAP formulation, and setting up regional bureaux and woreda offices. The MCB appears little engaged in monitoring or learning from ongoing (donor-supported) local government/education capacity development efforts. For both ministries, keeping in touch with diverse regional developments will be a major challenge in the future. The Amhara regional state government - with its constitutional empowered status - appears to be a major source of innovation (including in performance enhancement through incentives), and to have benefited from the accountability pressure of an active regional state council. Woreda governments also have active accountability mechanisms and, at least in better-off regions, support from communities in the form of contributions for education infrastructure and even materials. Unmet or dissatisfied demand for education is not yet manifested in rural areas in private primary schools, although private secondary schools are increasingly in evidence: and not just in urban environments. Education and training boards at the woreda and kebele levels, and parent-teacher associations are empowered in relation to teachers and oversight of schools. The MCB and its regional bureaux have been charged with their development and strengthening.	At the federal level, the NRB has played a crucial role in designing the devolution policy, putting systems in place, orienting key players at all levels, driving forward implementation of the Presidential vision of community empowerment (through CCBs and service delivery surveys), and the production of an impressive array of guidelines and training manuals. There is notable (mutual) distrust between the NRB, provincial governments, some politicians and key cadres of the bureaucracy. This is manifest in the NRB's reluctance to countenance practical objections or suggestions and, in certain cases, even to engage in discussion about the implementation of aspects of policy components. The federal MoE has yet to address the need for it to change its role and restructure accordingly. The key stakeholders at provincial level appear to be politicians (Chief Minister of Punjab and MPAs). The provincial education department (as is normal among provincial departments) routinely interferes with district government because it still effectively controls their senior staff. The chemistry of the interaction between the (new) political player at district level (the Nazim) and the (newly constrained) district coordinating officer is fundamentally important as a determinant of how services are delivered. The private sector is a major player in education in the Punjab, but it is yet to be adequately and constructively regulated and encouraged by the provincial government. Teachers are important politically influential actors at provincial and district levels. They have tended in the past to dominate proceedings of (now defunct) school management committees.	The 'centre of gravity' of education service development appears to be the woreda government in Ethiopia, which is on the whole constructively supported by regional state governments. In both countries, the role of the federal Ministry of Education appears overdue for review, restructuring and rationalisation of staffing. The private sector is important as a service provider, but it is not a politically influential player in the education sector in Pakistan. In Ethiopia it is growing in quantitative importance only in secondary education. Teachers in Ethiopia have so far been more amenable than their counterparts in Pakistan in performing their jobs, and in cooperating with employers and local communities. The 'gemgema' system of organisational and individual performance assessments may have played a role in this. Since 1998 PTAs have been more empowered than their equivalents in Pakistan. In both countries, the importance of the role of the head teacher as a leader, motivator, positive example, and quality controller is becoming more apparent, and attention is at last being given to them as a focus for assistance and advice.
Internal management	We concluded there are multiple dimensions to the problem of structure and staffing of the federal civil service including the lack of management and support staff of required competence to support technical professionals; and 'people management' factors (e.g. unwillingness to delegate or plan work). Organisational development approaches to capacity building (advocated by the DfIDE-sponsored study of local government capacity building) appear to be worth trying. The MCB is clearly responsible for the coordination of capacity building strategies. Learning from ongoing (donor-supported) programmes should come within this brief, but they have attracted little attention so far. A series of ongoing (and hitherto unevaluated) approaches to improving human resource management is being piloted at federal level. Regional governments (with their own civil service commissions) are not bound by these experiments and are pursuing their own. Emerging problems with the management of information between levels of government may be addressed by future activation of VSAT satellite connectivity of all administrative centres. This facility offers significant, but only partially explored, potential for capacity development. Procurement procedural problems will emerge in poorer regions and woredas where there will have to be more reliance on contractors for infrastructure construction. Staff 'pooling' systems for common services in the woredas appear to be a pragmatic approach to staff shortages, and the limited budgets to employ them. Staff turnover problems are unquantified at present. At woreda level they could well emerge in cadres provided with commercially valuable training in future.	The WB/ADB/DfID Devolution study devoted a lot of attention (50 pages) to the issue of incentives, particularly perverse or absent incentives to performance at all levels of the delivery system, but particularly district level. It concluded that devolution has made little impression on prevailing patterns. Morale amongst senior administrative staff at district level has suffered. Their authority in relation to elected Nazims has been reduced. Education officers' loyalties are more to provincial education offices, rather than district governments. There are few matters where district administrators have discretion without referral upwards to the province. The budgeting system is dysfunctional, split into recurrent and development expenditures. The former is largely payroll and is thus controlled by the province; the latter is very limited and often allocated without regard to the routine budget, or development priorities, but instead divided equally among council members, who then allocate it to 'pet constituency schemes'. Absenteeism among teachers is a major problem. The significant factors that have affected performance, morale and professionalism are: • the perceived (low) status of teaching as a career; • loose or non-existent supervision; • weak local accountability; • often very poor infrastructure; • absence of effective in-service training and support; • political influence; and • weak voice mechanisms at the level of monitoring committees at district and union levels, and the new formula for school councils in the Punjab is largely untested. Many members of the 'stock' of education officers are unsuited for their managerial responsibilities: they are approaching retirement, have no management background, and are political pawns manipulated by council members and under the influence of MPAs.	The basic patterns of accountability at local government level - the key variable in 'internal management' issues - are sound in Ethiopia, but not in Pakistan. In the latter case, this affects incentives perversely, with disastrous results. Unless the issues researched and analysed at length in the Devolution in Pakistan study are addressed, conventional approaches to capacity building will (continue to) be futile. In Ethiopia, the centre is in most urgent need of analysis of the causal factors behind its endemic weaknesses. Problems at regional and woreda levels are more amenable to conventional injections of resources, guidance, equipment and supervision from the regional level.
Core conclusions	We conclude that: • No definitive causal connections can be drawn about improved performance, (past) capacity building inputs, or the changes brought about. Little learning (about what works, what doesn't and why) in capacity building is taking place. Firm evidence related to service delivery performance and the underlying factors is also scarce. • The reform efforts being undertaken are serious, and fully 'owned' throughout government structures. • Since the late 1990s there has been a marked change in the attitude of regions towards the woredas. • The existence of a range of positive, mutually supportive factors in favour of organisational change and improved performance, including: - unconditional block grants for woredas - firm constitutional underpinnings of regional government; - (political) consensus on education and its priority at least since the late 1990s; - the possibility of incentives for, and priority attached to, capacity building; - the low incidence of petty We conclude that: • No definitive causal connections can be drawn about improved performance, (past) capacity building inputs, or the changes brought about. Little learning (about what works, what doesn't and why) in capacity building is taking place. Firm evidence related to service delivery performance and the underlying factors is also scarce. • The reform efforts being undertaken are serious, and fully 'owned' throughout government structures. • Since the late 1990s there has been a marked change in the attitude of regions towards the woredas. • The existence of a range of positive, mutually supportive factors in favour of organisational change and improved performance, including: - unconditional block grants for woredas - firm constitutional underpinnings of regional government; - (political) consensus on education and its priority at least since the late 1990s; - the possibility of incentives for, and priority attached to, capacity building; - the low incidence of petty political interference; - public pressure and accountability; and - the fact that major concerns about 'capacity' are at the centre rather than at local level.	Our conclusions are that: • No definitive causal connections can be drawn about improved performance, (past) capacity building inputs, or the changes brought about. Little learning (about what works, what doesn't and why) in capacity building is taking place. • Incentives facing principal stakeholders and players matter. The factors that limited the effectiveness of a range of past capacity building efforts still appear to be as real now as they were then. Factors that have been mentioned as important negative influences on potential for delivery performance improvements include the lack of: - provincial ownership of the reforms; - regime stability; - political maturity; - links between plans and budgets; - continuity of senior staff; - trust - integrity and objectivity; and - and management authority over staff. • Given the magnitude of the political imperative for communal empowerment and accountability, time is of the essence, but social re-engineering takes time. • Cynicism about 'training' is (justifiably) setting in; 'capacity building' is becoming a devalued currency. The recent $60 million federal government programme (with minimal conditionality) sends the wrong signals.	The cases do not reveal reliable data or empirically backed insights into the relationship between capacity (building) and the performance of organisations or service delivery systems. They do, however, cast light on the conditions under which capacities can be said to exist or to thrive, if given certain types of support. All four factors hypothesised to pay a significant role in affecting the relationship between capacity, change and performance appear to be important. Their relative importance varies according to the historical, constitutional political and socio-economic context. Political competition or rivalry can be deeply problematic, particularly if it extends to interference in administrative or staffing matters. This is compounded if popular accountability is weak, and if resources are not 'cleanly' and unambiguously devolved to the level of government at which they need to be allocated, utilised and accounted for. Mechanisms exist, and are beginning to be used in both countries for conditional grants which - for the first time in the case of Pakistan - are ensuring that a core national priority is adequately resourced. There appears to be scope in both countries for internationally referenced high-level policy dialogue on international capacity building 'lessons of experience' to which the present ECDPM study could contribute. This dialogue would provide the first-ever opportunity for the stakeholders to reflect on 'what works and why' in capacity building.

Notes


1 David Watson and Adnan Khan (2005) Capacity and Capcity Building in the Punjab for Devolved Education Service Delivery: Comparing current approaches with international experience; deriving possible pointers for policy and future action in the Punjab. Discussion Paper for DflD Pakistan (available on request from Nargis Sultana at DflDP).

2 In order to ensure comparability, data taken from the World BankWorld Development Report 2004 (Tables 1-7) unless otherwise stated.

3 From UNDP Human Development Report 2003.

4 Government structures in the two countries compared in Figure 1.