This publication should be cited as: Loquai, C. and Le Bay, S. 2008. Building capacities for monitoring and evaluating decentralisation and local governance. Experiences, challenges, perspectives. (ECDPM InBrief 19). Maastricht : ECDPM

> Building capacities for monitoring and evaluating decentralisation and local governance. Experiences, challenges, perspectives. (PDF)

This InBrief is aimed to stimulate the debate on developing local capacities to monitor and evaluate decentralisation and local governance processes. It draws on the results of action research jointly conducted by the Malian Ministry of Territorial Administration and Local Government (MATCL) and the Réseau de Réflexion et d’Echanges sur le Développement Local (REDL), a Bamako-based network of development organisations. The SNV Netherlands Development Organisation-Mali and ECDPM were asked to facilitate this process of joint stock-taking and analysis, which resulted, among other outputs, in the publication of eleven case studies presenting experiences and study results from six West African countries.

Introduction

In the context of the democratisation movements of the late 1980s, many African countries launched a new generation of decentralisation policies aimed to build or strengthen more democratic, participatory and accountable forms of governance. In the years since, these reform efforts have attracted substantial development assistance provided by the international community.

Implementation of decentralisation programmes, particularly the creation or strengthening of local government, with elected councils, legal persona and own resources, has changed these countries’ institutional landscape and governance at the central and local levels. Discourse on the advantages of decentralisation has also raised hopes that local governance will become more responsible and responsive to citizens’ needs and thus contribute to improving their living conditions. However, the commitment and pace of reform has varied considerably from one country to the next.

For their part, donors and development agencies have shown increasing interest in assessing the results, outcomes and impacts of decentralisation, including the related development assistance (Steinich 2000, Sebahara 2004, Hutchinson and La Fond 2004). This tendency, and donors’ search for appropriate assessment methods, is in line with the present concern for aid effectiveness and a more general interest in gauging governance in developing countries. Donors and development organisations are now also displaying increasing willingness to invest in strengthening monitoring and evaluation (M&E) capacities of local stakeholders in decentralisation processes and citizen control with a view to building local accountability systems (Hilhorst and Guijt 2006, Massuangahe 2005). At the same time, many donors are seeking ways to render their own M&E systems more participatory .

The African governments involved in democratic decentralisation also acknowledge the need to invest in national and local capacity to monitor and evaluate some of the changes these reform processes have induced.

This InBrief examines a number of initiatives to build the capacity of local stakeholders to monitor and evaluate decentralisation and local governance processes. It builds on the results of case studies done in six West African countries, as well as discussions of these studies at a regional seminar in Bamako, 17–18 May 2006. This event provided an opportunity for structured exchange and learning from the cases and experiences presented.

The context of decentralisation and local governance in West Africa

Decentralisation and local government in West Africa are anchored in different traditions, spanning pre-colonial authorities, colonial administrations (mainly French and British) and post-independence decentralisation and local government reform efforts. Since the early 1990s most countries in this region have formulated new decentralisation policies aimed explicitly to promote democratic and more participatory forms of local governance.

Yet for many of these countries, the road of reform has been bumpy. Decentralisation processes have been stop-and-go rather than following a linear path. This was perhaps to be expected, in view of the complexity and multidimensional character of decentralisation reforms. The francophone countries of West Africa had particularly major reforms to undergo. Upon independence, local government in these countries was confined to a small number of urban municipalities, while most of the predominantly rural population was administered by state delegates. These rural residents had no right to vote and little access to basic public services.

Since the early 1990s the situation has changed. Democratic decentralisation has been anchored in constitutional laws and creation of hundreds of new local governments. For example, Mali alone established more than 680 new rural municipalities. Free and pluralist local elections have been held and local governments have been made responsible for planning, implementing, monitoring and assessing progress in development at the sub-national level. The hope is that elected local governments will be more accountable to citizens and more easily controlled than central state administrators.

Table 1 presents some basic information about decentralisation and the nature of local government in the six West African countries in which case studies were conducted.




Why invest in building local capacities for M&E of decentralisation?

Why are donors thinking about M&E of decentralisation? Why should they invest in exercises aiming to build capacities in partner countries to monitor and evaluate decentralisation? Should national and local actors in decentralising states be interested in developing systems and tools for assessing the results, outcomes and impacts of reform processes?

In answer to these questions, with reference to the case studies, literature and discussion on the subject, the following can be said:

• Donor agencies and development organisations supporting decentralisation processes want to know to what extent and under what conditions their support to these reforms can contribute to development goals, such as poverty reduction, economic development and good governance (Reyes and Valencia 2004: 69).

However, as the recently published reference document of the European Commission rightly states, assessments of outcomes and impacts of assistance to decentralisation are still ‘works in progress’ (European Commission/Europe Aid 2007: 68).

Moreover, until recently, M&E practices tended to emphasise the information needs of donor agencies and central governments more than enhancement of local stakeholders’ capacity to make their own evaluations. This approach has had limited success as far as ownership and utilisation of evaluation results are concerned. It relies more on external experts than on local knowledge and has failed to contribute enough to strengthening local systems of accountability (Watson 2006: VIII, Simon 2004: 91). These latter, however, are highly desirable in the context of projects and programmes to promote democratic decentralisation and local governance.

• National authorities dealing with decentralisation are sometimes portrayed as reluctant to invest in M&E. However, this image has been countered by the readiness of Mali’s Ministry of Territorial Administration and Local Government (MATCL) to promote a process of stock-taking and exchange of experiences with M&E tools for local government actors, as well as by the interest expressed by other African countries in the results of this exercise.

It is important to note that national authorities often have different expectations of M&E systems than donor representatives and local-level actors. The former tend to be interested in instruments that can help them to coordinate and centralise information on local government performance, as managing data from a variety of locations and providing feedback requires specific capacities.

• Mayors committed to democratic local governance increasingly realise their need for tools with which to show citizens how they are performing and why the municipality has difficulty dealing with certain issues that citizens might view as high priority. In this regard, M&E tools could help them to make progress and constraints visible to citizens and donors alike.

• To citizens, decentralisation reforms and local government will be credible only if they have sustained positive impacts on people’s lives and provide them with more opportunities to participate in decision-making or to exert influence in local affairs. Elections provide opportunities for political participation, but only every few years. Nonetheless, other channels of participation and citizen control of local government have tended to be neglected.

Experiences from the case studies

Today more than in the 1990s, development organisations are willing to experiment with methodological approaches and tools aimed to reinforce local actors’ capacities to monitor and evaluate decentralisation and local accountability structures. Recent enquiries in the West African region have uncovered noteworthy initiatives that involve local stakeholders of decentralisation in designing and testing such tools. However, as many of these efforts have gone undocumented for a wider audience, they have been largely unnoticed in the regional and international debate. Moreover, there has been little discussion of these experiences, even among development organisations and their partners within the region.

The Réseau de Réflexion et d’Echanges sur le Développement Local (REDL), a Bamako-based network of development organisations, MATCL, SNV and ECDPM therefore considered it worthwhile to facilitate a process of identification, analysis and exchange of such experiences in West Africa, to promote mutual learning.


Case studies and the stock-taking exercise

The experiences chosen for the stock-taking exercise all meet three criteria that were used to identify the cases to be studied:

• They were jointly designed with multiple stakeholders at the local level with a view to integrating different perspectives.

• They aim to develop or strengthen the M&E capacities of local government and involve different actors in the process of monitoring and evaluating decentralisation of local governance processes. These actors might include various levels of local government, de-concentrated technical service providers of the central state, communities, the private sector, supervisory authorities of local governments, local associations, traditional leaders and civil society non-government organisations (NGOs).

• They aim to promote more transparent and accountable forms of local governance and make local government more responsive to citizens.

Furthermore, all of these experiences were externally assisted by donor agencies or NGOs. Some, however, were replicated at a later stage by national or local decentralisation actors to other regions or countries without support of the initial sponsor.

In line with a common methodology suggested by the case study facilitators, teams were formed and asked to document and analyse their experiences in designing, fine-tuning, implementing and using the relevant assessment tools. The team leaders were asked to involve in the stock-taking and analysis process all of the different categories of local stakeholders who had contributed to and used the tools (Loquai and Le Bay 2005).

Four months later, in May 2006, a regional seminar to share experiences was organised in Bamako under the auspices of the MATCL. The event was attended by more than 100 participants from six West African countries, including many mayors, civil society representatives, private sector representatives and various authorities responsible for decentralisation at the national and local level. Representatives from donor agencies were also in attendance, as well as those from the development organisations supporting the decentralisation processes.

Table 2 overviews the case studies and classifies them according to the types of capacity-building tools and approaches involved in each.


Abbreviations: ANCB = National Association of Benin Municipalities, DNCT = National Directorate for Local Government (Mali), FIDESPRA = International Forum for Development and Exchange of Knowledge and Expertise Promoting Self-Sustained Rural Development, GPRD/SIF= Ghana Poverty Reduction Programme of the Social Investment Fund, GTZ = German Technical Cooperation Agency, Helvetas = Swiss Association for International Cooperation, KIT = Royal Tropical Institute (Netherlands), MLGRDE= Ministry of Local Government, Rural Development and the Environment (Ghana), MATCL = Ministry of Territorial Administration and Local Government (Mali), MATD= Ministry for Territorial Administration and Decentralisation (Burkina Faso), NCA = Norwegian Church Aid, NDPC= National Development Planning Commission of Ghana, SNV = Netherlands Development Organisation, UNCDF = United Nations Capital Development Fund.

Source: Classification based on presentations and written cases discussed at the regional seminar in Bamako, 17–18 May 2006.


Developing capacity to analyse and monitor local development at the municipal level

In many francophone West African countries, newly established local governments are learning to formulate and implement municipal or regional development plans. A common problem they experience is the lack of baseline data and statistical information to draw upon to analyse the social, economic and cultural situation in the territory. Often, national statistical systems have not been adapted to decentralisation. They might not produce sufficiently disaggregated data for local planning, or municipal level planners might lack easy access to the information that is available. Moreover, municipalities and districts often lack specialised staff to collect background data and diagnose development issues before engaging in the planning process.

The case studies from Mali, Cameroon and Niger¹ all deal with participatory approaches to establishing municipal baseline information with a view to improving planning, monitoring and evaluation of local development. With the assistance of external facilitators, the municipalities employed participatory approaches to assemble and analyse data jointly with other local development stakeholders (e.g. de-concentrated technical services, community-based organisations, village chiefs and the private sector) and to construct a baseline that could be used for strategic planning and eventually adapted for M&E purposes.

In Mali, this baseline exercise was complemented by the design and testing of geographic information systems (GIS) for rural municipalities. As feeding and manipulating a GIS requires computer literacy and basic cartography skills, primary responsibility for system updating and maintenance lies with municipal advisory centres. These are based at the district level and provide various capacity-building services to the municipalities of a given district (Dumont and Samaké 2007).

In Ghana, where districts have their own statistical services, the challenge was slightly different: development of a new methodology for poverty profiling and mapping. Enhanced information was needed on the level, causes and geographical distribution of poverty at the district level in order to facilitate pro-poor planning, M&E of district plans and pro-poor targeting of national development programmes. Here too, an important element of the approach was participation of a spectrum of local development stakeholders, such as community-based organisations, village associations, traditional leaders and NGOs, in constituting databases and district poverty maps (Dery and Dorway 2007).

Experiences with these different tools provide a number of lessons:

• The process of constructing baselines and conducting monographic studies enhances knowledge of the local governments’ economic, geographic and socio-cultural potential and thus allows it to be used more fully.

• The tools give decision-makers more clarity about the constraints that local governments face and about populations’ needs, thus providing a more realistic basis for planning.

• Such tools can help bring municipal planning better in line with national-level poverty reduction policies and with sector policies.

• In the course of the exercise, the municipal council usually assumes increasing responsibility for steering and owning the process.

• The participatory approach often contributes to development of team spirit within the municipal council, which enhances initiative.

These exercises all showed positive effects on the capacities of local governments and municipal advisory centres to collect and select relevant statistics. They also resulted in stepped-up collaboration between technical services, supervisory authorities, local governments, civil society and private sector agents at the local level.

So far, most of the baselines have been used for planning and less as tool for M&E of local development and governance. For that latter purpose, they need to be adapted, made more selective, indicator-focused and converted into a database that can be regularly updated. A step in this direction is the effort in Mali to feed data into a GIS. Plans to establish similar electronic databases were under way in Cameroon and Ghana at the time of the stock-taking exercise.



Performance self-evaluation tools for local government

In 2004, the National Directorate for Local Government of the MATCL of Mali issued a publication explaining a tool for performance self-evaluation of local governments. These guidelines were later included in a toolkit for mayors distributed to all 703 municipalities of the country.²

The tool was the result of a long process of design, testing and fine-tuning a participatory approach to performance self-evaluation, an exercise assisted by a number of development organisations, in particular, SNV, Helvetas and GTZ. These organisations all took active part in helping their partners at the local level (municipalities) to develop, test and use the approach and provided feedback to stimulate the tool’s gradual improvement and adaptation for use in different contexts.

The proposed methodology puts the municipality in the driver’s seat of an evaluation of its own performance, which is repeated at regular intervals (figure 2). Members of the local council compare their self-evaluation with evaluation results provided by various groups of other local actors, such as community-based associations, local interest groups, private sector representatives, staff of de-concentrated technical services and supervisory authorities (Le Bay et al. 2007).


The tool proposes that the self-evaluation revolve around five key areas of municipal performance: (i) internal organisation, (ii) financial and administrative management, (iii) mobilisation of financial and human resources, (iv) planning and programming of local development and (v) services, products and infrastructure. For each area a number of indicators was jointly defined against which performance could be assessed (using scores).

Experience with the tool has illustrated the important role of external facilitation and mentoring the first time the self-evaluation exercise is conducted. It has also revealed some potential pitfalls that users of the tool might encounter, such as cultural barriers to articulating and dealing with constructive criticism and being self-critical in public.

Experience has also highlighted problems that the various stakeholders may come up against if they focus solely on performance-based results and disregard the wealth of communication that takes place before and after the exercises. In fact, these exchanges may well pave the way for shared responsibility and consensual decisions.

Testing and utilisation of the tool has had many positive results on the evaluation capacity of the municipalities and others:

• Municipalities are now able to measure their performance (achievements and weaknesses) and to analyse it themselves; they have learned how to develop argumentation and mediate between different viewpoints.

• Understanding has improved of the roles of the different actors and of the legislation on decentralisation and local government.

• Municipal councillors, mayors and contract staff of municipalities now realise that they must be more accountable to citizens and supervisory authorities.

• Municipal staff have learned to use evaluation results, adapting decision-making and management in line with findings.

• Evaluation results have helped local officials to formulate better targeted and more complete requests for capacity-building assistance to municipal advisory centres.

Inspired by publication of the guidelines for the self-evaluation tool in Mali, neighbouring countries have begun to devise similar tools:

• In Niger, a number of development agencies, together with the de-concentrated state services and municipalities, started testing and adapting the self-evaluation tool in 2005. An innovative feature of the adapted tool is its emphasis on encouraging municipalities and their partners to think at the local level about integrating goals of the Millennium Declaration. The adapted tool also allows the national poverty reduction strategy to be taken into account in municipal planning, monitoring and evaluation. The present challenge is to get the tool validated by the Ministry of the Interior and Decentralisation and used on a much wider scale throughout the country.

• In Benin, mayors have strongly contested an external evaluation of the performance of the municipalities. Instead, they have mandated the National Association of Municipalities (ANCB) to design an alternative approach. In collaboration with a number of donors and ANCB branches at the departmental level, the ANCB is presently developing a municipal performance self-evaluation tool inspired by the one from Mali. The Benin tool is also to serve as a means of better targeting external support for capacity building for the country’s young municipalities.

• In Burkina Faso, the Ministry for Territorial Administration and Decentralisation (MATD) plans to make systematic use of self-evaluation at various levels of its future systems for M&E of decentralisation. The approach proposed for the municipal level builds on the Malian tool. However, in view of the very recent establishment of the rural municipalities, the self-evaluation process will initially emphasise helping these entities to reflect on the kinds of capacities they need to acquire to become functional and achieve their goals.

More recently, a development programme in Senegal produced a two-step evaluation methodology for measuring municipal performance. The first step is an obligatory external performance assessment of local governments. The second step is a voluntary performance self-evaluation conducted with a view to identifying the different needs of local governments. With the programme, the central state has shown its interest in the performance of public action at the local level. It would like to see local governments and development programmes further develop this tool so that it can be used to identify local government capacity-building needs. On the whole, the tool serves to promote good governance and performance-based allocation of resources to local governments.

These initiatives show that performance self-evaluation approaches have considerable potential for replication. This is especially so because they help to pinpoint the effects of capacity building in terms of improved performance of local government.






Strengthening citizens’ control and local stakeholders’ capacity to monitor decentralized service delivery

The two experiences presented under this heading differ in terms of their entry points, rationale and approach. Nevertheless, both have as their end goal to improve the quality, effectiveness and transparency of delivery of public services at the local government level. To achieve this objective, the case from Benin promotes citizen control in the primary education sector, while the Malian experience uses joint monitoring of basic health indicators.

In Benin, the pilot phase of a participatory local impact monitoring methodology (SILP) has involved 15 schools in three municipalities of the department of Atakora. School attendance in these in northern territories has lagged behind the national average. The trial forms part of Benin’s poverty reduction strategy, which gives priority to education and decentralisation policy.

Use of SILP is aimed at providing supplementary information for quantitative evaluations of the barriers blocking the proper operation of decentralised public services. It is also intended to facilitate identification and implementation of appropriate corrective measures by citizens themselves (Floquet et al. 2007).

For this purpose, SILP follows an iterative process of consultation and exchange, involving sector actors at a number of levels (municipal, departmental, national) and various groups of actors (e.g. pupils, teachers, parents’ associations, citizens, local government, women selling food to pupils, de-concentrated educational departments of the central state, central institutions and development partners). The stress lies on two aspects of the public spending cycle: tracing the resources allocated and evaluating service quality. Figure 3 gives an overview of important steps in this participatory monitoring process.

Both aspects are jointly reviewed by public and community service users and suppliers, applying national norms and standards and their own criteria. An external moderator facilitates discussion and evaluation according to these jointly defined criteria.

The evaluation is followed by a debate on corrective measures, which are then summarised in a collective action plan. Implementation of the plan is steered by parent associations and school administration, but jointly monitored and reviewed on regular basis with municipal councillors.

Initial results of the pilot phase of the SILP approach show that the methodology can improve knowledge on strengths and weaknesses in using financial resources devolved from the central state to the decentralised level (to the departments, municipalities and schools). Even after only a few months of testing, the method appeared to have helped various local actors to better assume their respective roles in enhancing public scrutiny of the use of public funds. The act of mobilising their thoughts and energy for a common cause also improved the efficiency of public spending. Moreover, there is evidence that the SILP approach disseminates itself, as it is now being used in municipalities that were not included in the test sample.

However, the strategy also has pitfalls. In the absence of capable moderation, latent conflicts can surface that hamper constructive discussions. Also, if not properly prepared and supervised, SILP can lead to covert tactics and exclusion of actors, instead of self-corrective strategies. External support is thus essential in the test phase and probably well beyond.

Mali’s testing of a basic health-sector information system similarly rendered service delivery more effective and transparent. However, for this exercise the initial objective was a different one. Because the transfer of resources and capacities for municipal-level health service provision is advancing only slowly in Mali, the idea was to test a tool that could contribute to the process of devolution (Toonen et al. 2007).

A basic package of information was developed for key public health actors at the local government level, including elected officers, community health associations and technical departments. The information kit, called ‘SIEC-S’,³ had been produced jointly by the Dutch Royal Tropical Institute (KIT), SNV, the Malian Ministry of Health and the community-based health associations that run many of the local health centres. Action research was then conducted on how to use this package for monitoring and managing basic information on public health at the municipal level, involving municipal councillors, the de-concentrated health services of the central state and representatives of local community heath associations (ASACOs).

The strength of the SIEC-S approach is its enabling non-specialists on health, including the illiterate, to take part in discussions on health system results, progress in public health and reasons for failures and success. As participants’ comments illustrate (see box), use of SIEC-S improved local stakeholders’ understanding of key health statistics and of indicators directly relevant to their daily work. This, in turn, strengthened municipal councillors’ ability to discern top priorities for action, to take informed decisions on health matters and to negotiate with the Ministry of Health. The joint collection, sharing and analysis of health-related indicators and information enhanced cooperation between the actors concerned. For this reason, the project has constituted a very practical experience that paves the way for the transfer of health-sector powers to local governments.


Opening external M&E systems to local perceptions

In 2004, CARE Mali made the design and implementation of participatory M&E systems for its programmes a priority of its new long-term strategic plan. As the test case for a new integrated participatory M&E system it chose the Support Programme for Municipalities and Grassroots Organisations, co-financed by the Norwegian Agency for Development Cooperation (NORAD). The focus of the programme, which was based in the region of Mopti, was natural resource management and local governance. At that time, it was still in its initial stages of implementation (Coulibaly et al. 2007).

Design and testing of the new participatory M&E system brought together a range of actors involved in resource management and new governance structures. Participants spanned the village, municipal, district (cercle) and regional levels and were drawn from both civil society and the de-concentrated technical departments of the central state. All had been intensely involved in designing the programme.

These actors were later to take on roles in the newly emerging M&E system, intended to meet both CARE’s internal needs for monitoring data and Mali’s need for better information and accountability systems for its new local governance structures.

Although the M&E system had been running for barely a year at the time of the stock-taking exercise, the authors of the case study could already point to several lessons learned:

• Participatory M&E is an effective way of transferring skills to local actors, but time and patience are required to put such an approach into practice. Especially in a poor region like Mopti, where educational standards tend to be low, participants need to be given plenty of time to absorb the information they receive.

• It is vital to choose able participants with a basic level of capacity from among the ‘beneficiaries’ if the process is to be successful. Illiteracy, for instance, has proven to be an obstacle to participants taking ownership of some of the M&E tools.

• The commitment of the steering team is a key success factor of participatory M&E. It is important that the team clearly distinguish this method from earlier, less participatory methods of managing projects.

Problems encountered during the test phase were, among others, related to the diversity of languages and dialects spoken in the region and differences in educational levels of the participants. As a solution, all important documents were translated into the three main languages (Dogon, Peulh and Bambara) spoken in the region. Moreover, at meetings the participants were divided into groups according to language spoken and educational background. This made the process time-intensive, but ensured that the people involved could communicate and make their points.

In a similar vein, to open strategic planning and M&E to the perspectives of local governance stakeholders, Norwegian Church Aid (NCA) conducted an assessment of the state of decentralisation and local governance in three regions of northern Mali (Gao, Timbuktu and Kidal) in 2005. The assessment was less anchored in a participatory M&E approach, but more focused on capturing and analysing citizens’ perceptions by means of a traditional survey approach. To get a differentiated picture of citizens’ perceptions on the state of decentralisation and local governance in the three regions and to ensure that the views of marginalised and vulnerable groups were taken on board, the researchers interviewed officers of civil society organisations working with these groups and representatives of citizens directly (Cissé et al. 2007).

The evaluation concluded that integrating an analysis of citizens’ perceptions can improve tools and current systems for monitoring and evaluating local governance. A greater emphasis on citizens’ perceptions, opinions and assessments would also better equip elected councillors and supervisory authorities to ensure that approaches adopted for governance and development are relevant, viable and sustainable.

Bearing in mind the lack of opinion polls and the very few surveys on local governance themes among electors that have been conducted in West Africa, the survey of perceptions commissioned by the NCA makes an interesting contribution to current thinking about barometers of governance.⁴

The last experience, in Mali, is from the United Nations Capital Development Fund (UNCDF), which has been active in promoting local governance in northern Mali since the latter 1990s (Sylla and Ongoïba 2007). In the context of a broader UNCDF initiative, UNCDF-Mali started an evaluation of the poverty reduction impacts of its Support Programme for Rural Municipalities in Timbuktu (PACR-T).

Data collection proved a real challenge. The main constraint was the mediocre quality of the information available on the reference situation. Baseline figures were meagre and a lot of statistical information was unusable because it was not broken down to the municipal level. To remedy the situation, the project team and external evaluators decided to try a new qualitative surveying approach drawing largely on the perceptions of local communities and a participatory assessment method.

Hence, a conceptual framework for analysing the impact of decentralisation on various dimensions of poverty was jointly designed with partners, and local perceptions were used to rank the villages and wards of each municipality in three categories, from poorest to least poor. Community participation in the planning process and use of municipal investment funds in each community were then analysed in order to identify which of the three categories had received the most investment.

A clear advantage of this method is its allowing ‘poverty’ to be defined and assessed from the viewpoint of the beneficiaries of the programme’s activities. In this case, this led to a focus on the impact of socio-economic investment, because the dimension of poverty most widely experienced in the Timbuktu region is that of gaining access to basic socio-economic services.

As this case study concludes, it is regrettable that monitoring and evaluating the impact of decentralisation on poverty reduction is often regarded as only a concern of researchers and donors. The authors argue that decision-makers in developing countries should make M&E a more systematic management practice. This would allow a counter-checking of the underlying hypotheses of development approaches. It would also contribute to improving the living conditions of poor populations and enable populations to better analyse and understand their rights and options as citizens.


Implications of the shift towards budget support

For a number of years, some donors have shown a strong tendency to shift towards budget support as a new instrument for assisting processes of decentralisation and state reform in developing countries. This is particularly true of the European Union, which is one of the largest donors, but it also applies to bilateral donors such as the Netherlands, Belgium and the United Kingdom.

This trend places new demands on the evaluation capacities of donor agencies and decentralisation actors in developing countries. Disbursement of budget support usually takes place in tranches, with the funding level depending on specific conditions being put in place and progress achieved in relation to performance indicators. The government of the partner country usually has to propose and negotiate the performance indicators with the donor. It also has to design an adequate M&E system and report on progress. Progress is then jointly reviewed with donors.

Implementing and monitoring budget support thus requires capacity to identify and systematically track key indicators of local governance. Ideally this process should involve actors of decentralisation at the national and local level, including representatives of users of decentralised public services and countervailing forces within civil society. Moreover, as mentioned above, the government of the recipient country must put in place systems for monitoring indicators and compiling information.

In 2006, the European Commission launched a pilot project providing sector budget support in Mali. This was the Support Programme to Administrative Reform and Decentralisation (PARAD). Indicators were identified in a participatory process, involving national and local-level actors of decentralisation, sector ministries and the donor community. Table 3 lists the programme’s 12 performance indicators.



Information on these indicators is provided via the monitoring system for the country’s poverty reduction strategy paper and a computerised M&E tool related to the National Programme in Support of Local Government.

The Malian experience shows that years may be required to develop and operationalise a nationwide M&E tool that involves the local government level in collecting and analysing basic indicators. Moreover, the success of such an effort will depend on existing (statistical) capacities and technical support structures for municipalities, as well as on the capacities of the central state to gradually improve its own systems of data collection and analysis. In this regard, the annual performance reviews with donors of indicators for budget support should be seen as an opportunity for mutual learning.

The first annual review of the PARAD programme indicates that it may be necessary to strengthen the capacities of the actors involved and to adapt the indicators jointly in a timely fashion, as problems of data interpretation and significance invariably arise. Moreover, it may be necessary to build capacities to verify data sources at the local level, because performance-based disbursement of budget support can introduce new biases. Hence, it makes sense to develop techniques of triangulation and cross-checking of data generated by an M&E system conceived for the review of budget support.

One way to do this is to draw on information produced by other M&E systems, such as those of bilateral donors operating at the local level. Another way is to enable the responsible central and supervisory authorities to conduct surveys of citizen perceptions. In this regard, agreements on verification of data reliability and provision of capacity-building support for central and supervisory authorities appear particularly important, because the focus on a few performance indicators can introduce prejudice and incentives for manipulation of data.


Lessons learned

The case studies and exchange of experiences brought forward a wealth of lessons learned.

The following were particularly in evidence:

• Joint design and testing of tools needs time. Designing and testing M&E tools that involve different actors in decentralisation at the national, regional and local levels takes time. When working with councillors and local civil society actors who have little or no experience with M&E tools, they need to be allowed time to learn how to identify, discuss and interpret indicators and statistics. This is a key aspect of capacity building for M&E of decentralisation and local governance. Furthermore, many cases show that trust among actors and working procedures are not built overnight. In multi-ethnic and multi-lingual contexts, time for translations is also required. Otherwise the various stakeholders involved in designing and testing a tool could feel uncomfortable interacting and articulating their viewpoints.

• Identification and fine-tuning of indicators is a process. This holds true to some extent for most M&E exercises. But it is particularly relevant for the experiences described in this brief. Decentralisation and local governance have a process dimension. It is therefore unrealistic to try to define too many indicators at the start of a test-run of a specific M&E tool. Even once formalised, it is important that a tool retain some flexibility for fine-tuning, if necessary even changing indicators to reflect the dynamic nature of reform processes. For instance, a newly established municipality may initially choose to focus performance self-evaluation on key functions, financial and resource management, running the registry office and mastering the process of development planning. Later, as more responsibilities and resources are devolved, performance self-evaluation may come to include active fields as well, such as natural resource management, promotion of local economic development and provisioning of basic services (e.g. health, education, water). New indicators would then need to be defined and tested for assessing performance in these fields.

• Strategic alliances have benefits. A number of our case studies emphasise the importance of strategic alliances between donors, development agencies, local governance actors and central authorities in testing and replicating M&E tools for local governments. Undoubtedly, an approach that is jointly tested, used in a variety of local contexts and validated by the central government will be better suited for broad, nationwide dissemination and institutionalisation than a small initiative tested in just a few localities by only one actor. Strategic alliances can pay off in terms of time as well, as they allow testing a tool simultaneously in different parts of a country and pooling of financial and human resources of different agencies and institutions.

• There are challenges involved in managing the dynamics of multi-stakeholder processes. As the case study on the performance self-evaluation tool in Mali illustrates, those engaging in multi-stakeholder exercises and strategic alliances when designing and testing M&E tools should be cautioned: the path from design to widespread use may be long and fraught with problems. If an approach is to be participatory and draw on the support of a wide range of representative actors, with different views and opinions, contributions have to be carefully managed. The assistance of an external consultant or resource person experienced in applying such an approach is a valuable, if not necessary, asset.

• M&E results need to be followed up. Most of the approaches and tools described provide participants of the M&E exercises with new insights on the performance and effects of local governance. These include not only information on positive changes and good performance, but also on things that went wrong, were only moderately productive or which need to be changed. Care has to be taken that corrective measures are firmly agreed and followed up. Otherwise, local actors’ interest and incentive to engage in M&E will fade. Besides, all of the stakeholder groups will be challenged to adapt their attitudes and ways of working so as to dissolve any sources of misunderstandings or distrust identified during M&E exercises.

Challenges

M&E tools for use by stakeholders of local governance and decentralisation are bound to face many challenges. For instance, participants might not be used to working together and might therefore lack clarity on their respective roles, rights and duties. The following are – amongst others – areas of challenges highlighted in the case studies drawn upon in this brief.

Dealing with historical experiences and administrative tradition

After a long history of authoritarian rule, citizens of many West African countries are unused to posing questions about governance. In particular, the rural majority of the population still tends to be barely informed of their rights and duties as citizens. They are not aware of or are hesitant to make use of options for holding their elected local representatives accountable. Even those who are informed about their rights, for example, to attend council sessions which are open to the public, may not dare to attend or speak up if not explicitly encouraged to do so. This reality has to be taken into account when jointly designing M&E tools with stakeholders of local governance.

Reducing cultural barriers to constructive criticism

The case studies illustrate that cultural barriers have to be overcome when testing M&E tools. Initially, the participants of (self-) evaluation exercises in Mali were reluctant to voice criticism directly. At public meetings in particular, open criticism was not considered culturally appropriate. The design and use of M&E tools should gradually help actors to deal with such hesitations, anchored as they are in culture and local custom.

This also means that a tool successfully tested in one municipality may not be used in exactly the same way in another. In this regard, the assistance of a facilitator or experienced user can help address cultural barriers and create an atmosphere of trust.

Ensuring that design and utilisation of M&E tools is affordable

All of the methodological approaches and tools presented in this brief have been promoted by development organisations or developed with their active support. Development organisations have provided methodological advice and (co-)financed facilitators, meetings and necessary materials. In many cases, it is difficult to get a fair idea of the cost of the design and utilisation of the proposed M&E tools, including the ‘cost of participation’. From the discussions at the regional seminar it became clear that when designing a tool too little attention has generally been given to the cost of its continued utilisation by the local government and other participating actors. The challenge is therefore to devise methodologies that can help local governments to upgrade their M&E capacities and produce information at a cost commensurate to their financial capacities and the availability of local stakeholders to engage in a joint M&E process.

Achieving sustainability of capacity-building efforts

Sustainability has different dimensions in the context referred to here. One dimension is certainly the abovementioned financial one. Another is more institutional and linked to the complexity of the proposed methodologies and tools. Simple tools that are easily understood and applied by actors with diverse educational and professional backgrounds lend themselves to more sustainable use than complex tools. This is a clear lesson from the stock-taking exercise and related discussions. Another factor to enhance the institutional sustainability of an M&E tool – apart from ownership of the methodology by local actors – is validation and efforts by central authorities to spread the use of a tool throughout a country. This can help to institutionalise an M&E approach that has proven successful and ensure that local governments set aside or receive the resources necessary for continued capacity building and use of the tool.

Avoiding instrumentalisation of local M&E tools and capacity

After an M&E tool has been successfully used at the local level, attention must be paid to making certain that utilisation continues to strengthen the M&E capacities of local governments and contributes to local self-governance. As the case studies show, a participatory process of M&E tool design and testing enables local governments to gradually move towards conducting M&E exercises on their own in a first phase. Then, in a second phase, when local governments and other stakeholders are employing the tool successfully, the central state, donors and development organisations may become interested in standardising the information generated in locally conducted M&E exercises with a view to comparing data from different locations. Yet this can run counter to the objectives of strengthening autonomous local M&E capacity and empowering actors at the local level.


Conclusions and recommendations

The experiences described in this brief, leave no doubt that it is worthwhile to invest in the capacities of different local actors to monitor and evaluate the outcomes of democratic decentralisation processes, local governance and municipal development.

Multi-stakeholder approaches involving a spectrum of local actors – such as local government, civil society, the private sector and de-concentrated departments of the central state – in designing and testing innovative M&E tools, can have a number of positive effects beyond strengthening M&E capacity. These include among others:

• building trust among local stakeholders with different interests, thus reducing resistance to devolution;

• making local governance and service provision more efficient by improving procedures and mobilising citizen initiative and local resources;

• improving information flows between different actors and levels of local government;

• sensitising citizens to their rights and their duty to hold local representatives accountable.

The attentive reader will find a wealth of advice in the case studies. For their part, the authors would like to indicate four recommendations:

• There is a lot of scope for donors and their partners to learn from existing tools for building M&E capacity at the local level. Moreover, those tools often lend themselves to scaling-up and replication in other country contexts. More efforts should be made to document and disseminate these tools, including challenges encountered in the process of testing and utilisation.

• Too often, design of systems to monitor and evaluate decentralisation is led largely by the national level, with insufficient account taken of the information needs of local government and other local-level stakeholders. Donors and national authorities committed to democratic decentralisation should invest more in the capacities of stakeholders of the new local government systems. This would enable them to assess the effectiveness of the new local governance systems, to learn about their respective roles and to analyse the impacts of decentralisation and political reform processes on their lives.

• Efforts to develop M&E capacity in a participatory way with local-level stakeholders of decentralisation processes are necessary and laudable. Nevertheless, it is wise to involve national authorities too in such initiatives, as they can help to institutionalise approaches. Moreover, they can follow up on the many problems emerging from local-level M&E exercises that must be addressed by national-level decisions.

• Development partners can achieve a lot through strategic alliances. Such alliances and coordination of M&E approaches is important to prevent a proliferation of different tools. Too many tools and disparate initiatives could result in confusion among stakeholders who have as yet limited experience in local governance and put the homogeneity of the political system at risk.

References

Bratton, M., M. Coulibaly and F. Machado. 2000. Popular perceptions of good governance in Mali. Afrobarometer Working Papers No. 9 (March).

Cissé, F. in collaboration with S. Diakité and H. Sidibé. 2007. Mali, Public perceptions as a barometer of local governance. Bamako: MATCL, REDL, SNV, ECDPM.

Coulibaly, A.K., R. Diarra Konaré, M.Y. Keita and A. Ag Aboubacrine. 2007. Mali: Participatory monitoring and evaluation as a means of empowering local government in the region of Mopti. Bamako: MATCL, REDL, SNV, ECDPM.

Dery, B. and A. Dorway. 2007. Ghana: District-based poverty profiling, mapping and pro-poor planning as a monitoring and evaluation tool. Bamako: MATCL, REDL, SNV, ECDPM.

Dumont, F. and B. Samaké. 2007. Mali, Geographical information systems (GIS) for the development of rural municipalities. Bamako: MATCL, REDL, SNV, ECDPM.

European Commission. 2005. Draft financial proposal ‘Programme d’appui à la politique réforme administrative et à la décentralisation (PARAD)’ (12 September).

Commission/EuropeAid. 2007. Supporting decentralisation and local governance in third countries. (Tools and Methods Series, Reference Document No. 2). Luxemburg: Office for Official Publications of the European Communities.
Floquet, A., R. Mongbo and S. Woltermann. 2007. Benin: Public control in the education sector, the pilot phase of the participatory local impact monitoring methodology (SILP). Bamako: MATCL, REDL, SNV, ECDPM.

Hilhorst, T. and I. Guijt. 2006. Participatory monitoring and evaluation: A process to support governance and empowerment at the local level. A guidance paper. Amsterdam: Royal Tropical Institute.

Hutchinson, P. and A. La Fond. 2004. Monitoring and evaluation of decentralization reforms in developing country health sectors: Conceptual framework and careful study design will improve decentralisation evaluations, Partners for Health Reform Plus.

Le Bay, S. in collaboration with M. Y. Maïga and O. Tiénou. 2007. Mali: Assessment of local government performance, experiences with a self-evaluation tool. Bamako: MATCL, REDL, SNV, ECDPM.

Le Bay, S. and C. Loquai (ed.). Renforcement de capacités pour le suivi et l'évaluation de la décentralisation et de la gouvernance locale en Afrique Occidentale. Echanges et apprentissages. Bamako: MATCL, REDL, SNV, ECDPM, (forthcoming).

Lodenstein, E., U. Caspari, F. Dumont. 2007. Mali municipalities in figures: Needs and realities. Bamako: MATCL, REDL, SNV, ECDPM.

Loquai, C. and S. Le Bay. 2005. Building capacities for monitoring and evaluating decentralisation and local governance in West Africa. Facilitating capitalisation and learning. Methodological guidance for the preparation of case studies. Bamako: MATCL, REDL, SNV, ECDPM.

Massuangahe, I.J. 2005. Participatory monitoring & evaluation at district and local level: Mechanisms, evidences and practice (MEP). Consolidated review of case studies, best practices and regional experiences launched on GovNet, PovNet, Evalnet and Net. Maputo: UNDP Mozambique.

MATCL/DNCT, SNV, Helvetas, PACT/GTZ-DED 2004. Outil d’auto évaluation des performances des collectivités territoriales. Bamako.

Reyes, C.M. and L.E. Valencia. 2004. Poverty reduction, decentralization and community-based monitoring systems. In: The evolving roles of CBMS amidst changing environments. Proceedings of the 2004 CBMS Network Meeting, June 16-24, 2004, Senegal and Burkina Faso edited by CBMS Network Coordinating Team.

Sébahara, P. 2004. Monitoring and evaluation of support for decentralisation and local governance: A case study on Burkina Faso. (Inbrief No. 7). Maastricht: ECDPM

Sène, G. and Z. Ouédraogo. 2007. Niger: Planning and M&E in municipalities, focusing on poverty reduction. Bamako: MATCL, REDL, SNV, ECDPM.

Sere, K. and E. Champagne. 2006. Réformes de la décentralisation au Burkina Faso, à Madagascar et au Rwanda : Vers de nouveaux instruments de gestion par la performance. Proposition de présentation, Senior Policy Workshop on Local Governance and Pro-Poor Outcomes in Africa, Kigali, 31 October to 2 November 2006. Mimeo.

Simon, K. 2004. Theoretische Gründe für partizipative und Selbst-evaluierung von Verwaltungszusammenarbeit: Demokratietheorie, Policy-Analyse und Institutionenökonomie. In: Verwaltungen, Experten und Bürger im Reformprozess. Wirkungen und Evaluierung von Verwaltungszusammenarbeit mit Entwicklungsländern edited by K. Simon. Baden-Baden: Nomos Verlagsgesellschaft (pp. 87-105).

Steinich, M. 2000. Monitoring and evaluating support to decentralisation: Challenges and dilemmas. (Discussion Paper No. 19). Maastricht: ECDPM.

Sylla, D. and H. Ongoïba. 2007. Mali: Evaluating the impact of decentralisation. Bamako: MATCL, REDL, SNV, ECDPM.

Tamini, J., I. Sylla, M. Ischer and C. Asanga. 2007. Cameroon: Strategic planning and monitoring of municipal development. Bamako: MATCL, REDL, SNV, ECDPM.

Toonen, J., D. Dao and T. Hilhorst. 2007. Mali: Towards a basic health-sector information system for municipal actors (SIEC-S). Bamako: MATCL, REDL, SNV, ECDPM.

Watson, D. 2006. Monitoring and Evaluation of Capacity and Capacity Development. A case study prepared for the project ‘Capacity, Change and Performance’. (Discussion Paper No. 58b). Maastricht: ECDPM.


Notes

1 For more details see Lodenstein et al. (2007), Sène and Ouédraogo (2007) and Tamini et al. (2007).

2 The guide is MATCL/DNCT et al. (2004).

3 SIEC-S stands for ‘Système d’Information Essentielle pour la Commune dans le secteur de Santé’, i.e. ‘System of Basic Health-Sector Information for Municipalities’.

4 See, in particular, the current work of the Afrobarometer (Bratton et al. 2000) and the Impact Alliance’s Local Governance Barometer project, which is intended to be used as a tool for measuring the performance of local government throughout Africa (www.impactalliance.org).


Acknowledgements

The authors thank all those who contributed to the joint process of stock-taking, analysis and exchange of experiences described in this document. In particular we thank the authors and members of the teams that prepared the case studies.We also express our gratitude to Ibrahima Sylla, Decentralisation Advisor at the Malian Ministry of Territorial Administration and Local Government (MATCL) for his indefatigable commitment to the process.We also thank Jan de Witte (SNV) and Jan Vanheukelom (ECDPM) for their continued support during the publication and dissemination process of the case studies and this paper. Last but not least, we thank those organisations that generously supported the joint stock-taking exercise and production of this InBrief.

Readers’comments and suggestions are most welcome. Please address them to Christiane Loquai (email: cl@ecdpm.org) or Sonia Le Bay (e-mail: slebay@snvworld.org).To order a printed copy of this publication in English or French, please contact ECDPM’s Information Officer, Jacquie Dias (e-mail: jd@ecdpm.org).