Monitoring Water Resources Governance Progress Globally: Experiences from Monitoring SDG Indicator 6.5.1 on Integrated Water Resources Management Implementation

Abstract

Improved water resources governance supports important social, economic, and environmental objectives. The 2030 Agenda recognizes improved water governance to be critical for achievement of the Sustainable Development Goals (SDGs) and commits to monitor the progress of implementation of integrated water resources management (IWRM). This paper critically reviews the approach to monitoring SDG indicator 6.5.1 on implementation of IWRM. Firstly, the paper places the indicator monitoring within the context of other initiatives to measure water governance. Secondly, it analyzes experiences of application of the SDG indicator 6.5.1 methodology to evaluate the strengths and weaknesses of the indicator and presents the key findings of the 2017/2018 global baseline assessment of IWRM implementation. Baseline reporting shows that degree of IWRM implementation globally is 49%, though country scores range from 10 to 100%. Disaggregating the data by country and by aspect of water resources governance provides a diagnostic tool to identify areas of high and low progress, and, therefore, where increased resources and attention are required. The article concludes by suggesting how the next iteration of SDG indicator 6.5.1 monitoring cycle can be made into a tool for advancing the IWRM implementation and improved governance practices on the ground. It also proposes how the methodology can be strengthened to address current limitations, including aspects relating to integrity, accountability and transparency.

1. Introduction

1.1. Water Resources Governance for Sustainable Development

Water is central to all human activity and ecosystem health. All productive sectors of society—including agriculture, energy, transport, tourism, and industry—depend on continued and timely supply of water resources and vital water ecosystem services [1,2]. Decisions on how to allocate and use water resources across these sectors and other users are fundamental to sustainable development and human well-being [3]. Growing pressures on water, including population growth, environmental degradation and climate change, make the task of water management increasingly complex and the need for improved water governance more urgent [4,5]. Balancing the uses of (often limited) water resources amongst the many competing users in an efficient, sustainable, and equitable manner requires appropriate institutional and regulatory frameworks to be in place. These are key for improved water governance.

Integrated Water Resources Management (IWRM) is an approach for creating such water management and development frameworks [6]. One of the clearest commitments to further IWRM globally is the dedicated IWRM target of the Sustainable Development Goal 6 (SDG 6) on water and sanitation. SDG 6 target 6.5 calls upon countries to “By 2030, implement integrated water resources management at all levels, including through transboundary cooperation as appropriate” [7]. The achievement of this target is monitored and reported through two dedicated indicators—SDG indicator 6.5.1 on the degree of implementation of IWRM, and indicator 6.5.2 on the proportion of transboundary basin area with an operational arrangement for water cooperation. This paper will focus on the use of the SDG indicator 6.5.1, as a measure of IWRM implementation progress in countries and an indicator to measure water governance more broadly.

1.2. Defining Water Governance

As the Organisation for Economic Co-operation and Development (OECD) notes, the ‘water crisis’ has often proven to be a crisis of governance [8], where water scarcity is largely caused by mismanagement of available resources. The need for effective water governance arrangements has been recognized, both in literature [4,9,10], and in the international development agenda [4,11,12]. There are many definitions of water governance, but for the purposes of this paper, water governance is defined as “the political, social, economic and administrative systems that are in place to develop and manage water resources and deliver water services to different levels of the society” [9]. Concisely, water governance encompasses a set of rules, practices and norms that determine who gets water, when and how [13]. This includes the systems in which water-related decisions are taken and the processes by which stakeholders engage within those systems.

A large volume of work has gone into defining water governance [9,10]. To a large extent, such efforts have been informed by the broader discourse on principles of ‘good governance’ and how these principles can be integrated in the water management sphere [4,9,13,14]. Recent efforts include the OECD’s 12 Principles on Water Governance, covering three dimensions of governance—(1) Effectiveness, (2) Efficiency and (3) Trust and engagement [15] (the Principles were developed by the multi-stakeholder network OECD Water Governance Initiative). Adopted in 2015, the framework covers key principles related to capacity, financing, stakeholder engagement, monitoring and evaluation, appropriate scales of management, and data and information, amongst others (Figure 1). The Principles acknowledge among other things that governance is contextual and there is no one-size-fits-all solution. Water governance is a means to an end to resolve problems and hence “forms” of water governance should follow “functions” of water governance.

In similar efforts, the UNICEF and UNDP SIWI Water Governance Facility (UNICEF and the United Nations Development Programme Water Governance Facility at SIWI) defines a conceptual framework for governance of water and sanitation (WASH sector) by outlining key components of the enabling environment. The enabling environment within water resources management is commonly understood as covering the policy, planning and legal frameworks surrounding the development and management of water resources [16]. The framework identifies structural factors, institutional factors outside the WASH sector, and the key water governance functions to foster better understanding of the governance within the sector [13]. In addition, the UNDP Oslo Governance Centre and UNDP-SIWI Water Governance Facility (2013) proposes three key components of a water governance assessment, including (a) Actors and institutions; (b) Governance principles; and (c) Performance [17].

A key aspect of the frameworks mentioned above is the inclusive assessment approach that involves relevant stakeholders. It is commonly agreed that well-designed governance assessments should create a platform for stakeholders to engage with and contribute to the debate with their local, contextual knowledge, and thereby be able to drive or instigate necessary amendments to the policy and regulatory environments [18].

Despite the efforts to define universal frameworks and principles to assess water governance, the individual contexts of resource availability, water challenges, cultural, socio-economic, political and regulatory settings require tailor-made governance arrangements for each specific geography. Thus, a single universal blueprint for good water governance design is most likely not an attainable reality. As Woodhouse and Muller (2016) [10] conclude in a recent review, water governance is highly contextual. Water governance systems must, therefore, be designed with their specific functions in mind, rather than being based on normative approaches [10]. However, it may also be argued that fundamental guiding principles and frameworks can help embed key good governance principles in the tailor-made and context-specific systems and be useful to practitioners.

1.3. Measuring Water Governance

Many of the frameworks and principles devised for effective water governance assessments seek to understand the elements of the enabling environment and define the overall qualities of the policies and regulatory frameworks that can foster sustainable, efficient and inclusive water management. Beyond an understanding of the key elements, measuring the performance and progress of water governance is a necessary step to keep track of whether the existing governance systems deliver their intended benefits, and to inform the need for, and effectiveness of, implemented improvements and reforms. Given the unique governance structures, institutional design, formats and cultures of specific geographies, defining governance indicators that can be applied globally, while still maintaining significance for local conditions, has proven to be particularly challenging.

Widely used governance assessments, looking at governance more broadly, include World Bank Worldwide Governance Indicators looking at six dimensions of governance [19] and Transparency International’s Corruption Perceptions Index [20]. Within water domain, various approaches to assessing and monitoring water governance have emerged over the last decades. Some take point of departure in broader good governance concepts, such as accountability and transparency [21], while others have focused on the enabling environment [16] and performance of relevant institutions [22,23]. The OECD recently developed a set of OECD Water Governance Indicators [24]. The Water Governance Indicator Framework is composed of 36 water governance indicators a checklist containing 100+ questions on water governance and it is complemented by an Action Plan. Indicators apply at different scales (city, basin, national or other) and for different water management functions (water resources, water services, water disasters).

It is generally acknowledged that condensing information to specific indicators can create powerful decision-making and progress evaluation tools [25]. That said, as De Stefano (2010) notes in a review of water policy assessment indicators, it has been especially challenging to identify meaningful numerical indicators that can be used to assess policy and management dimensions, as these may take years to take measurable effect, and they need to bear global relevance [26].

Limitations of simplifying the complexity of governance interactions to a single indicator or index must be acknowledged. For example, many governance indicators rely, to some extent, on subjective perceptions of the interviewees or experts [20]. There may also be de facto differences between the governance arrangements in place formally, and the ways in which these are followed in practice [27]. OECD further identifies data availability, data collection through expert views, data comparability over space, and time, and the difficulty in drawing causality linkages between policies and their outcomes as key difficulties when working with governance indicators [24]. Nevertheless, establishing mechanisms and related indicators for governance assessment is important to track whether countries are on the right track to more sustainable water governance instruments and achievement of sustainable development objectives [17].

The global ambition to track progress on water and sanitation, including governance aspects, was confirmed through the adoption of the dedicated SDG 6 on water and sanitation [11]. The specific indicators were agreed upon in lengthy consultations between the IAEG-SDGs (The Inter-Agency and Expert Group on Sustainable Development Goal Indicators (IAEG-SDGs)), in consultation with United Nations (UN) Agencies, civil society, academia and UN member countries [28]. Building on the Millennium Development Goal sanitation and drinking water indicators, the ambition under the 2030 Agenda was to cover the whole water cycle under a dedicated water goal (SDG 6), acknowledging the central role of water in sustainable development [29]. Further, responding to the stakeholder consultations that marked strengthened institutions and better governance as key to achieving better water management [30], a dedicated target 6.5 on water resources management was included under SDG 6, with IWRM as the central element.

The IWRM had been a central element of water resources management and governance reforms in many countries since the 1990s with many countries making progress on implementation [31]. Despite the maintained focus, implementation remained slow and complex [16], and much work remained to be done to adopt IWRM approaches, especially at the lower levels of water development and management. The selection of IWRM as a dedicated target 6.5 and indicator 6.5.1 (the degree of IWRM implementation) within the SDG 6, was a result of both, the continued international commitments to furthering IWRM implementation, and the established track record of IWRM measurement from previous global surveys in 2007 and 2011 using a survey questionnaire.

1.4. The Evolution of IWRM at International Level

Since the 1990s, the integrated water resources management (IWRM) approach has become a globally accepted framework for sustainable water resources management. A widely used definition of IWRM is that of the Global Water Partnership (GWP), that defines IWRM as “... a process which promotes the coordinated development and management of water, land and related resources in order to maximise economic and social welfare in an equitable manner without compromising the sustainability of vital ecosystems and the environment” [32]. GWP further describes IWRM as management of water resources “... in a sustainable and balanced way, taking account of social, economic and environmental interests. It recognises the many different and competing interest groups, the sectors that use and abuse water, and the needs of the environment” [6]. A key pillar of IWRM is the management of resources at a basin level—thus based on hydrographic boundaries rather than traditional administrative divisions.

The IWRM approach has been adopted widely over the last three decades as a means to improve effective water governance and management. The foundations for IWRM were laid in the 1977 Mar del Plata conference calling for programmes for ‘integrated management of the resource’ and integrated planning of water management as part of the resulting action plan [5]. The Dublin Principles on Water and Environment adopted in the 1992 International Conference on Water and the Environment outlined the need for water to be governed in a participatory manner and at the lowest appropriate level, involving all stakeholders, calling for a holistic management of the resources [33]. The Dublin principles were presented at the influential Rio Earth Summit [34] where the recommendations from the Dublin conference were subsequently included in the Agenda 21 [16], which could be considered one of the most significant turning points for the IWRM agenda globally.

Since then many countries have experienced benefits from the integrated management approaches [31]. The experiences from practical implementation have also shifted to a better understanding of IWRM as a valuable process rather than an end goal [35]. Further recognition of the significance of IWRM in global environmental and sustainable development agenda (Figure 2), has been reaffirmed recently with the inclusion of a dedicated IWRM target under the 2030 Agenda [7].

Despite gaining general recognition, IWRM has been subject to numerous criticisms over the last decades, such as the lack of clearly defined path for implementation [37], lack of focus on livelihoods and the poor [38], and the neglect of historical and political dynamics to manage resources management at a basin level [39]. It is particularly the lack of clarity and specificity of the concept that has often been the target of criticisms [37]. Some authors have noted that it is the ambiguity of the term itself that could be seen as one of the fundamental barriers to wider implementation [40].

While not without its challenges, and critics, the need for furthering IWRM approaches has not lost its relevance globally, at the highest levels [41]. IWRM creates the framework under which water management operates, and thus underpins the success of achievement of all targets under SDG 6, including safe drinking water delivery, sanitation, water quality, efficient water use and healthy water ecosystems. Thus, target 6.5 is fundamental to enabling progress on all targets under SDG 6, and beyond. The global synthesis report on SDG 6 [42] outlines linkages of SDG 6 to other goals and targets under the 2030 Agenda, highlighting the vital role that water holds in achieving social, economic and environmental sustainability, and number of goals under the SDG framework.

Most importantly, the dedicated indicators provide an important tool for countries to better understand and analyse their respective challenges of IWRM implementation, and inform needs for changes, priorities and reforms within water resources management and governance.

IWRM implementation takes place in various governance contexts that can enable or hinder IWRM. However, the SDG indicator 6.5.1 allows countries not only to assess the implementation of IWRM, but also allows them to assess some of the key aspects of water governance. The focus of IWRM is often to improve the enabling environment and management instruments for sustainable water management, therefore IWRM implementation and assessment process can be seen as a means to operationalizing important aspects of water governance, though it is often challenging to put the theory into practice [43].

2. Methodology

2.1. Measuring IWRM—History Leading to the Current Approach

Successful management of water resources is a long-term, complex and continuous process. It requires the input and interaction of governments, agencies and organizations at an international, national, regional, and local level, including the private sector, civil society, and dedicated individuals. In turn, measuring the impacts of such efforts is not a straight forward task, especially when it comes to identifying clear and meaningful indicators of IWRM progress. However, with the acknowledgment of IWRM approaches as central to achieving global SDG 6, and the 2030 Agenda, a meaningful indicator for tracking the progress on IWRM implementation is necessary.

The 193 Member States of the UN adopted the 2030 Agenda in September 2015, by adopting 17 Sustainable Development Goals and 169 associated targets shaping the agenda for global sustainable development over the next 15 years [12]. The resolution also established that indicators were to be developed by the IAEG-SDGs to monitor all targets. Indicator methodologies were developed over the following 2 years, with inputs from UN member countries, UN Agencies, and wider stakeholder consultations. The final indicator framework was adopted on 6th of July 2017, including a dedicated indicator 6.5.1 “Degree of integrated water resources management implementation” [11].

Previous efforts to measure the progress of implementation of IWRM with a global scope include assessments undertaken in 2007 [44] and 2011 [16]. Both assessments made use of a survey-based approach, measuring the national status of IWRM implementation by using a ladder of ‘degree’ of implementation. The development process of the SDG indicator 6.5.1 on the degree of IWRM implementation methodology built on these previous experiences, given the positive track record, and high response rate from countries. The 2007 survey received responses from 104 countries, while the global survey undertaken in 2011 received responses from 134 countries [16]. The high, and growing, response rates are considered to reflect not only the intent of countries to report on their global commitments, but also a signifier of their interest in assessing own progress on the implementation of IWRM and identification of areas where implementation lags behind.

2.2. SDG Indicator on the Degree of IWRM Implementation Methodology

At its core, IWRM provides a framework to ensure that water resources are developed, managed and used in an equitable, sustainable, and efficient manner. The SDG indicator on the degree of IWRM implementation (indicator 6.5.1) seeks to address these dimensions through the survey sections and individual questions within each section.

The SDG indicator 6.5.1 is measured on a scale of zero to 100 (representing the degree (%) of IWRM implementation), based on 33 questions in a country self-assessment questionnaire. Each question is scored in increments of 10 (from 10 to 100) with 6 defining thresholds: Very low (0), Low (20), Medium-low (40), Medium-high (60), High (80) and Very high (100). Each question has a unique description of the interpretation of the various thresholds to guide respondents (See Figure 3 for example from survey question 1.1a on the status of implementation of national-level water resources policy) [36].

The individual threshold descriptions were devised based on the ‘degree of implementation’ descriptions and country experiences from previous global surveys. The threshold descriptions were further revised based on the technical inputs from the indicator working group, the IAEG-SDGs and member state comments. Importantly, the methodology was piloted in 5 countries (Jordan, the Netherlands, Peru, Senegal, and Uganda), focusing on testing of the methodology and receiving country feedback on its technical feasibility, usefulness for policy making, institutional models for implementation, and capacity requirements. Piloting exercise also focused on ensuring that all individual question threshold descriptions maintain a general consistency of status of implementation that can be comparable in aggregating the final score. Consistency and clarity of these thresholds was part of the pilot country feedback [45].

Countries were also able to score the individual questions ‘in between’ two thresholds, where the country-specific circumstances were more representative of an intermediate stage between two of the thresholds, by using the scores that fall ‘between’ two predefined thresholds (i.e., 10, 30, 50 etc.).

Survey questions cover four main dimensions of IWRM:

• Enabling environment: The conditions that help to support the implementation of IWRM, which includes policy, legal and strategic planning tools;
• Institutions and participation: The range and roles of political, social, economic and administrative institutions and other stakeholder groups that help to support implementation;
• Management instruments: The tools and activities that enable decision makers and users to make rational and informed choices between alternative actions; and
• Financing: The budgeting and financing made available and used for water resources development and management from various sources.

Each of the main survey sections is further divided in two subsections: representing ‘National’ and ‘Other’ levels. The ‘other’ levels cover implementation at subnational, basin/aquifer, local and transboundary levels, and addresses the target 6.5 formulation of implementing IWRM “at all levels” (full questionnaire including survey questions and their descriptive thresholds provided in Supplementary Materials).

Each of the individual survey questions is scored on a scale of zero to 100, in increments of 10. An average score is then calculated for each of the four survey sections (rounded to the nearest whole number). The final SDG indicator 6.5.1 score is then calculated as the average of the four survey section scores, representing a scale of zero to 100.

A total of 193 countries were approached during data collection process over 2017 and 2018, covering all UN member states. The data collection process consisted of three general phases: identification of the national focal point (FP) (I), rollout of survey documents to confirmed national FP (II) and data collection and validation (III).

A crucial part of the indicator calculation methodology was to ensure that all responses represent the national status of implementation. National responses are therefore submitted by officially confirmed national focal points submitting the response on behalf of the country (one official response per country representing the national status of IWRM implementation). In the preparatory phase of data collection, much effort was put into identifying the institution and person responsible for coordinating and delivering the national response. About 75 per cent of the FPs are affiliated with national ministries responsible for water management (e.g., ministry of water, ministry of environment, or similar). Other institutions include National Statistical Offices or state water agencies, or similar.

The national focal points were responsible for data collection in countries, gathering stakeholder inputs, filling out the survey document, internal quality assurance and response submission to UN Environment (the Custodian Agency of the indicator). Acknowledging the particularly central role of the focal points in assessment delivery, additional efforts were made to provide the necessary training on survey interpretation and data collection processes through online training sessions in the preparatory phases of the data collection. A total of eight training sessions were held, covering different time zones, and the 6 official UN languages—Arabic, Chinese, English, French, Russian and Spanish. Further supportive workshops and webinars were also held as part of the integrated monitoring of the SDG 6, with participation of the respective UN agencies and country representatives [46,47]. All training materials and session recordings were made available on the SDG indicator 6.5.1 website [48] for permanent record and distribution. In addition to focal points, any interested parties—e.g., colleagues that were expected to support or contribute to data collection—were also invited to participate. A total of 67 participants took part in the live training sessions, covering 43 countries.

All countries were encouraged to undertake stakeholder consultations during data collection to ensure that the national response best reflected the views of various sectors and users (such as energy utilities, agricultural users, NGOs, public utilities, municipalities), and thus represents a consensus on the status of implementation of the various elements of IWRM. More than 35 countries organized structured stakeholder workshops to discuss the national assessment (most with support and facilitation from UN Environment and the Global Water Partnership (GWP)), representing about 20% of total number of countries reporting. The workshops were designed to encourage consensus on the country score by key stakeholders, which typically included representatives from various ministries related to water (e.g., environment, agriculture, natural resources, water resources ministries), state and private authorities in the domain of water or water and energy, NGOs and research organisations.

All results were submitted to the UN Environment by the national FP in a final submission, and quality assured by the indicator 6.5.1 Helpdesk team, and often revised in dialogue with the FP (e.g., corrections to calculations, adding missing scores, providing narrative explanations behind the scoring—particularly for N/A responses or questions scored as zero or 100, representing the two extremes of the scoring range).

3. Results

Global Baseline Results on Implementation of IWRM 2017/2018: Key Findings

From 193 countries contacted, a confirmation on FP was received from 185 countries. No national focal points were confirmed for 8 countries either due to missing response altogether, or inability of countries to identify the exact FP for the indicator. From 185 countries with confirmed national FP, 177 countries made survey data submission in due time. The remaining countries were either unable to collect data or unable to submit before deadline. The reasons for lack of submissions are further detailed in the global status report on degree of IWRM implementation [36].

Out of 177 timely submissions, 5 submissions were not able to pass data validation due to incompleteness or incorrectly filled questionnaires. Thus, a total of 172 country submissions form the global baseline for SDG indicator 6.5.1, covering more than 80 per cent of the countries in most regions and each Human Development Index group, around 75 per cent of total global population, and around 80 per cent of country area. Notable exceptions in terms of population and/or area include India, Thailand, Canada and the United States.

The key objective of the global data drive was establishing a global baseline of IWRM implementation (through indicator 6.5.1), based on the country reported indicator scores of 0–100%, and to analyse the baseline in respect to the global aspirational target of a very high degree of IWRM implementation (91–100%). Further, the analysis also focussed on the individual survey section and question responses, in order to identify those aspects of IWRM implementation that lag behind. The analysis also seeks to understand the status implementation of IWRM across all levels of implementation—including national, subnational and transboundary—addressing the global target wording.

Global baseline data show that country implementation of IWRM ranges from very low to very high, with a global average score of 49, corresponding to medium-low. The general interpretations of the overall 6.5.1 indicator score, can be seen in Figure 4 [36]. The aggregated global score interpretations are consistent with the general interpretation of the thresholds used in the question scoring for each respective IWRM implementation category (see Methodology section).

Analysis of the national 6.5.1 scores shows that roughly 40 per cent of countries are implementing most elements of IWRM through long-term programmes (medium-high category and above). Approximately the same number (41%) of countries have adopted most elements of IWRM and implementation is underway, but uptake of arrangements and stakeholder engagement may be relatively low (medium-low category). The remaining 19% of countries with low and very low degree of implementation have only started developing elements of IWRM. Geographically, there is a wide spread of scores within each region. On average, Latin America and the Caribbean, Central and Southern Asia, Oceania, and Sub-Saharan Africa have the lowest average implementation levels (Regional distribution in Figure 5 [36]).

Results show that subnational, basin and transboundary implementation lags national implementation in nearly every comparable instance (Figure 6). Figure covers following question pairs, representing national vs. other level scores for comparable questions: Gender objectives Q2.1e vs. Q2.2c; stakeholder participation Q2.1c vs. Q2.2b; budget for investments Q3.1a vs. Q3.2a; basin management plans Q1.1c vs. Q2.1b; subnational policy Q1.1a vs. Q2.1a; basin/aquifer organisations Q2.1a vs. 2.2a (see all questions in Supplementary Materials). These differences raise some concerns on whether implementation of national-level measures is being operationalized on local levels where the majority of operational management takes place, and where the national level policy is executed. Approximately three-quarters of countries with transboundary basins (rivers or aquifers shared between two or more countries) report that they have established some form of agreements, organizational frameworks, data sharing and financial arrangements for transboundary water management. However, the degree of implementation, or operationalization, of these aspects varies greatly, even for neighbouring countries. Similar tendencies are observed in the national reporting of SDG indicator 6.5.2, measuring the proportion of transboundary basin area in each country with an operational arrangement for water cooperation [49].

Global averages for the four survey sections show that implementation scores are similar for policies, laws and plans (51), institutions and participation (53), and management instruments (51). Financing lags the other dimensions by about 10 points (41). It should be noted that these are global averages and at a country level the differences represent a wide spread of scores. On the individual question level, the lowest average global scores were calculated for implementation of gender-related objectives on transboundary level (32) and subnational budgets for IWRM investment (35). Highest scores globally were assigned to implementation of national institutional capacity and cross-sectoral coordination (58 and 63, respectively).

As the data collected represents the SDG indicator 6.5.1 baseline, it is not possible to state whether countries are “on track” to meet the global target until the next reporting cycle is completed. However, findings from previous global surveys in 2007 and 2011, and experiences on the ground, indicate that implementation of IWRM may take several decades. The baseline analysis estimates that about 60% of countries are unlikely to meet the global target unless progress on IWRM implementation is significantly accelerated.

The global status report provides further detailed analysis of the baseline data, including case studies and country experiences derived from free text replies submitted together with the individual question scores [36].

4. Discussion

4.1. SDG Indicator 6.5.1—Global Approach for Measuring Implementation of IWRM and Water Governance at Multiple Levels

The SDG indicator on degree of IWRM implementation (6.5.1) baseline assessment of 2017/2018 has contributed significantly to the further refinement and improvement of a globally applicable IWRM assessment methodology. Together with the 2007 and 2011 surveys, it provides a solid track record of a survey-based approach to IWRM status assessment, and, even with its methodological challenges, the approach has proven to be able to deliver measurable and beneficial results on the country progress across key dimensions of IWRM.

A number of amendments were made to the SDG indicator 6.5.1 methodology when compared to previous methodologies, in order to respond to some of the challenges encountered during the previous surveys. These included the subjectivity in interpretation of the various thresholds of the ‘degree of implementation’, an aspect which has experienced a major improvement through addition of narrative explanations for each threshold. The SDG indicator 6.5.1 baseline survey differs from previous global assessments in that it requires an aggregation of a final indicator score, for the purposes of country reporting to the United Nations Statistics Division (UNSD). This had not been the case previously, where conscious efforts were made to avoid an aggregated score. Thus, the SDG indicator 6.5.1 provides a single score of 0–100% of implementation of IWRM for each country, providing the basis for the global baseline on implementation of IWRM approaches, while the individual question results provide relevant information for national policy-makers.

A strength of the SDG indicator 6.5.1 approach is also its focus on the various levels of implementation—national, but also the subnational (local administrative and river/aquifer basin) level arrangements. As survey results have shown, there is a clear IWRM implementation challenge. The implementation of IWRM approaches on national level often do not reach the lower levels of implementation, which is where much of the ground-work for water management and resource development takes place.

The last decades have seen a push for countries to establish an enabling environment for IWRM [50], and global surveys show that majority of countries have implemented some degree of IWRM plans, strategies, and IWRM-based policies and management instruments [16,36]. From an indicator perspective, such aspects often focus on so-called input and process indicators [17]. Input indicators, in this context, cover the existence of legal and regulatory frameworks, while process indicators address the actions in place to reach certain objectives. The SDG indicator 6.5.1 survey through the individual questions covers what could be argued to be input and process indicators, but similarly falls short in covering the impact indicators. Within SDG 6, targets 6.1 through 6.4 and target 6.6 contain indicators that cover the ‘impact’ aspects of resource management, such as water quality, ecosystems, clean water and sanitation.

Similarly, broader water governance assessments account for the existence of necessary regulatory frameworks that could enable and foster more sustainable, equitable and inclusive water governance [15]. However, operationalizing these provisions requires actionable management instruments at appropriate levels—subnational, basin and transboundary. These include management arrangements that can ensure sustainable financing for resource management, concrete forums and channels for stakeholder engagement, and protocols for data monitoring and exchange, amongst other elements. The IWRM indicator can provide insight not only in the existing provisions for an enabling environment, but also on the instruments that allow to operationalize improved water governance at all levels of management, and thus can be utilized as a measure of water governance. The survey questions to a large extent address all three governance assessment areas outlined by the OECD governance principles framework [15]. The individual principles, though not all, are addressed by survey questions focusing on stakeholder participation, cross-sectoral coordination, basin-level management, financing, capacity, data and information sharing, amongst others.

Table 1. SDG indicator 6.5.1 survey questions supporting measurement of OECD governance principles.

OECD Principle on Water Governance	SDG Indicator 6.5.1 Questionnaire Question (s) *
EFFECTIVENESS of water governance
Principle 1. Clearly allocate and distinguish roles and responsibilities for water policymaking, policy implementation, operational management and regulation, and foster co-ordination across these responsible authorities.	1.1b National water resources law(s) 2.1a National government authorities’ capacity 2.1b Coordination between national government authorities representing different sectors
Principle 2. Manage water at the appropriate scale(s) within integrated basin governance systems to reflect local conditions, and foster co-ordination between the different scales.	1.1a National water resources policy, or similar, based on IWRM 1.1c National IWRM plans 1.2c Arrangements for transboundary water mgt. in most important basins/aquifers 2.2a Basin/aquifer level orgs. for leading implementation of IWRM 2.2e Organizational framework for transboundary water mgt. for most important basins/aquifers 3.2c Data and information sharing within countries at all levels
Principle 3. Encourage policy coherence through effective cross-sectoral co-ordination, especially between policies for water and the environment, health, energy, agriculture, industry, spatial planning and land use.	2.1b Coordination between government authorities representing different sectors on water resources, policy, planning and mgt. 3.1a National monitoring of water availability 3.1b Sustainable and efficient water use mgt. 3.1d Mgt. of water-related ecosystems 3.1e Mgt. instruments to reduce impacts of water-related disasters
Principle 4. Adapt the level of capacity of responsible authorities to the complexity of water challenges to be met, and to the set of competencies required to carry out their duties.	2.1a National government authorities’ capacity for leading implementation of national IWRM plans or similar 2.1f Developing IWRM capacity at the national level
EFFICIENCY of water governance
Principle 5. Produce, update, and share timely, consistent, comparable and policy-relevant water and water-related data and information, and use it to guide, assess and improve water policy.	2.1b Coordination between national government authorities representing different sectors on water resources, policy, planning and management. 3.1a National monitoring of water availability 3.2c Data and information sharing within countries at all levels 3.2d Transboundary data and information sharing between countries
Principle 6. Ensure that governance arrangements help mobilise water finance and allocate financial resources in an efficient, transparent and timely manner.	4.1a National budget for investment including water resources infrastructure 4.1b National budget for the recurrent costs of the IWRM elements 4.2a Sub-national or basin budgets for investment including water resources infrastructure 4.2b Revenues raised from dedicated levies on water users at basin, aquifer or subnational levels 4.2c Financing for transboundary cooperation
Principle 7. Ensure that sound water management regulatory frameworks are effectively implemented and enforced in pursuit of the public interest.	1.1b National water resources law(s) 2.1a National government authorities’ capacity for leading implementation of national IWRM plans or similar
Principle 8. Promote the adoption and implementation of innovative water governance practices across responsible authorities, levels of government and relevant stakeholders.	2.1c Public participation in water resources, policy, planning and mgt. at national level 2.2b Public participation in water resources, policy, planning and mgt. at the local level 3.2c Data and information sharing within countries at all levels
TRUST and ENGAGEMENT in water governance
Principle 9. Mainstream integrity and transparency practices across water policies, water institutions and water governance frameworks for greater accountability and trust in decision-making.	2.1c Public participation in water resources, policy, planning and mgt. (national) 2.1d Business participation in water resources development, mgt. and use (national) 2.2b Public participation in water resources, policy, planning and mgt. (local level)
Principle 10. Promote stakeholder engagement for informed and outcome-oriented contributions to water policy design and implementation.	2.1c Public participation in water resources, policy, planning and mgt. (national) 2.1d Business participation in water resources development, mgt. and use (national) 2.1e Gender-specific objectives for water resources mgt. (national) 2.2c Gender-specific objectives at sub-national levels
Principle 11. Encourage water governance frameworks that help manage trade-offs across water users, rural and urban areas, and generations.	2.1c. Public participation in water resources, policy, planning and mgt. (national) 2.1e Gender-specific objectives for water resources mgt. (national) 2.2b Public participation in water resources, policy, planning and mgt. (local) 2.2c. Gender-specific objectives at subnational levels
Principle 12. Promote regular monitoring and evaluation of water policy and governance where appropriate, share the results with the public and make adjustments when needed.	2.1f Developing IWRM capacity at the national level

* ’mgt.’ stands for ‘management’ in all instances.

provides an overview of the correspondence between 6.5.1 survey questions and OECD governance principles. Cross-analysis shows that SDG 6.5.1 survey questionnaire questions address all 12 OECD principles of water governance. Survey questions are able to address aspects of the principles to varying degrees (as per the specific contents of each principle [4]), but it is noted that the process of SDG indicator 6.5.1 data collection, and thus the monitoring exercise in itself, contributes to implementation of all 12 Principles.

Principles 11 and 12 are only partially covered by the survey questions. Notable areas that the SDG indicator 6.5.1 questionnaire does not address explicitly are transparency accountability and integrity (Principle 9), as well as promotion of innovative water governance (Principle 8). The assessment relating to stakeholder involvement addresses these aspects to a certain extent. The stakeholder consultations and status assessment exercise in itself directly contribute to furthering transparency and accountability. However, it could be argued that for the indicator to be a means of governance measure, these should be addressed in the questionnaire more explicitly. Given the nature of the FP affiliations, it may be challenging to implement questions relating to transparency and integrity as part of the self-assessment with a specific score. However, questions relating to specific measures taken to promote transparency, integrity and accountability could be included in the survey as open-ended response fields to foster reflection and stakeholder discussions on these topics.

As with other governance indicators and assessment approaches, the challenge lies in linking the implementation of various enabling environment and regulatory aspects to clear outcomes on the ground. Ultimately, high-level good governance measures are implemented in view of achieving tangible impacts on the ground, in the form of more equitable and efficient allocation of water resources, improved water quality, community wellbeing and ecosystem health. It could be argued that it is through the other indicators under SDG 6, that many of the implementation impacts of IWRM on water resources management can be truly assessed and monitored. These include indicators for water stress, water-use efficiency and freshwater ecosystem health.

There is also considerable complementarity between indicator 6.5.1 and indicator 6.5.2, which measures the proportion of transboundary basin area (rivers, lakes and aquifers) within a country covered by ‘operational arrangements for water cooperation’. Indicator 6.5.2 utilises four criteria for determining whether an arrangement is operational, namely whether a joint body or mechanism exists, whether meetings between States take place at least annually, whether data and information is exchange at least annually, and finally whether joint management plans or objectives have been set [49]. These criteria therefore supplement the 6.5.1 questions relating to whether transboundary arrangements are in place (Q1.2c), whether organisational frameworks have been established (Q2.2e), whether data and information is shared (Q3.2d), and whether sufficient financing for transboundary water cooperation is in place (q.4.2c). While there is complementarity between the two indicators, the first reporting exercise showed that if different experts within the same government are responsible for submitting transboundary data related to 6.5.1 and 6.5.2, inconsistencies may arise in terms of what is reported at the national level. This, therefore, calls for greater coordination to ensure that the results for 6.5.2 indicator are fed into the 6.5.1 results and vice versa. Another challenge faced across national reporting related to transboundary basins is the potential discrepancies that may arise between two or more countries sharing the same basin. More effort is needed to ensure that countries co-ordinate and aim to harmonize data related to the transboundary basins that they share. This provides an important opportunity for countries to develop a shared understanding not only of the current status of transboundary cooperation, but also to agree upon the common steps that might be taken to strengthen cooperation.

4.2. Key Challenges and Lessons Learned During Baseline Application of SDG 6.5.1 Methodology

The survey-based approach inherently possesses challenges related to objectivity, transparency and comparability of the results. These, to a large extent, relate to the views of the person (or persons) vetting and submitting the final results, but also to the interpretations of the questions themselves. The following sub-sections briefly discuss the key challenges and lessons learned of the SDG indicator 6.5.1 assessment approach.

4.2.1. Objectivity and Transparency in Country Assessments

While in many countries the national Focal Points (FP) consulted other colleagues, sectors and stakeholders, it is hard to completely eliminate the range of potential bias that may be introduced by the responding person or countries’ wider strategic and political priorities, and views, or their interpretation of what implementation of IWRM may translate to in practice.

To support countries in data collection, but also to enable a cross-sector dialogue, and to ensure that the views of various stakeholders are taken into account in the national assessment, national stakeholder workshops were held in the baseline data collection round in 2017/2018, and the submitted assessments represented negotiated consensus between the attending stakeholders. The majority of (known) workshops were facilitated by GWP’s Country Water Partnerships (the primary role of CWPs in workshop facilitation was to explain the survey methodology, and to guide participants to ensure understanding of the survey questions, and results calculations), based on a standardized approach developed together with UN Environment, the Custodian Agency for SDG indicator 6.5.1. A total of 36 GWP-facilitated workshops were held, covering a total of 1058 participants and 709 institutions.

For any other workshops held and facilitated by countries, without custodian agency and GWP involvement, it is presumed that stakeholder number and compositions vary to a large extent, as only some countries have provided full overview of stakeholders involved. Based on communication with FPs, it is estimated that at least 10 additional countries held some form of stakeholder consultations to fill the questionnaire, thus making for an estimated 26% of countries that filled the survey with support of stakeholder consultations, as part of a global baseline. The methodology does not explicitly require for all countries to undertake consultative workshops, as in many countries the locally available funding is not sufficient. Thus the number of countries with documented workshops is low. However, it should be the ambition that the percentage of countries undertaking stakeholder consultation increases over time.

To increase the transparency of stakeholder engagement, it is also recommended that in future reporting cycles all countries report on their processes for filling out the questionnaire, including details of stakeholder engagement and the specific stakeholders involved. This may also inform on extent to which potentially influential actors may steer the direction of the responses in the consultation. Experiences show that it is important that focus in countries is not only on the number of stakeholders consulted, but also on involving diverse and relevant stakeholder groups that represent sectors such as agriculture, water services delivery, health, civil society, water data and information management, financing etc. Workshop reports from baseline study indicate that this has often been the case, although the diversity of sectors involved differ highly from country to country.

Stakeholder consultations are also crucial to ensure that survey questionnaires reflect the experiences of different levels of administration—the centralized national level ministries or authorities may not have a full overview over the local scale circumstances and implementation challenges. This aspect is partly addressed in the ‘other levels’ and ‘federal’ questions of the questionnaire, which on average scored lower than national level implementation. It was also confirmed during the national data collection processes, where several federal countries were not able to or were challenged to report on the local level implementation status.

4.2.2. Fostering Consensus on Interpretation of Assessment Questions and Thresholds

Previous global assessments also showed that there were challenges in ensuring that survey respondents have similar interpretation of the assessment thresholds, i.e.,—low or very low degree of implementation of cross-sectoral coordination mechanisms may be interpreted differently by different countries, depending on the local needs, or ambitions for such mechanisms to be in place. To address this challenge, specific guidance was provided for all survey questions, for each of the main assessment thresholds, outlining general interpretation of the threshold in the context of the question (See Figure 3 example). This helped countries to interpret the various thresholds, but also to ensure better comparability of the results globally.

Even with specific guiding threshold descriptions provided, however, there may be country circumstances that affect the interpretation of the questions or the underlying assessments. In addition to various levels of ambition or local challenges, there may be political [39] (and other) factors that affect the evaluation of some specific dimensions of IWRM. This inherently affects the comparison of the assessment results between countries, where a similar IWRM implementation score may, in fact, mean different things in different countries.

Such factors are beyond the influence of data collecting organisations. But to ensure that responses are substantiated, all countries were asked to provide further narrative explanations to their score assignments during the quality assurance process. These were intended to help better understand the reasoning beyond the responses, but also encouraged countries to reflect on the various thresholds of the scoring. Examples include narrative information on specific challenges facing the implementation, laws and regulations adopted, and description of other measures taken to further IWRM. These notes provide a valuable source of information on implementation in practice. From a methodological point of view, it also helps to better understand how individual countries have interpreted survey questions, and the assessment thresholds, and helps to dissect some critical aspects, that may otherwise be masked in the final aggregated score.

Consideration has also been given to possible sources of bias stemming from design of the survey, in interpretation and scoring of questions. For example, the order of threshold presentation (lowest first, and highest last), may impact on scoring. However, given the logical order (low—to high implementation signifying country progress) and specific descriptions of each threshold, it is believed to be an insignificant source of bias.

Further quality assurance steps included requiring countries to revisit questions where the narrative explanations did not appear to match with the question scores, elimination of non-substantiated N/A responses, and requiring more details on questions scored as 0 and 100, representing the two scoring range extremes.

4.2.3. Ensuring Comparability of Results Over Time and Amongst Countries

What may be a bigger challenge than the comparability of the results between countries is the comparability of the results of the same country over time. Administrative and personnel changes may translate to changes in national focal points, but also changes in the reporting environment within countries, introducing new priorities, challenges or biases in the reporting. This cannot be fully remediated other than providing consistency in survey questions and especially clarity in the definition of the guiding thresholds. In their workshop reports, countries also reported that it was easier to reach consensus on the scores when they could be based on evidence. Therefore, the narrative explanations and reasoning behind scoring becomes particularly important—as a point of reference for any future reporting in countries—for the new national focal points and custodian agency alike.

4.2.4. The Challenges of Setting Global and National Targets

All SDGs have targets to help focus and coordinate efforts towards the overall goals. Clearly, the indicators need to be able to help measure progress towards the targets. With the indicator results being scored on a scale of zero to 100, it is natural that targets should correspond to the same scale. As the implementation of IWRM is an ongoing and continuous process rather than an end in itself, it is challenging to set targets that intuitively represent an ‘end-point’. This is partly addressed by the threshold descriptions for a score of 100 for many of the individual questions, which typically stipulate that capacity and funds are sufficient for a particular IWRM element, that objectives are achieved and that the elements are continually reviewed and revised (i.e., acknowledging the ongoing nature of governance).

At the global level, in line with the 2030 Agenda, setting aspirational targets helps to drive and accelerate implementation [51]. The global, aspirational target, defined by the SDG indicator 6.5.1. Working Group and Custodian Agency is an overall score between 91 and 100, a ‘very high’ degree of implementation [36]. It is recognized that it is extremely challenging for countries to score 100 on every question, and a score of 100 on most questions means that a country has the majority of mechanisms in place for excellent water resources management. Where a few elements require minor improvements (e.g., scores of 90 or 80), the established processes of review and revision are expected to result in these improvements. It is therefore suggested that an overall score of 91–100 indicates that a country has implemented IWRM at all levels, as called for in SDG target 6.5.

For countries to achieve an average of 91 (very high), the majority of questions would need to be scored as ‘high’ or ‘very high’. This is based on an indicator score calculated as an average of 33 questions (if all are applicable). However, countries would be able to score selected questions in the lower in the middle-low range, and maintain the average score of 91 or higher, given majority of questions are scored high or very high. Knowing that effects of governance changes may take decades to manifest, the measure of success of IWRM implementation will also be the rate of increase in the indicator score. The global average target of 91 or higher remains a global target to be achieved by 2030.

Recognising that countries are at very different stages of implementation, and that achieving a score of 91 or higher by 2030 may be unrealistic, it is expected that countries may wish to set targets in line with national contexts and priorities. The baseline questionnaire can be a useful tool in multi-stakeholder processes to agree on such national targets. As levels of ambition, country contexts and priorities change over time, these national targets may be periodically reviewed. Again, it is important to remember that any improvements a country can make in implementing any of the elements of water resources management (reflected by higher question scores), are likely to lead to more sustainable and equitable outcomes on the ground.

5. Conclusions

Globally, a useful pattern emerges from the 172 country responses on the global status of IWRM implementation, especially across the specific dimensions of IWRM. While a single indicator score is calculated for the purposes of tracking progress on the SDG target 6.5 at the global level, and known challenges in the methodology exist, the survey approach has been able to deliver a global baseline for indicator 6.5.1 on IWRM implementation with scores that can be comparable between countries and over time. In future reporting cycles, attention should also be kept on the individual country progress and especially on using the survey and the data collection processes to further the implementation of IWRM nationally. In data collection, focus should be on increasing stakeholder engagement at verification at different governance levels in the countries and providing the necessary support to countries for these consultation processes.

The SDG indicator 6.5.1 surveys can be used as a relatively simple diagnostic tool to identify areas of low or high IWRM implementation—thus using the monitoring and reporting exercise to identify the status of implementation of key elements of IWRM, and where implementation can be advanced in line with national priorities. Furthermore, the process of bringing multiple stakeholders together to reach consensus on responses to the survey over the course of SDG reporting can provide a valuable mechanism for intersectoral coordination and collaboration. The SDG indicator 6.5.1 reporting exercise in itself can provide a platform for stakeholder engagement and cross-sectoral coordination, as well as data exchange, thereby supporting implementation of key principles of IWRM and improved water governance. Involving stakeholders in regular governance assessments is key to better understanding of national and local formal and informal policies, laws, rules and regulations governing the interactions between the different levels of water users and de facto governance structures, and in itself function as a better governance mechanism.

Experiences from the SDG indicator 6.5.1 baseline assessment also underline the importance of collecting descriptive open-ended information from countries, providing reasoning behind the individual scores. In light of possible changes in focal point assignments, and institutional setup, this may be an important point of reference not only for the custodian agency interpreting survey results, but also for countries themselves, in their future reporting cycles.

Despite the methodology improvements undertaken from previous global IWRM surveys, it is acknowledged that country responses are likely to retain an element of subjectivity, particularly where multi-stakeholder processes are less extensive. Ultimately, while results are indicative and country-driven, the self-assessed country reporting is designed to be useful to the countries themselves in furthering IWRM implementation. Therefore, the most important issue pertains to what countries do with the information, and how IWRM implementation advances over time, rather than the comparison of scores between countries.

A number of improvements have been suggested to the questionnaire contents–including clarifications on the scope of the questions in some cases, and review of bias that may be induced through presentation of the assessment ladder (e.g., relating to the order in which the thresholds are presented). Considering that accountability, integrity and transparency are key building blocks for improved governance it should be considered how the IWRM indicator, as a measure for water governance under SDG 6, should address these governance dimensions in future assessment cycles. This could be done through amendments in the current methodology such as added open-ended questions, incorporation of relevant indicators from other governance initiatives, such as the OECD governance indicators, or additional sections in the questionnaire. Cross-reference with SDG indicator 6.5.2 on transboundary cooperation, and future harmonization of reporting approaches between the two indicators is also needed.

Overall, it is acknowledged that the SDG 6.5.1 indicator methodology can be further improved across a number of areas. However, only minimal refinements are recommended to the existing survey questions in order to ensure comparability of baseline reporting dataset with future reporting cycles. The SDG 6.5.1 indicator has created a framework for measuring progress of IWRM implementation with a methodological continuity under the 2030 agenda, which has not been possible previously. In the future, it may be possible to match the regular assessments of governance and management aspects of water resources management with their respective ‘impact’ indicators under Goal 6 (and other goals) which will provide for an opportunity to test the link between IWRM ‘theory’ and evidence. This could potentially be extended to analyse the impacts of IWRM implementation linking to some of the other target areas under the 2030 Agenda, such as those relating to freshwater ecosystems.

Disclaimer

The views expressed herein are those of the authors only and do not necessarily reflect the views of their respective organizations.