- •Sustainability Assessment
- •Sustainability Assessment
- •Academic Press is an imprint of Elsevier
- •First published 2013
- •Notices
- •British Library Cataloguing-in-Publication Data
- •A catalogue record for this book is available from the British Library
- •Library of Congress Cataloging-in-Publication Data
- •A catalog record for this book is available from the Library of Congress
- •For information on all Academic Press publications visit our website at store.elsevier.com
- •List of Abbreviations
- •1 Sustainability Assessment of Policy
- •1.1 Introduction
- •1.2 Rationale
- •1.3 Understanding Discourses
- •2 Sustainability Climate of Policy
- •2.1 Introduction
- •2.2 Emergence of Policy Sustainability
- •2.2.1 Population and Resource
- •2.2.2 Modernity and Sustainability
- •2.3 Concept of Sustainability
- •2.3.1 Steady-State Economy
- •2.3.2 Carrying Capacity
- •2.3.3 Ecospace
- •2.3.4 Ecological Footprints
- •2.3.5 Natural Resource Accounting/Green Gross Domestic Product
- •2.3.6 Ecoefficiency
- •2.4 Sustainability Initiative
- •3 Characterizing Sustainability Assessment
- •3.1 Introduction
- •3.2 Resource System
- •3.3 Social System
- •3.4 Global System
- •3.5 Target Achievement
- •3.5.1 Detection of Changes
- •3.5.2 Determining Operation Scale
- •3.5.3 Harmonizing Operation Sequence
- •3.6 Accommodating Tradition and Culture
- •3.7 Selection of Instrument
- •3.8 Integration of Decision System
- •3.9 Responding to International Cooperation
- •4 Considerations of Sustainability Assessment
- •4.1 Introduction
- •4.2 Socioeconomic Consideration
- •4.2.1 Nature of Poverty
- •4.2.2 Nature of Resource Availability
- •4.2.3 Nature of Economy
- •4.2.4 Nature of Capital
- •4.2.5 Nature of Institutions
- •4.3 Consideration of System Peculiarities
- •4.3.1 Temporal Scale
- •4.3.2 Spatial Scale
- •4.3.3 Connectivity and Complexity
- •4.3.4 Accumulation
- •4.3.5 Nonmarketability
- •4.3.6 Moral and Ethical Considerations
- •4.4 Consideration of Component Peculiarities
- •5 Issues of Sustainability Assessment
- •5.1 Introduction
- •5.2 Issues Related to Society
- •5.2.1 Social Modernization
- •5.2.2 Societal Relationship
- •5.2.3 Radicalization and Convergence
- •5.2.4 Boserupian/Neo-Malthusian Issues
- •5.2.5 Social Ignorance
- •5.2.6 Social Attitudes
- •5.3 Issues Related to Policy Discourse
- •5.3.1 Discourses of Story Line
- •5.3.2 Discourses of Disjunction Maker
- •5.3.3 Discourses of Symbolic Politics
- •5.3.4 Discourses of Sensor Component
- •5.4 Issues Related to Actors
- •5.4.1 Influences of Macroactors
- •5.4.2 Positioning of Actors
- •5.4.3 Way of Arguing
- •5.5 Black Boxing
- •6 Components of Sustainability Assessment
- •6.1 Introduction
- •6.2 Social Adequacy
- •6.3 Scientific Adequacy
- •6.4 Status Quo
- •6.5 Policy Process
- •6.6 Policy Stimulus
- •6.7 Participation
- •6.8 Sectoral Growth
- •6.9 Resource Exploitation
- •6.10 Traditional Practices
- •6.11 Role of Actors
- •6.12 Framework Assessment
- •6.13 Scope Evaluation
- •6.14 Evaluation of Implementation
- •6.15 Instrument Evaluation
- •6.16 Structural Evaluation
- •6.17 Cause Evaluation
- •6.18 Cost Evaluation
- •6.19 Impact Assessment
- •6.20 Quantitative Approach
- •6.21 Anthropogenic Evaluation
- •6.22 Influence of Other Policies
- •7 Linkages of Sustainability Assessment
- •7.1 Introduction
- •7.2 Parallel Linkage
- •7.3 Linkage of Ascendancy
- •7.4 Linkage of Descendancy
- •7.5 Linkage of Hierarchy
- •7.6 Horizontal Linkage
- •7.7 Quasi-political Linkages
- •7.8 External Linkage
- •7.9 Market Linkage
- •7.10 Evaluation of Link to the Past
- •7.11 Actors and Story Line
- •7.12 Practices and Story Line
- •7.13 Reflection of Image of Change
- •7.14 Integrating Information
- •7.15 Forecasting
- •7.16 Assessing Options
- •7.17 Post-decision Assessment
- •8 Assessment of Policy Instruments
- •8.1 Introduction
- •8.2 Approaches of Implementation
- •8.3 Attributes of Instrument
- •8.4 Choice of Instruments
- •8.5 Instruments as a Component of Policy Design
- •8.6 Addressing the Implementation of Instruments
- •9 Social Perspectives of Sustainability
- •9.1 Introduction
- •9.2 Participation Evaluation
- •9.3 Process Evaluation
- •9.4 Retrospective Policy Evaluation
- •9.5 Evaluation of Policy Focus
- •9.6 Deductive Policy Evaluation
- •9.7 Comparative Modeling
- •9.8 Deductive Modeling
- •9.9 Optimizing Perspectives
- •9.10 Political Perspectives
- •10 Factors of Sustainability Assessment
- •10.1 Introduction
- •10.2 Actor as Policy Factor
- •10.3 Global Resource Factor
- •10.4 Local Resource Factors
- •10.5 Participation Factor
- •10.6 Participation Catalyst
- •10.7 Economic Factors
- •10.7.1 Influence of Macroeconomic Factors
- •10.7.2 Influence of Microeconomic Factors
- •10.7.3 Influence of Private Investment
- •10.7.4 Influence of Public Investment
- •10.7.5 Influence of Economic Incentives
- •10.8 Administrative Factor
- •10.8.1 Right and Tenure
- •10.8.2 Decentralization
- •10.8.3 Accessibility
- •10.9 Market Influence
- •10.10 Historical Factor
- •10.11 Other Factors
- •11 Tools for Sustainability Assessment
- •11.1 Introduction
- •11.2 Indicators for Evaluating Resource Dimension
- •11.2.1 SOR Indicators
- •11.2.2 NFR Indicators
- •11.2.3 Effectiveness Indicators
- •11.2.4 Comparing Indicators of Resources
- •11.2.5 Explanatory Variables
- •11.2.6 Tools for Assessing Human Dimension
- •12 Problems in Sustainability Assessment
- •12.1 Introduction
- •12.2 Boundary Problem
- •12.3 Problem with Social Concern
- •12.4 Role of Science
- •12.5 Institutional Difficulty
- •12.6 Implementation Problem
- •12.6.1 Circumstances External to the Implementing Agency
- •12.6.2 Inadequacy of Time, Resources, and Programs
- •12.6.3 Lack of Understanding Between Cause and Effect
- •12.6.4 Minimum Dependency Relationship of Decisions
- •12.6.5 Lack of Understanding of, and Agreement on, Objectives
- •12.6.6 Policy Tasks not Specified in Correct Sequence
- •12.6.7 Lack of Perfect Communication and Coordination
- •12.6.8 Rare Perfect Compliance of Implementing Body
- •13 Discussion and Recommendation
- •13.1 Discussion
- •13.2 Recommendation
- •13.3 Importance
- •Summary
- •References
Social Perspectives of Sustainability |
111 |
In the case of sustainability assessment of policy, concept validity seems more appropriate. This validity concerns the degree to which any assessment device measures what it purports to measure. The central question of the validity is whether the inclusion of created or chosen instruments compares the desired concept or not. Two steps may be needed to determine these forms of validity:
1.Conceptualizing different assessment areas to measure the concept.
2.Finding their central dimension.
For example, sustainability assessment of environmental role of forest land use can be assessed on the basis of existing forest area and/or reduction of water flow. Determining the central dimension (usually by factor or cluster analysis), which in this case may be called derived dimension, is also important. Each variable under consideration is then correlated with that derived dimension. If there is no central dimension, several independent measures of the concept could be used. Reliability is also another important concept needing to be determined, which usually deals with the instruments used for measuring a policy. As in this case, there may not be any mechanical instrument for measurement, more specifically a measure’s reliability is the degree to which the same scores would be obtained if it were possible to repeat assessment procedures.
9.2 PARTICIPATION EVALUATION
Participation evaluation is one of the important phases of assessing the social dimension of a policy. Participation evaluation is targeted to assess the potential and performance of policy actors. However, to identify the potential of the actors, participation evaluation also includes the definition and identification of problem and evaluates the constraints and opportunities of the system. The roles and potentials of actors are clarified on the basis of constraints and opportunities the actors enjoy within the policy. Those potentials are used as a background scale for measuring the performance of the actors. Different aspects of participation evaluation are presented in the Rapid Appraisal of Agricultural Knowledge System (RAAKS) model (Fig. 9.1). Using the guideline shown in the RAAKS model, only the participatory aspect of a policy evaluation may be possible. The model can help in building up the information base, but it helps little about the physical resource base. Thus, the policy evaluations need to go beyond the model. There are some limitations
112 |
Sustainability Assessment |
|
|
Phase—A |
Phase—B |
|
Problem definition and |
Analysis of constraints |
|
system identification |
and opportunities |
|
A1: Objective of |
B1: Actor analysis |
|
(their character) |
|
|
analysis |
|
|
|
B2: Task analysis |
|
A2: Identifying |
(who does what) |
|
|
|
|
|
B3: Communication |
|
A3: Diversity in |
analysis (coalition in |
|
actors objectives |
the understandings) |
|
A4: Identifying |
B4: Integration |
|
analysis (who links |
|
|
important factors |
|
|
with whom why) |
|
|
|
A5: Redefinition of |
|
|
|
B5: Coordination |
|
|
|
|
analysis (who is most |
||
problem situation |
|
|
|
||
|
|
|
|||
|
|
|
influential) |
||
What are the problems to be |
|
|
|
||
|
|
|
|
|
|
evaluated |
|
|
|
|
|
|
|
|
B6: Network analysis |
|
|
|
|
|
|
|
|
|
|
|
|
(sources of |
|
|
|
|
|
|
|
|
|
|
|
information) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
B7: Impact analysis |
|
|
|
|
|
(effectiveness and |
|
|
|
|
|
|
|
|
|
|
|
efficiency analysis) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
B8: Summary of analysis |
|
|
|
|
|
|
|
Phase—C |
Action planning |
C1: Management analysis (how |
the actors improve the system) |
C2: Actor potential analysis |
(who can do what) |
Planning (who does what, when, etc.)
Fig. 9.1 RAAKS model. Winter (1996).
of RAAKS model to identify the roles of certain actors particularly those who influence the policy from outside the system. Therefore, process evaluation may be taken as complementary to participation evaluation.
9.3 PROCESS EVALUATION
Process evaluation deals with how the policy has been formulated and/or implemented, and thus explains how external and internal are actors involved in the policy process. The sustainability and environmental components of resource policies are now coming under the influence of many pressure groups both endogenous and exogenous. However, though there could be more than one agency involved in policy formulation, usually one agency is involved in implementation, which is important from the perspective of policy evaluation. According to Winter (1996), the models of the policy processes are:
•Formal structural model,
•Pluralistic model,
Social Perspectives of Sustainability |
113 |
•Marxist model, and
•Corporatist model.
These policy models are not mutually exclusive. In fact more than one model could be used for policy formulation and it is not impossible that a different combination may be needed for a different society and each society may have a different process model. The institutional framework engaged in the policy formulation might have an effect on the policy output and such differences need to be considered during policy evaluation. The best way to achieve a better result is to consider policy implementation as a part of the policy process. Barrett and Fudge (1981) wrote:
it is essential to look at implementation not solely in terms of putting policy into effect but also in terms of observing what actually happens or gets done and seeking to understand how and why.
When a policy is about resources, some negotiation is usually needed between different actors and agents, as each of them tends to maximize their benefit. The political process, by which the policy is mediated, negotiated, and modified during its formulation and legitimization, does not stop when initial policy decisions are made but continues to influence policy through the behavior of those responsible for implementation and those affected by the policy actions for protecting their own interest. This view of implementation takes away the traditional approach on formal organizational hierarchies, communication, and control mechanisms and places more emphasis on:
•the multiplicity of actors and agencies involved and the variety of linkages between them,
•their value systems, interests, relative autonomies, and power bases, and
•the interaction that takes place between the actors and the agencies.
The identification of key stages in policy implementation is a crucial procedure in evaluation and extraordinarily it is one that is neglected in official programs of evaluation. The identification of stages depends on the type and nature of the policy in consideration, but five key elements have been suggested by Winter (1996):
1.Publicity—potential recipients know about the policy scheme through publicity. The evaluation identifies whether there was any bias in publicity or whether some recipients heard about the scheme before the others.
114 Sustainability Assessment
2.Advice/extension—recipients are made involved in the scheme through advice/extension. The evaluation deals with whether sufficient advice was available to recipients or not. Whether the advice was accurate or unduly optimistic or pessimistic. Whether there was any bias in the advice so that some recipients are more or less likely to respond than others.
3.Participation in the scheme—identifies how the recipients are involved in the policy scheme. The evaluation delineates the administrative ease to embark on the scheme, whether any financial problem exists which may exclude the target groups and once started whether the scheme was managed and administered efficiently.
4.Monitoring and adaptation—identifies the attempts for solving subsequent problems. The evaluation tries to define the kind of monitoring action, how and for what objectives the monitoring action was done and whether the monitoring led to any modifications of the policy.
5.End point—determines whether the end point is spatially and temporally logical to the goal of the policy.
However, there is a limit to the ability to evaluate the roles of players through a process model. In the case of sustainability assessment of policies, such evaluation is generally retrospective (not progressive).
9.4 RETROSPECTIVE POLICY EVALUATION
Retrospective evaluation is different from progressive evaluation. In progressive evaluation, goals and process are given, evaluation tries to track whether the process going to achieve the goals but in retrospective evaluation status of achievement is given but there is a need to establish the goal and processes to explain why the achievement was poor or perfect. Other application of social science theory, methods, and techniques to identify and assess the processes and impacts of governmental policies and programs may remain similar. Possible steps of retrospective policy evaluation are presented in Table 9.3.
In assessing the sustainability perspectives of policies, retrospective evaluation is far and away one of the most important aspects of outlining actor relationship. Hoggwood and Gunn (1984) identified five main categories of technique for retrospective evaluation:
1.evaluating the situation before and after implementation,
2.modeling,
|
Social Perspectives of Sustainability |
115 |
|
|
|
|
|
Table 9.3 Stages in Retrospective Policy Evaluation |
|
|
|
|
|
|
|
Task |
Methods |
Difficulties |
|
|
|
|
|
Establish broad policy |
Examine contents of parliamentary acts |
Secrecy; conflicting goals; |
|
aims, i.e., policy goals |
and statutory instruments, regional |
propaganda versus reality, ulterior |
|
|
directives, ministerial, government or |
motives; post hoc rationalization |
|
|
regional statements, cabinet papers, |
|
|
|
interviews with key officials or politicians |
|
|
|
|
|
|
Identify specific |
As above with additional emphasis upon |
As above |
|
objective or desired |
the policy refining carried out by |
|
|
outputs |
government agencies |
|
|
|
|
|
|
Identify agencies and |
As above with additional experiences |
Multiagency projects |
|
individuals responsible |
upon interviews with key personnel |
|
|
for implementation |
|
|
|
|
|
|
|
Specify key stages in |
As above with additional emphasis upon |
Post hoc relationship can be a |
|
implementation process |
the experiences of recipients |
particular problem |
|
|
|
|
|
Assess achievement |
Define criteria for success, decide on |
Quality of the data, problem of |
|
|
means of measurement, collect data |
other influences, or side effects |
|
|
analysis |
|
|
|
|
|
|
Source: Winter (1996). |
|
|
|
|
|
|
|
3.experimental method,
4.quasi-experimental methods, and
5.retrospective cost benefit analysis.
“Before and after implementation” might appear to offer the most straightforward approaches to assessing the achievements of a policy, but in practice they present considerable difficulties. It is rarely possible, for example, for the policy evaluator to be farsighted enough to establish the full information prior to introducing a policy scheme. Even if information is available, difficult questions may remain whether apparent policy achievements are exclusively the consequence of a particular policy.
In many ways, modeling is a refinement of “before and after” studies. Models attempt to incorporate systematically all those factors that may have influenced a policy outcome. Lack of suitable data and erroneous assumptions about causal relationships provide a formidable challenge to those wishing to evaluate policy through modeling.
As the name implies, the experimental method relies on testing a policy on a particular group of people, ideally retaining a control group with identical characteristics with the experimental group on which the policy is not applied. The experimental method in policy