IN-PLAN PRACTICE
Online Version
Monitoring (and evaluation) are critical tools supporting public authorities in implementing and improving integrated spatial development after the plan has been adopted. It allows the project team to assess whether the intended progress has been made and whether corrections are necessary. The monitoring takes part in parallel to the implementation of the plan and has the main purpose of identifying potential issues ahead of time to take corrective action. This chapter provides guidance and highlights some key concepts and elements for developing and making use of a systematic monitoring system suitable for integrated spatial development.
Establishing a general shared understanding about the purpose of a monitoring system, its benefits, and its limitations is vital. If this is not achieved early in the process of integrated planning, there is the risk to wind up with an ineffective system or waste time in recurring discussions at later stages. To avoid this, the development of a monitoring framework should be started as early as possible and gradually refined and enhanced throughout the different phases of an integrated spatial planning process.
But where to start? A good starting point for the discussion is to look at the differences between monitoring and evaluation activities, which are oftentimes misunderstood or confused with each other. They serve similar but distinct purposes. Monitoring is the principal responsibility of the team in charge of implementing the integrated spatial development plan and focuses on measuring performance and results. It does not question the relevance and effectiveness of the strategy or the plan itself, which is the main purpose of evaluation activities.
Within the context of IN-PLAN we put an emphasis on monitoring activities as they are usually carried out by the project team. Establishing a monitoring system provides several benefits such as the opportunity to learn and improve during the implementation phase, enhance the effectiveness of the plan, the possibility to inform other actors about progress and challenges in achieving the goals of the municipality, increased transparency and accountability as a public body.
To steer the discussion when developing the monitoring system, it is advisable to look at some example goals and discuss how they could be measured and the necessary data obtained. But it doesn’t stop there. Who is then responsible for interpreting the data and draw conclusions from it? Is the project team solely responsible or are other municipal departments involved in this step? Questions such as this help to define clear roles for carrying out the monitoring. A good way to start is to define who is responsible for the collection of data, which bodie(s) are involved in analysis and interpretation of the data and how conclusions can then be drawn from the results.
Another important question is periodicity. The monitoring process should be carried out in a regular interval. This helps everyone involved to prepare for the reporting in advance and align other (internal) processes and their outcome with the timeframe of the monitoring process. The actual frequency depends on the available resources as well as the technical and political context. A reasonable timespan is every two years which is also used for monitoring SECAP plans.
The “Lahti 2030 Sustainable Energy and Climate Action Plan” is a strategic initiative by Lahti, Finland, to enhance sustainability by 2030. It focuses on renewable energy, energy efficiency, sustainable mobility, and climate adaptation. Progress is monitored every two years, with emissions recalculated every four years, and continuous monitoring by the city ensures effectiveness. The city does that by itself: The Environmental Developments Unit of the City of Lahti oversees implementation and monitoring; specific bodies are assigned to the individual measures of the Action Plan for reporting and accountability.
For further information see 2030 Climate Neutrality Action Plan of the City of Lahti.
Integrated spatial development operates within an extraordinarily complex system. Outcomes are influenced by a range of interrelated factors that usually make it difficult to assess the direct impact of singular actions. Planning and its activities are one element influencing real world outcomes but not the only one. In addition to that there are limits regarding what can be measured and availability of data.
Let’s look at an example and assume that one of the overall goals of the plan is to promote sustainable mobility which reduces negative effects on the environment, improves health and helps create vibrant neighbourhoods. Progress on this goal can be measured by an indicator such as the modal share of eco-friendly modes of transportation (walking, cycling, public transport). This is an indicator that is available in many countries and cities and is usually regularly updated. Let’s assume that the plan includes the goal to increase the modal share of eco-friendly modes of transportation to 70% by 2030. This is an example for a so-called outcome-based indicator.
Monitoring outcome-based indicators is very meaningful when looking at intended results. However, it does not provide any information on the progress of implementation measures. It’s easy to imagine a scenario in which the modal share increases in line with the plan’s goals, not because of city action, but because of rising gasoline prices.
Outcome-based indicators can be complemented or replaced by input or output-based indicators. They provide the project team with a better understanding regarding the progress in implementing the plan. Using the existing example of promoting sustainable mobility, we can easily come up with the amount of infrastructure expenditure as an illustrative example for an input-based indicator (e.g. yearly amount of Euros per capita invested to improve public transport in the city). The number of kilometres of newly constructed tram tracks would then be the corresponding example for an output-based indicator.
The example shows that outcome, output and input-based indicators all provide valuable information and that they can complement each other. It is of course also possible to focus either solely on outcome-based indicators or solely on input and output-based indicators. In such a case this should be clearly communicated and coherently implemented to avoid confusion.
We have used the example of sustainable mobility so far as it is quite intuitive, and data availability is comparatively good. However, there are always constraints that make concessions necessary. Social cohesion is an example that would be much harder to measure and find regularly updated data. In such cases efforts should be made to improve data availability in the future and to pragmatically focus on input and output-based indicators meanwhile (e.g. increase in number of local neighbourhood improvement initiatives supported by the city).
In the end it is important to stay pragmatic when defining the set of indicators for monitoring. Strive for a complete set of indicators but stay within the boundaries of available resources. Focus on the most relevant indicators and avoid the inclusion of indicators that provide little or no value for the purpose of the monitoring system. Otherwise, you run into the risk of creating an overly complex monitoring system that will be difficult to execute.
There are numerous valuable resources and guidelines providing help to define indicators for a monitoring system. For example, have a look at the guidelines by URBACT, the concept of SMART targets and the Handbook on Sustainable Urban Development Strategies by the European Commission.
For further information see Handbook of Sustainable Urban Development Strategies.
After the adoption of the integrated plan the monitoring process can be initiated. Ideally a compact section describes the key points of the monitoring process in the draft plan itself and thereby has become part of the official document that can be referenced if necessary. Depending on the extent and design of the agreed process preparatory activities may have to start immediately after adoption. In general, the monitoring process can be expected to run as smooth as possible when its benefits and roles have been clearly defined as we have elaborated on in the previous chapters.
But how to proceed now that initial data on progress surfaces and the data has been looked at? Discuss the raw data within the project team and consult and engage with other actors and experts to come up with a shared understanding of the results. Guiding questions for this step include:
These questions can be discussed in various formats (e.g. internal meetings, interviews, workshops, …). Whatever the form of these discussions, it is important to capture the generated insights in the form of short summaries that can then be used in the next step of the monitoring process and form the basis for an implementation report for utilising and communicating the monitoring results.
The City of Vienna has developed a monitoring system with a comprehensive set of quantitative and qualitative indicators that can be publicly examined. The set of indicators has been continuously updated and allows to track the progress on achieving the goals of Vienna’s Smart Climate City Strategy. Previous results have been publicly reported in the form of a monitoring report that graphically visualises progress towards the goals.
For further information see Smart City Wien. Monitoring Report in English: Monitoring Report 2017. Latest set of indicators used for monitoring: SKCSW Indicators (only available in German).
We have shown in the earlier chapters that monitoring integrated planning should be designed in such a way that it provides clear benefits and accountability. Now that initial findings on the monitoring process have been collected it is possible to produce an implementation report that summarises the results and that can be shared and communicated within a broader context.
Relevant stakeholder groups that may be addressed now include:
The type of information and the level of detail that each of these groups need may vary. It is also possible that it is the responsibility of actors outside of the project team to inform about the progress and monitoring of the plan. Digital tools such as a dashboard can also be a great way to communicate the results of the monitoring process and are more easily held up-to date compared to a general report.
Whatever the approach used to inform the groups outlined here, try to cater to each group’s information needs and be as open and transparent about the results and underlying data as possible. Especially goals that have been identified in the monitoring process as lagging behind should not be seen as a shortcoming, but as an opportunity to highlight the necessity to take further action and to find new partners to improve on implementation.
The digital Sustainability Report provides an annually updated overview on the progress of the development of Stockholm’s Royal Seaport. For each strategic area, readers can find out more about how requirements for green space factors, energy, waste, transportation, materials, and indoor environment are met.
For further information see Stockholm Royal Seaport 2030.
