T OOL U SABILITY : I MPROVEMENTS TO E VALUATION T OOL

There were identified some limitations and improvement opportunities in the use of the indicators during the case study evaluation.

Indicators three-areas of evaluation

Firstly, the evaluation means for systems performance were designed considering that there is publicly available data at the district scale. There was the challenge of not having access to district level statistics. Therefore, adjustments, assumptions and interpretation of statistics needed to be made. An example of it is indicator No. 6-Convenience for active mobility. On it, the system performance evaluation was related to the trends in the registered local accidents for active mobility.

Nowadays, the relation of the district accidents can only be measured if surveys estimate it or if there are private sources for district-related measurements about it. Since there was no district-related information on it, an estimation was made from the proportion of the city accidents. For further use of the tool, either district-level statistics need to be generated or estimations can be used again. On this indicator in particular, it was decided not to change the evaluation mean “ district accidents proportion in relation to the city ones” as even if not proven due to the lack of available data, one cannot state the city active mobility safety by analysing only everything as a whole, as the conditions, the infrastructure and the affluence of use are very varied along with the different city districts.

Secondly, on the layout evaluation, most of the data collection was meant to be from on-site observations; this means, visiting the project area and walk through it to respond questionnaires related to the physical features of the districts. The on-site observations represent a qualitative measurement. The intend to conduct these is to observe the physical conditions of the built environment and their overtime change. The on-site observations results are highly subjective, as there can be presented significant variations among the grading given by either experts or professional familiar with the field of planning and urban design, or if it is provided by a user with no relation of this expertise field. The best fit found on how to adapt the layout grading was to combine the on-site observations with project plans audit. Then, quantitative data can be incorporated, and the resulting grade can become a fairer measurement. An example of this is the indicator No. 4-Public life that enables social cohesion. This indicator requires to observe qualities of the public space and how many spaces with these characteristics are available in the district (units per hectare). Such characteristics are the variation on type: Open private spaces, open public spaces, playgrounds, promenades, squares, landscape park; the different uses: recreation, sports, gastronomic use, room for various unplanned and unexpected uses; the available urban furniture and equipment: seating areas, bicycle parking areas, public transport stops, lighting, and lastly, how welcoming are they for all users and age groups. These four characteristics-means for evaluation can be observed on a field study, but the availability per hectare requires to measure areas either in project plans or city maps.

Therefore, the best-identified way to come up with a fair way of evaluating it was to combine on-site observations and project plans audit. The availability per hectare was measured in project plans.

59 Lastly, it was generated a survey for the user experience evaluation. The propose was to come up with at least two questions per indicator, one related to satisfaction, and another one related to behaviour and preferences. The intend to make it a quantitative assessment with closed-ended questions resulted in a numerical understanding not so flexible for people’s opinion. Some of the surveys were collected directly by interviewing the residents in the district. Enabling conversation allows us to learn about how particular lifestyles result in different experience of the same project area. For example, when evaluating the indicator no 3. Safe and attractive public space, 97% of the people responded they feel either super safe or safe in the district area. However, when having conversations with the neighbours, it was learned about the impact of the popularity of the public space and the inconvenience it could cause for neighbours. One of them mentioned about early morning walk to do bathing in the waterfront and having to walk around broken glass bottles result of people’s party in the waterfront area, which is highly popular in summer times. Even if the overall maintenance and cleanness were highly ranked by most of the residents, being a user that experience the public space early in the mornings can provide a very different picture. An idea on how to improve the responses related to the user’s experience is to look ad demographic profiles of the residents and to try to have a representative poll of answers on the surveys, on which more interest and profiles can be reflected, to have a greater understanding of the experience of all age groups and interests throughout the day and year usage of the space.

Data collection and analysis

The data collection was a significant process and required to look at many varied sources, as the indicators evaluate three different areas. The data analysis and calculations do not represent significant complexity, but if there is no access to data sources, then the calculations or data analysis becomes unreliably. On the analysed case study, the project owner develops yearly POE. This data availability assisted in understanding patterns throughout the years and allowed access to district-level data. But in many other districts in the city, there are no ongoing evaluations, and it could be difficult to get access to district-level statistics as for example on the indicator No. 6-Convenience for Active Mobility where it is pretended to estimate the correlation of active mobility accidents on the district level. Consequently, the tool usability is highly dependent on the availability of district level public or private data.

Adjustments made to the indicators

On the indicator No. 1-Safe storm and rain experience, the layout evaluation was resumed. Initially, the strategies for storm and flooding were listed separately. After understanding the high correlation of them, it was decided to have a layout evaluation that assesses storm and flooding together. As mentioned before, it was required to adjust some indicators and incorporate project plans audit, to result in a reliable way of grading the Layout through the on-site observations combined with numerical values that support the grade obtained. Another example of this is the indicator No. 7-Wind comfort. This indicator evaluates on layout the relation of building height to the street width, as well as the street variation throughout its length. The first element has as unit value a proportion and the second one meters. For this, it was required to consult maps and project plans to give a realising understanding of the parameters.

60 To summarise, even if there were difficulties in collecting the data, the tool was possible to utilise on this case study. The tool provided a holistic overview of the project, assessing its sustainability qualities in terms of liveability, resulting in a high correlation of liveability and sustainability.

61

62