Evaluation and understanding of the effects of hospital design – mapping healthcare spatial configuration processes

Type of the Paper: Full Paper

Track title: Topic 3: Engagement – co-creation, co-design, design and stakeholder management processes.

Evaluation and understanding of the effects of hospital design – mapping healthcare spatial configuration processes

Andrea Brambilla ¹*, Göran Lindahl ² , Jens Widmark ^{2, 5}, Paul Barach ³, Peter Lanbeck⁴, Stefano Capolongo ¹

¹ Design and Health Lab, Department of Architecture, Built Environment and Construction Engineering (DABC), Politecnico di Milano, 20133 Milan, Italy; andrea1.brambilla@polimi ; stefano.capolongo@polimi.it

² Center for Healthcare Architecture (CVA), Division of Building Design, Department Architecture and Civil Engineering (ACE), Chalmers University of Technology, SE-412 96 Goteborg, Sweden; goran.lindahl@chalmers.se

³ Jefferson College of Population Health, Thomas Jefferson University, Philadelphia, PA 19107, USA and School of Medicine and Law, Sigmund Freud University, 1020 Vienna, Austria; paul.barach@jefferson.edu;

⁴ Skåne University Hospital, Malmö, Sweden; peter.lanbeck@skane.se

⁵ Faculty for Health and Society, Dept. of Criminology, Malmö University; jens.widmark@mau.se

Abstract:
The aim of the paper is to report and reflect on an evaluation study of a hospital project in Sweden, where the relationships between healthcare performance objectives and indicators of the built environment were studied. The starting point of the study was the performance outcomes set by the hospital and a retrospective analysis of how these could be related to the design of the new hospital. We used qualitative data collection methods, including semi-structured interviews and focus groups, and the findings were compared with the intended design goals of the project. The paper discusses evaluation against the backdrop of existing studies on project evaluations and makes the argument that there is a pressing need for new tools and methods to strengthen design work. We argue that it is valuable to evaluate the work processes in healthcare against spatial configurations in addition to studying the effects of specific design features. Connecting healthcare process indicators to spatial design plans addresses several challenges and possibilities that are addressed in the paper. The study presented is mainly qualitative with an explorative approach.

Keywords: healthcare built environment; methods development; mapping; healthcare processes; design effects; evaluation

1. Introduction: Built environment and evaluation
Understanding the built environment, and healthcare facilities in particular, is a challenging endeavor. Understanding the effects of a particular design or layout on the activities planned to be executed requires robust, reliable methods and approaches (Brambilla & Capolongo, 2019; Li et al., 2018). It is interesting to note that in the field of medicine and, to some extent, healthcare management, measure-oriented approaches and quality assessment tools are regularly used as the foundation of clinical activities and to foster accountability and support continuous quality improvement of the services delivered (Brambilla et al., 2021). This is, however, neither done for the design and planning of healthcare facilities nor in the evaluation of organisational or clinical outcomes related to facilities. In the history of research on building evaluation, we find aspects related to the efficiency of the buildings per se (Preiser & Vischer, 2005), but also initiatives to understand the users and their context to understand what is valuable and meaningful to them (Vischer, 2008). This is an area that spans from quantitative aspects to qualitative aspects and includes both organizational and individual objectives. This paper begins with a study conducted at the New Hospital in Malmö (Nya Sjukhuset I Malmö, NSM) in Sweden, as well as discussions within the framework of the international working seminar "Healthcare Design and Evaluation" held at Politecnico di Milano. This is an ongoing project in which there is an identified need to explore new possibilities to evaluate the new hospital's healthcare processes against the facilities it uses. The study is based on an evaluation of clinical and organisational objectives and to what extent they can be related to the effects of or as a consequence of the facilities. To this is added a discussion on the challenges of meaningfully measuring and evaluating healthcare facilities.

2. Background: To measure or not to measure buildings

Research on facility efficiency is not new; it has been ongoing since the 1960s in various forms. This effort has focused on the performance of the buildings themselves and the effects of the building components. Methods and processes for documenting the effects of buildings and premises have also been developed (see, e.g., Leaman et al., 2010; Preiser, 2005). Especially methods to evaluate buildings once they have been occupied and taken into use. However, what remains to be developed are methods for linking business goals to the design of premises that can be included in systematic follow-ups. This is a challenge in healthcare, where operations usually change in connection with technological transformations for diagnostics and treatment, the evolution of medicine, organisational and leadership changes, etc. The impacts of facilities are therefore difficult to describe without systematic support from structured and systematic methods incorporating both clinical processes and aspects of the facilities.
In the pre-stage, where simulation models are developed, there are methods available for evaluations. These are generally surrounded by a higher degree of uncertainty, and their level of usefulness is yet to be recognised (Schaumann et al., 2020). Once a building is in use, a common method is to do a post-occupational evaluation (POE). POE is seen as a way to ensure that a building delivers what is actually expected and its consequences in the form of "ends" instead of how this is provided ("means") (Adrian, 2003, p. Two main issues are proposed to guide the work: "How does the building work?" and "Is this what it intends to do?" (ibid.). The second question here is what most often challenges systematic evaluations, as there are seldom robust baselines or pre-study datasets to compare against, nor are evaluation criteria set at the briefing stage. Additionally, as noted above, the intended use may not be exactly what is needed once a facility is completed. This often results in the effect that new is better, the “newness effect,"  which is usually true for new buildings in their first phase of use but might not last.

Reliable tools are needed to successfully evaluate buildings and provide feedback during and after the design and construction process (Macmillan, 2004). Four approaches that are proposed, which can be used both in combination or separately, are: i) observations; ii) questionnaires and interviews; iii) informative discussions and physical reviews; and iv) statistical tests and analyses of performance. In the chapter “Learning more from what we build” of Macmillan’s book “Designing Better Buildings,"  Bill Bordass states that many consider the best results to be produced through a combination of "hard" and "soft" values (Macmillan, 2004). Francis Duffy, in 1990, recommends conducting in-depth measurements by considering that they must be operational through validity and reliability in the measurement process, that is, rigorous and practical to carry out (Duffy, 1990). The measurements should be clear to specific stakeholders and the general public as well, in such a way that it is clear what is being measured and for what purpose it is being measured; in other words, why it is advantageous to make the specific measurement or evaluation. An evaluation must also be performance-based, which is directly related to individual or organisational outcomes. The measurements should also be comparative to enable information to be stored in relational databases and encourage benchmarking and comparisons both within and between organizations. Furthermore, Duffy suggests that an evaluation should be broad and equally capable of evaluating the intricate details of a specific workstation at a large factory premise (Duffy, 1990).

It is also important to ensure that the evaluations are made in a robust manner, even if the information cannot be validated as in laboratory-controlled experimental studies (Leaman et al., 2010). The results of an evaluation also depend on the circumstances in which the evaluation is made, how the buildings or facilities are operationalized, and in what context they are evaluated; nevertheless, the methods adopted for the evaluation should be similar or identical for different settings. Indeed, the ways in which the evaluation is carried out should, whenever possible, be repeatable, credible, and provide reliable results. The key challenge, however, remains: can an organisation effectively do what it proclaims, and to what extent are its successful operations related to the facilities they use?

3. The case and methods: The case study of Nya Sjukhuset i Malmö Hospital (NSM)

The starting point for this paper is the operational goals developed for the NSM project, a complementing and refurbuising project for a regional hospital, one of two sites of Skåne University Hospital, serving 1.4 million people. The total size of the healthcare building is 108,000 square meters. The project also includes service buildings, replacement buildings, and technical infrastructure. The project's budget is SEK 12.3 billion for construction and SEK 1.9 billion for equipment. The decision to carry out the redevelopment was made by the Regional Council in 2013, with a final decision on budget and scope made in 2017. The initial allocation of SEK 6 billion is to be completed between 2014 and 2020 (Region Skåne, 2021b).

The new healthcare facilities will consist of two interconnected buildings of 10 and 11 floors, respectively (Region Skåne, 2021a). The project will add a total of 244 beds, 23 surgical theatres, and an intensive care unit of 14 beds. In addition, new service structures and facilities will be built, e.g., a new culvert system covering the whole hospital, sterile units, additional facilities, a new mortuary, waiting rooms, and administrative workplaces. The laboratory facilities will be completely replaced. The healthcare building's planning began in 2015, construction began in 2018, and it is expected to be completed in 2024–25. The NSM project is based on the necessary adaptation to evolving healthcare needs and the maintenance difficulties of ageing buildings (Region Skåne, 2021). The new facilities must be flexible and adaptable to future working methods and care requirements.

The project is guided by operative goals that are divided into political goals and the planning principles established by the Regional Board in Skåne. The effect goals are monitored within the NSM project through an annual oversight report to the steering group. The follow-up refers to how the project enables the goals to be met.

Based on these effect goals and how they have been structured for evaluation, the study at NSM discussed, tested, and translated these goals, whenever possible, into evaluation aspects specifically for the physical environment.

3.1. Research Methods

We used qualitative data collection tools, including semi-structured interviews and focus groups with staff involved in the project, with findings compared to the intended design goals of the development. We interviewed 15 key staff members, managers, and unit managers affected by the project, i.e., people already working in functions relocating into the new building or about to. Three focus group interviews/discussions were also conducted based on the interview data.

The convenience sample of respondents was based on the relevance and accesibility of key persons on the project. The approach is valid for small-scale research projects aimed at mapping an area for development for further research questions (Denscombe, 2020).

It is also important to note that the evaluation of the project was done on the basis of set goals and in relation to other similar projects.

4. Results: What did we learn from the Malmö case

The operational goals that guided the analysis of NSM were divided into five main aspects: i) political directives; ii) patient safety; iii) flows; iv) sustainability; and v) knowledge management and development. As the political directives were not set from an evaluating point of view, this has been omitted in this study. In total, the study touched upon 100 different operational goals, which provided a broad and comprehensive basis for the development of forthcoming evaluation efforts. The objective or functional statements of the factual work environment were easier to evaluate given explicit evaluation criteria, whereas patient flow and safety were more complex to assess from a person-centered perspective (Widmark et al., 2022).

Each aspect or operational goal in the study was found to have several evaluation alternatives, ranging from employee and patient surveys and interviews to pure patient register analyses based primarily on reports found within the healthcare system, care production statistics, and quality registers. In the case of surveys, they were advantageously combined and offered insights about most of the five aspects.

The forthcoming implementation of the evaluation of premises and buildings at NSM is largely going to be based on comparative data before and after the move. This means that there is a need to start part of the evaluation as early as possible after project completion in order for data to be available and accessible, i.e., not be forgotten or lost to memory due to staff changes and departures. Depending on which approach is used for each evaluation area, an overview also needs to be made of how the information is entered and collated in the chosen system of documentation. Several people in their interviews raised the importance of drawing direct conclusions from the documentation systems, as the inputs and definitions of information do not always correspond to the actual situation. Such an analysis can at the same time identify systematic “errors” that can be expected to be consistent, i.e., the measurements before and after moving to the new premises are expected to have an equal margin of error. In that case, they can still be used for comparisons.

We conclude, based on the interviews conducted and the reflections made by the project group during the NSM project, that there are many evaluation approaches possible for the new premises within NSM, depending on what is sought. In this paper, we focus on the concerns of translating operational goals aimed at being enacted on the new premises into an evaluation related to aspects connected to the new facility design and aspects. Although the premises and physical location can be measured and evaluated through the measurement of specific functions and criteria, there is a pressing need to capture and assess the experiences and opinions of those who presently use and experience the premises. Even if the premises meet the required technical requirements, there is a need to additionally evaluate and consider the staff’s experience and what they consider meaningful. This involves both the care staff and patients, including relatives, as well as the administrative and service staff in a hospital environment.

5. Discussion: Key points and challenges

An important aspect to consider in the ongoing conduct of this study with evaluations of hospital premises is the crucial point of systematically introducing evaluations over time after moving to the new facilities. For example, in a hospital project in Kolding, Denmark, conclusions were drawn that a significant number of problems were only discovered five years after operations moved into the new buildings (Godtsygehusbyggeri, 2020). It is conceivable that an initial evaluation reflects a bias in the form of a positive effect based solely on the fact that the buildings are new—the newness effect hinted at above. Only after longitudinal examinations and replicable assessments can in-depth and definitive conclusions be drawn.

The proposal on how the Skåne Region should proceed with the evaluation of NSM as a result of this study is thus primarily to establish transparent and accurate measurement methods for the current situation in the existing premises before moving to the new ones. A review of available systems and an overview of the possible new systems to structure evaluations are needed. A key requirement is the ability to compare data over time (before and after moving) in the most accurate and standardised way possible. A proposal for the inclusion of external researchers for more detailed evaluations of specific impact goals has also been discussed, including, for example, doctoral dissertation projects, bachelor's and master's level theses, etc. If databases are established, privacy issues must be considered, and in cases involving research affecting personal data, ethical permits will be needed.

During the execution of the study, it was proposed to establish a platform for evaluations of hospital premises within the Skåne region. Knowledge gathered from both NSM and other new designs, construction, and conversion of healthcare buildings within the region can be compiled on this platform to serve as a basis for both future projects and the development of evaluation models and approaches. This can also be related to the activities in the Health System of the Future action (Region Skåne, 2022), Skåne's regional process for adapting healthcare to future requirements.

6. Conclusions and research outlook

The study conducted on the NSM project, as a preliminary pilot of assessment and evaluation, is important to consider as an approach to design and action research that argues for the necessity of incorporating robust measurement and evaluation tools of the effects of buildings on the built environment development process. This is particularly relevant for the healthcare sector, where features of the physical built environment can have impacts on several managerial and clinical aspects such as organisational outcomes, occupant wellbeing, clinical results, stress, population health and satisfaction, etc. Such tools are also relevant and needed when addressing the usability aspects of buildings and pushing for a real user-centered approach in the architecture and construction industries. Future research will need to deepen this approach, replicate studies in different contexts, and provide longitudinal protocols for robust measurement and monitoring of improvements in healthcare facilities. The development of structured pre- and post-occupancy measurements and the mapping of processes to facilities will also be needed. Such research outlooks will address the question, “Does it get better knowing the effects?” To the best of our knowledge, the answer to that question is most likely a qualified "yes,"  because it is possible to improve only what you can articulate, describe, and measure.

Acknowledgments

The research has been approached within the framework of the international working seminar “Healthcare Design and Evaluation" organized with the support of Politecnico di Milano DABC “Cultural Activities 2021” – proposer: Dr. Andrea Brambilla, DABC STRATEGIC LINE N° 1: Innovative projects for architecture, spaces and services: health, safety, inclusion, education and emergency.

References

Leaman, A. (2003). Post occupancy evaluation. Gaia Research.Brambilla, A., & Capolongo, S. (2019). Healthy and Sustainable Hospital Evaluation—A Review of POE Tools for Hospital Assessment in an Evidence-Based Design Framework. Buildings, 9(4), 76. https://doi.org/10.3390/buildings9040076Brambilla, A., Lindahl, G., Dell’Ovo, M., & Capolongo, S. (2021). Validation of a multiple criteria tool for healthcare facilities quality evaluation. Facilities, 39(5/6), 434–447. https://doi.org/10.1108/F-06-2020-0070Denscombe, M. (2012). Research proposals: A practical guide: A practical guide. McGraw-Hill Education (UK).Duffy, F. (1990). Measuring building performance. Facilities, 8(5), 17-20.Godtsygehusbyggeri. (2020, November 21). Erfaringar med de nye sygehuse: Enestuer på Sygehus i Lillebælt i Kolding [Experience with the new hospitals: Single wards at Lillebælt Hospital in Kolding]. Godtsygehusbyggeri. https://godtsygehusbyggeri.dk/inspiration/cases/erfaringer-med-de-nye-sygehuse-enestuer-paa-sygehus-lillebaelt-i-koldingLanbeck, P. (2021, September 14). If we invest we should measure. Interview with Head of NSM Project [Personal communication].Leaman, A., Stevenson, F., & Bordass, B. (2010). Building evaluation: Practice and principles. Building Research & Information, 38(5), 564–577. https://doi.org/10.1080/09613218.2010.495217Li, P., Froese, T. M., & Brager, G. (2018). Post-occupancy evaluation: State-of-the-art analysis and state-of-the-practice review. Building and Environment, 133, 187–202. https://doi.org/10.1016/j.buildenv.2018.02.024Macmillan, S. (Ed.). (2004). Designing better buildings: quality and value in the built environment. Taylor & Francis.Preiser, W. F., & Vischer, J. (Eds.). (2005). Assessing building performance. Routledge.Region Skåne. (2022, March 12). The health system of the future [Framtidens hälsosystem]. vardgivare.skane.se. https://vardgivare.skane.se/kompetens-utveckling/projekt-och-utvecklingsarbete/framtidens-halsosystem/Region Skåne. (2021a, August 14). Malmö Sjukhusområde. skane.se. https://www.skane.se/organisation-politik/bygg-fastighetsutveckling/malmo-sjukhusomrade/Region Skåne. (2021b, September 8). Fakta Nya Sjukhusområdet Malmö. Skane.Se. https://www.skane.se/organisation-politik/bygg-fastighetsutveckling/malmo-sjukhusomrade/fakta-om-malmo-sjukhusomrade/Region Skåne. (2021, September 9). Rebuilding and new construction. skane.se. https://www.skane.se/organisation-politik/bygg-fastighetsutveckling/byggprojekt-pa-vara-sjukhusomraden/darfor-byggs-det-om-och-nytt/Schaumann, D., Putievsky Pilosof, N., Gath-Morad, M., & Kalay, Y. E. (2020). Simulating the impact of facility design on operations: A study in an internal medicine ward. Facilities, 38(7/8), 501-522.Vischer, J. C. (2008). Towards a user-centred theory of the built environment. Building Research & Information, 36(3), 231–240. https://doi.org/10.1080/09613210801936472Widmark, J., Lindahl, G., & Lanbeck, P. (2022). Rapport utvärderingsmodeller, Effektmål Nya sjukhuset Malmö (NSM). [Report on evaluation models, Objectives New University Hospital Malmö]. CVA, Chalmers.