The paper reports and reflects on an evaluation project at Malmö Nya Sjukhus, NSM, Sweden, where the relationship between healthcare performance objectives and indi-cators in the built environment have been studied. The starting point of the study was the outcomes in performance set by the hospital and a retrospective analysis of how these could be related to evaluation of the design of the new hospital. The paper discusses evaluation against a backdrop of existing studies on evaluations and with an argument that that there is a need for tools and methods to strengthen design work in general. The main argument is that it is valuable to evaluate the work processes in healthcare against spatial configurations in addition to studying effects of specific design features. Con-necting healthcare process indicators to spatial design also envision several challenges and possibilities that are addressed in the paper. The study presented is mainly quali-tative with and explorative approach.Show Less
Type of the Paper: Full Paper
Track title: Topic 3: Engagement – co-creation, co-design, design and stakeholder management processes.
Evaluation and understanding of the effects of hospital design – mapping healthcare spatial configuration processes
Andrea Brambilla 1,2*, Göran Lindahl 2,1 , Jens Widmark 2, 6, Paul Barach 4,1, Laura Cambra Rufino3, Peter Lanbeck5, Stefano Capolongo 1
Names of the track editors:
Names of the reviewers:
Journal: The Evolving Scholar
Submitted: 01 January 2021
Accepted: 01 June 2021
Published: 02 June 2021
Citation: name of authors [if more than 3 authors use the name of the 1st author followed by et al. e.g smith et al.], title of the article, name of the journal, volume, year, DOI
This work is licensed under a Creative Commons Attribution xxx (CC xxxx) license.
©year [name of the author(s)] published by TU Delft OPEN on behalf of the authors
1 Design and Health Lab, Department of Architecture, Built Environment and Construction Engineering (DABC), Politecnico di Milano, 20133 Milan, Italy; andrea1.brambilla@polimi ; firstname.lastname@example.org
2 Center for Healthcare Architecture (CVA), Division of Building Design, Department Architecture and Civil Engineering (ACE), Chalmers University of Technology, SE-412 96 Goteborg, Sweden; email@example.com
3 Escuela Tecnica Superior de Arquitectura de Madrid, Universidad Politecnica de Madrid, Spain; firstname.lastname@example.org
4 Jefferson College of Population Health, Thomas Jefferson University, Philadelphia, PA 19107, USA and School of Medicine and Law, Sigmund Freud University, 1020 Vienna, Austria; email@example.com;
5 Skåne University Hospital, Malmö, Sweden; firstname.lastname@example.org
6 Faculty for Health and Society, Dept. of Criminology, Malmö University; email@example.com
The aim of the paper is to report and reflect on an evaluation study of a hospital project in Sweden, where the relationships between healthcare performance objectives and indicators of the built environment were studied. The starting point of the study was the performance outcomes set by the hospital and a retrospective analysis of how these could be related to the design of the new hospital. We used qualitative data collection methods including semi-structured interviews and focus groups and the findings were compared with the intended design goals of the project. The paper discusses evaluation against a backdrop of existing studies on project evaluations and with an argument that there is a pressing need for new tools and methods to strengthen design work. We argue that it is valuable to evaluate the work processes in healthcare against spatial configurations in addition to studying the effects of specific design features. Connecting healthcare process indicators to spatial design plans addresses several challenges and possibilities that are addressed in the paper. The study presented is mainly qualitative with an explorative approach.
Keywords: healthcare built environment; methods development; mapping; healthcare processes; design effects; evaluation
1. Introduction: Built environment and evaluation
Understanding the built environment, and healthcare facilities in particular, is a challenging endeavor. To understand the effects of a particular design or layout on the activities planned to be executed requires robust, reliable methods and approaches (Brambilla & Capolongo, 2019; Li et al., 2018). It is interesting to note that in the field of medicine and to some extent healthcare management, measure-oriented approaches and quality assessment tools are regularly used as the foundation of clinical activities and to foster accountability and support continuous quality improvement of the services delivered (Brambilla et al., 2021). This is however neither done for design and planning of healthcare facilities nor in the evaluation of organizational or clinical outcomes related to facilities. In the history of research on building evaluation we find aspects related to the efficiency of the buildings per se (Preiser & Vischer, 2005), but also initiatives to understand the users and their context to understand what is valuable and meaningful to them (Vischer, 2008). This is an area that spans from quantitative aspects to qualitative and that includes organizational as well as individual objectives. This paper takes as its starting point a study conducted at the New Hospital in Malmö (Nya Sjukhuset I Malmö, NSM) in Sweden and the discussions within the framework of the international working seminar “Healthcare Design and Evaluation" held at Politecnico di Milano . This is an ongoing project where there is an identified need to explore new possibilities to evaluate the new hospital’s healthcare processes against the facilities it uses. The study is based on an evaluation of clinical and organizational objectives and to what extent they can be related to the effects of, or as a consequence of the facilities. To this is added a discussion on the challenges of meaningfully measuring and evaluating healthcare facilities.
2. Background: To measure or not to measure buildings
Research on the efficiency of facilities is not new but has been ongoing since the 1960s in various forms. This effort has focused on the performance of the buildings themselves and the effects of the building components. Methods and processes for documenting the effects of buildings and premises have also been developed (see e.g., Leaman et.al 2010; Preiser, 2005). Especially methods to evaluate buildings once they have been occupied and taken into use. However, what remains to develop are methods for linking business goals to the design of premises that can be included in systematic follow-ups. This is a challenge in healthcare where operations usually change in connection with technological transformations for diagnostics and treatment, evolution of medicine, organizational and leadership changes etc. The impacts of facilities are therefore difficult to describe without systematic support from structured and systematic methods incorporating both clinical processes and aspects of the facilities.
There are methods available for evaluations in a pre-stage, where simulation models are developed. These are generally surrounded by a higher degree of uncertainty and their level of usefulness is yet to be recognized (Schaumann et al., 2020). Once a building is in use a common method is do a Post Occupancy Evaluation (POE). POE is seen as a way to ensure that a building delivers what is actually expected and its consequences in the form of "ends" instead of how this is provided ("means") (Adrian, 2003, p.). Two main issues are proposed to guide the work; "How does the building work?" and "Is this what it intends to do?" (ibid.). The second question here is what most often challenge systematic evaluations as there seldom are robust baselines or pre-study datasets to compare against, nor are evaluation criteria set at the briefing stage. Additionally, as noted above, the intended use may not be exactly what is needed once a facility is completed. This often results in the effect that new is better, the “newness effect”, which usually is true for new buildings in their first phase of use but might not last.
Reliable tools are needed to successfully evaluate buildings, and provide feedback during and after the design and construction process (Macmillan, 2004). Four approaches that are proposed, which can be used both in combination or separately, are i) observations, ii) questionnaires and interviews, iii) informative discussions and physical review, and, iv) statistical tests and analyses of performance. In the chapter “Learning more from what we build” of Macmillan’s book “Designing Better Buildings”, Bill Bordass states that many consider the best results to be produced through a combination of "hard" and "soft" values (Macmillan, 2004). Francis Duffy, in 1990, recommends conducting in-depth measurements by considering that they must be operational through validity and reliability in the measurement process, that is they must be rigorous and practical to carry out (Duffy, 1990). The measurements should be clear to specific stakeholders and the general public as well in such a way that it is clear what is being measured and for what purpose it is being measured; in other words, why it is advantageous to make the specific measurement or evaluations. An evaluation must also be performance-based, which is directly related to individual or organizational outcomes. The measurements should also be comparative to enable information to be stored in relational databases and encourage benchmarking and comparisons both within and between organizations. Furthermore, Duffy suggests that an evaluation should be broad and equally capable of evaluating the intricate details of a specific workstation at a large factory premise (Duffy, 1990).
It is important to also ensure that the evaluations are made in a robust manner, even if the information cannot be validated as in laboratory-controlled experimental studies (Leaman et al., 2010). The results of an evaluation also depend on the circumstances in which the evaluation is made, how the buildings or facilities are operationalized and in what context they are evaluated; nevertheless, the methods adopted for the evaluation should be similar or identical for different settings. Indeed, the ways in which the evaluation is carried out should whenever possible be repeatable, credible and provide reliable results. The key challenge, however, remains: can an organization effectively do what it proclaims, and how and to what extent is its successful operations related to the facilities they use?
3. The case and methods: The case study of Nya Sjukhuset i Malmö Hospital (NSM)
The starting point for this paper is the operational goals developed for the NSM project, a complementing and refurbuising project for a regional hospital, one of two sites of Skåne University Hospital, serving 1.4 millon people. The total size of the healthcare building is 108,000 sqm. The project also includes service buildings, replacement buildings and technical infrastructure. The budget for the project is SEK 12.3 billion for the construction project and SEK 1.9 billion for equipment. The decision to carry out the redevelopment was made by the Regional Council in 2013, with a final decision on budget and scope made in 2017. The initial allocation of SEK 6 billion to be completed between 2014–2020 (Region Skåne, 2021b).
The new healthcare facilities will consist of two interconnected buildings of 10 and 11 floors respectively (Region Skåne, 2021a). The project will add a total of 244 beds and 23 surgical theatres and an intensive care unit of 14 beds. In additon, new service structures and faciltities will be built, e.g. a new culvert system covering the whole hospital, sterile units, additional facilties, a new mortuary, waiting rooms and administrative workplaces. Laboratory facilities will be fully substituted. Planning of the healtcare building was started in 2015 and construction in 2018 and has a planned completion date in 2024-25. The NSM project is based on the necessary adaptation to the evolving healthcare needs and the maintainance difficulties of aging buildings (Region Skåne, 2021). The new facilities needs to be flexible and adaptable to future working methods and care needs.
The project is guided by operative goals that are divided into political goals and the planning principles established by the Regional Board in Skåne. The effect goals are monitored within the NSM project through an annual oversight report to the steering group. The follow-up refers to how the project enables the goals to be met.
Based on these effect goals and how they have been structured for evaluation, the study at NSM discussed, tested and translated these goals, whenever possible, into evaluation aspects specifically for the physical environment.
3.1. Research Methods
We used qualitative data collection tools including semi-structured interviews and focus groups with staff involved in the project, with findings compared to the intended design goals of the development. We interviewed 15 key staff members, managers and unit managers affected by the project, i.e., people already working in functions relocating into the new building or about to. Three focus group interviews/discussions were also conducted, based on the interview data.
The convenience sample of respondents was based on relevance and accesibility of key persons on the project. The approach is valid for small scale research projects aimed to map an area for development for further research questions (Denscombe, 2020).
It is also important to note that the evaluation of the project was done on the basis of set goals and in relation to other similar projects.
4. Results: What did we learn from the Malmö case
The operational goals that guided the analysis of NSM were divided into five main aspects: i) political directives, ii) patient safety, iii) flows, iv) sustainability, and v) knowledge management and development. As the political directives were not set from an evaluating point of view, this has been omitted in this study. In total, the study touched upon 100 different operational goals, which provided a broad and comprehensive basis for the development of forthcoming evaluation efforts. The factual work environment objectives or functional statements were easier to evaluate given explicit evaluation criteria whereas the patient flow and safety from a person centered perspective was more complex to assess (Widmark et al., 2022).
Each aspect/operational goal in the study was found to have several evaluation alternatives, ranging from employee and patient surveys and interviews to pure patient register analyses based primarily on reports found within the healthcare system, care production statistics and quality registers. In the case of surveys, they were advantageously combined and offered insights about most the five aspects.
The forthcoming implementation of the evaluation of premises and buildings at NSM is largely going to be based on comparative data, before and after the move. This means that there is a need to start part of the evaluation as early as possible after project completion, in order for data to be available and accesible, i.e., not being forgotten or lost to memory due to staff changes and departures. Depending on which approach is used for each evaluation area, an overview also needs to be made of how the information is entered and collated in the chosen system of documentation. Several people in their interviews, raised the importance of drawing direct conclusions from the documentation systems as the inputs and definitions of information does not always correspond to the actual situation. Such an analysis can at the same time identify systematic “errors” that can be expected to be consistent, i.e., the measurements before and after moving to the new premises are expected to have an equal margin of error. In that case, they can still be used for comparisons.
We conclude, based on the interviews conducted, and the reflections made by the project group during the NSM project, that there are many evaluation approaches possible for the new premises within NSM depending on what is sought. In this paper, we focus on the concerns of translating operational goals, aimed to be enacted in the new premises, into an evaluation related to aspects connected to new facility design/aspects. Although the premises and physical location can be measured and evaluated through the measurement of specific functions and criteria, there is pressing need to capture and assess the experiences and opinions from those who presently use and experience the premises. Even if the premises meet the required technical requirements, there is a need to additionally evaluate and consider the staff’s experience and what they consider meaningful. For a hospital environment, this involves both the care staff and patients including relatives, but also the administrative and service staffs.
5. Discussion: Key points and challenges
An important aspect to consider in the ongoing conduct of this study with evaluations of hospital premises is the crucial point of systematically introducing evaluations over time after moving to the new facilities. For example, in a hospital project in Kolding, Denmark, conclusions were drawn that a significant number of problems were only discovered five years after operations moved into the new buildings (Godtsygehusbyggeri, 2020). It is conceivable that an initial evaluation reflects a bias in the form of a positive effect based solely on the fact that the buildings are new - the newness effect hinted at above. Only after longitudinal examinations and replicable assessments can in depth and definitive conclusions be drawn.
The proposal on how the Skåne Region should proceed with the evaluation of NSM, as a result of this study, is thus primarily to establish transparent and accurate measurement methods for the current situation in the existing premises, before moving to the new ones. A review of available systems and an overview of the possible new systems to structure evaluations is needed. A key requirement is the ability to compare data over time (before and after moving) in the most accurate and standardized way possible. A proposal for the inclusion of external researchers for more detailed evaluations of specific impact goals has also been discussed including for example doctoral dissertation projects, bachelor's and master's level theses etc. If databases are established, privacy issues must be considered and, in cases involving research affecting personal data, ethical permits will be needed.
During the execution of the study, it was proposed to establish a platform for evaluations of hospital premises within the Skåne region. Knowledge gathered from both NSM and other new designs and construction and conversion of healthcare buildings within the region can be compiled in this platform to serve as a basis for both future projects and development of evaluation models/approaches. This can also be related to the activities in the Health System of the Future action (Region Skåne, 2022), Skåne's regional process for adapting healthcare to future requirements.
6. Conclusions and research outlook
The study conducted on the NSM project, as a preliminary pilot of assessment and evaluation, is important to consider as an approach of design/action research that is arguing for the necessity of incorporating robust measurement and evaluation tools of the effects of buildings on the built environment development process. This is particularly relevant for the healthcare sector where features of the physical built environment can have impacts on several managerial and clinical aspects such as organizational outcomes, occupant wellbeing, clinical results, stress, population health and satisfaction etc. Such tools are also relevant, and needed, when addressing usability aspects of buildings, and when pushing for a real user-centered approach of the architecture and construction industry. Future research will need to deepen this approach, replicate studies in different contexts and provide longitudinal protocols for robust measurement and monitoring of improvements in healthcare facilities. Development of structured pre- and post-occupancy measurements, mapping of processes to facilities will also be needed. Such research outlooks will be addressing the question “Does it get better knowing the effects?” To the best of our knowledge, the answer to that question is most likely a qualified “yes”, because it is possible to improve only what you can articulate, describe, and measure.
The research has been approached within the framework of the international working seminar “Healthcare Design and Evaluation" organized with the support of Politecnico di Milano DABC “Cultural Activities 2021” – proposer: Dr. Andrea Brambilla, DABC STRATEGIC LINE N° 1: Innovative projects for architecture, spaces and services: health, safety, inclusion, education and emergency.
Adrian, L. (2003). Post-occupancy Evaluation. In S. Roaf (Ed.), Benchmarking Sustainable Building Performance. RIBA Publications. https://www.usablebuildings.co.uk/UsableBuildings/Unprotected/AdrianLeamanPost-OccupancyEvaluation.pdf
Brambilla, A., & Capolongo, S. (2019). Healthy and Sustainable Hospital Evaluation—A Review of POE Tools for Hospital Assessment in an Evidence-Based Design Framework. Buildings, 9(4), 76. https://doi.org/10.3390/buildings9040076
Brambilla, A., Lindahl, G., Dell’Ovo, M., & Capolongo, S. (2021). Validation of a multiple criteria tool for healthcare facilities quality evaluation. Facilities, 39(5/6), 434–447. https://doi.org/10.1108/F-06-2020-0070
Denscombe, M. (2020). Research proposals: A practical guide (Second edition). Open University Press.
Duffy, F. (1990). Measuring building performance. Facilities, 8(5). https://www.emerald.com/insight/content/doi/10.1108/EUM0000000002112/full/pdf?title=measuring-building-performance
Godtsygehusbyggeri. (2020, November 21). Erfaringar med de nye sygehuse: Enestuer på Sygehus i Lillebælt i Kolding [Experience with the new hospitals: Single wards at Lillebælt Hospital in Kolding]. Godtsygehusbyggeri. https://godtsygehusbyggeri.dk/inspiration/cases/erfaringer-med-de-nye-sygehuse-enestuer-paa-sygehus-lillebaelt-i-kolding
Lanbeck, P. (2021, September 14). If we invest we should measure. Interview with Head of NSM Project [Personal communication].
Leaman, A., Stevenson, F., & Bordass, B. (2010). Building evaluation: Practice and principles. Building Research & Information, 38(5), 564–577. https://doi.org/10.1080/09613218.2010.495217
Li, P., Froese, T. M., & Brager, G. (2018). Post-occupancy evaluation: State-of-the-art analysis and state-of-the-practice review. Building and Environment, 133, 187–202. https://doi.org/10.1016/j.buildenv.2018.02.024
Macmillan, S. (2004). Designing better buildings quality and value in the built environment. Spon.
Preiser, W. F. E., & Vischer, J. (Eds.). (2005). Assessing building performance. Elsevier.
Region Skåne. (2022, March 12). The health system of the future [Framtidens hälsosystem]. vardgivare.skane.se. https://vardgivare.skane.se/kompetens-utveckling/projekt-och-utvecklingsarbete/framtidens-halsosystem/
Region Skåne. (2021a, August 14). Malmö Sjukhusområde. skane.se. https://www.skane.se/organisation-politik/bygg-fastighetsutveckling/malmo-sjukhusomrade/
Region Skåne. (2021b, September 8). Fakta Nya Sjukhusområdet Malmö. Skane.Se. https://www.skane.se/organisation-politik/bygg-fastighetsutveckling/malmo-sjukhusomrade/fakta-om-malmo-sjukhusomrade/
Region Skåne. (2021, September 9). Rebuilding and new construction. skane.se. https://www.skane.se/organisation-politik/bygg-fastighetsutveckling/byggprojekt-pa-vara-sjukhusomraden/darfor-byggs-det-om-och-nytt/
Schaumann, D., Putievsky Pilosof, N., Gath-Morad, M., & Kalay, Y. E. (2020). Simulating the impact of facility design on operations: A study in an internal medicine ward. Facilities, ahead-of-print(ahead-of-print). https://doi.org/10.1108/F-10-2018-0132
Vischer, J. C. (2008). Towards a user-centred theory of the built environment. Building Research & Information, 36(3), 231–240. https://doi.org/10.1080/09613210801936472
Widmark, J., Lindahl, G., & Lanbeck, P. (2022). Rapport utvärderingsmodeller, Effektmål Nya sjukhuset Malmö (NSM). [Report on evaluation models, Objectives New University Hospital Malmö]. CVA, Chalmers.
No comments published yet.