How to measure realism in scenic simulations

4 min

A group of researchers develop a validated tool to measure the levels of realism in simulation applicable to any scenario of any medical discipline.

Simulation is one of the fundamental pillars of the academic model of the European University of Madrid. The methodology is based on the recreation of environments with a high level of likelihood with reality.

Based on a three-year mixed study (Delphi method and statistical validation using Cronbach’s Alpha indices, Guttman’s Lambda 6 and correlations), the researchers of the SIMLAB group: Observatory of Didactic, Psychological and Bioethical Effects of Learning in High Realism Simulations, from the Faculty of Health Sciences have developed a tool called ProRealSim v1.0 to obtain global indices – and by dimensions – of realism in simulation.

It is a validated, online, bilingual (English/Spanish), free theoretical and mathematical tool, available on computers and smartphones, that will be very useful in health study programs that apply the methodology, given its versatility to be applied to any simulated scenario of any medical discipline.

Its developers have consulted experts from the main simulation centers in Spain and Latin America. They have mathematically determined the weight of the most significant units and dimensions that express the precision – conceptual or structural realism – and the truthfulness– functional realism – of the simulated participant, the simulator, and the scenography.

Having an objective figure – the average index of realism in simulation evaluated by two or more expert appraisers – will provide a fairer and more verifiable evaluation of a term that until now was quite elusive, complex, and diffuse.

The construct is supported by contributions from Dieckmann, Gaba, Rall (2007), Rudoplh, Reamer, Simon, Hamstra and collaborators (2014), Tun, Alinier, Tang and Kneebone (2015). It focuses the observational perspective on the most objective appraiser – the experienced designer teacher – to obtain consensus and compare the relationships and influences between the indicators and the scores.

In order to overcome the dissent over the term fidelity – given its complexity and lack of tools of this type -, the research team offers the tool for the use of large samples. This will be possible thanks to its wide digital accessibility and the use of objective, measurable, easy-to-interpret, recognizable, frequent, and familiar indicators in most scenarios.

One of the difficulties of any complex measurement system is the intra- and inter-observer disparity on simulated objects, acts, and reactions. Therefore, we emphasize the importance of a new figure of great scientific value, key in the interpretation of these terms: the realism appraiser.

By linking qualitative descriptors to a Likert scale from 1 to 10 -where 1 is very low and 10 is very high-, the appraiser will be able to assess the realism in simulation. Before this, he/she will receive training and coaching with the tool and will be credited by code.

The purpose of the initiative is to form a consolidated community of experts in simulation realism to deepen knowledge, discriminate the subtle differences and complexities of plausibility with strategies that allow to reduce the discrepancies between the evaluators, the observer’s fatigue and the lack of analysis of different contexts based on evidence.

At the same time, the tool seeks to promote a culture of interpretation of simulation realism that allows to understand the impact of the variables under study, to verify the usefulness of the dimensions, units, categories, and indicators created, and to compare the measurements of actual realism with those of realism perceived by the student.

This will help to know the cost/benefit of investments, having the advantage that the elements, assets and products of the simulation are validated by clinical experts, which will enable the analysis of how these metrics impact the design and construction of simulated environments, the learning process, and the transfer of knowledge. Researchers, teachers, technical staff, etc. who are interested in this field of study can contact the SIMLAB team through the feedback tab of the tool and/or by email: gleyvis.coro@universidadeuropea.es