Developing criteria of successful processes in co-creative research. A formative evaluation scheme for climate services

Abstract

Climate change and its socio-ecological impacts affect all sectors of society. To tackle the multiple risks of climate change the field of climate services evolved during the last decades. In this scientific field products to be applied in practice are developed in constant interaction between climate service providers and users. To judge the effectiveness of these co-creation endeavours, evaluation is crucial. At present, output and outcome assessments are conducted occasionally in this research field. However, the summative evaluation does not help to adjust the ongoing process of co-creation. Thus, our work focuses on the formative evaluation of co-creative development of science-based climate service products. As the first step, main characteristics of the product development process were identified empirically. Secondly, we determined the six sub- processes of climate service product development and related process steps. Thirdly, we selected the questions for the formative evaluation relevant to all the sub-processes and process-steps. Then, a literature review delivered the theoretical background for further work and revealed further quality aspects. These aspects from literature were brought together with our results from the empirical work. In the end, we created a new scheme of quality criteria and related assessment questions for the different sub-processes in climate services, based on both, empirical and theoretical work. As the authors take into account the process of co-production in a real-life case, the criteria and assessment questions proposed are operational and hands-on. The quality aspects refer to the five principles of applicability, theoretical and empirical foundation, professionalism, transparency of processes and the disclosure of preconditions. They are elaborated comprehensively in our study. The resulting formative evalu- ation scheme is novel in climate service science and practice and useful in improving the co-creation processes in climate services and beyond.