The goal of this intellectual output is to validate the effectiveness of the products developed in the first three intellectual outputs. For this purpose, we have applied different instruments for assessing the awareness of students of the importance of testing, as well as their knowledge of testing. The instruments are used in the courses of our pilot study in two phases, namely the baseline phase (before we started using out outputs) and the validation phase (after applying our outputs). We have developed three different instruments for this purpose: a rubric, a questionnaire and diagnostic assignments.
Thanks to the questionnaire and the diagnostic assignment, we can check whether there are some differences between students' self-assessment and a more objective assessment of their quality awareness. The rubric allows us to observe the evolution of the students throughout the course and we can check when there are changes in the behaviour of the students (e.g. improvement in the quality of their code).
- Rubric: it allows teachers from the different courses which are involved in the pilots to assess each assignment submitted by the students in terms of quality. It consists of 10 items (e.g., modularity, documentation, etc.) in a 4-Likert scale. All partners used this rubric in the baseline stage and are applying it during the validation stage. Thanks to this rubric, all students that participate in the project are assessed in the same way. To collect data in a homogenous format, the we have developed a web-based version of the rubric.
- Questionnaire: it allows us to know the perception of the students regarding the importance of quality issues when they programming. The questionnaire was the same for all the courses and was sent at the end of the term.
- Diagnostic assignments: the goal of such assignments is to observe which is the actual awareness and knowledge of quality issues of the students. Because each course is different, each lecturer in charge decided the questions and the format of her diagnostic assignment in detail, but all these assignments have a similar form and intention.
Data collected from the baseline stage
Currently, we have collected all the data related to the baseline stage from all the partners. We are now working on cleaning them up, and we are about to start analyzing them. The results and the subsequent discussion and analysis will be a future output of IO4.