Usability Test Results

In the first condition, i.e. the task that was given to the first group, 9 mistakes were made overall. It took a minimum of 21 steps to complete the task. In total, no mistakes were made by any participant regarding class manipulation. Only one participant failed to correctly manipulate a connector – this participant created an aggregation instead of a generalization. The majority of 8 mistakes were done during label manipulation, with similar likelihood in every participant. These errors were exclusively errors in assignment of correct labels (e.g. two labels “is” and “Automatic” instead of one label “isAutomatic”). Each student made an average of 1.5 mistakes during label manipulation. In the second condition, i.e. the task that was given to the second group, a total of 7 steps was minimally necessary to complete the task. A total of 16 mistakes were made, which corresponds to 3.2 mistakes per participant. Only two students incorrectly manipulated a class symbol, and a total of 5 mistakes were made during label manipulation. During connector manipulation, a total of 9 mistakes were made. The qualitative post-test survey shows that the majority of the students ranked the overall quality of SketchUML as “medium to high”, which corresponds to a score of 4 on the 5-point Likert scale that was employed in the survey questionnaire. Open questions regarding the participants' subjective opinion of the program (their likes and dislikes) and their ideas for improvement uniformly contained criticism on SketchUML's mode of feedback. Students mainly complained about difficulties to correctly spell names of attributes and operations. Positive feedback comprised the ease of creating connectors and class symbols. All students independently reported a low frustration level (despite the fact that they were not all able to complete the task correctly) and a high factor of enjoyment.