Responses to comprehension questions and verbal protocols as measures of computer program comprehension processes

ISSN： 1362-3001

Source： Behaviour and Information Technology, Vol.16, Iss.6, 1997-11, pp. : 320-336

Disclaimer: Any content in publications that violate the sovereignty, the constitution or regulations of the PRC is not accepted or approved by CNPIEC.

Previous Menu Next

Abstract

To study cognitive processes, such as computer program comprehension, researchers often use verbal protocols to collect a process trace. However, the difficulty of collecting and analysing verbal protocol data can discourage even the most resolute researcher. Therefore, alternatives to verbal protocols, such as responses to comprehension questions, are undeniably attractive. Unfortunately, there is little methodological research to justify the use of most alternative methods. The current study compares the use of verbal protocol data with responses to comprehension questions as measures of comprehension process. According to results from the protocol analysis data, programmers used significantly different comprehension processes to understand computer programs in two phases of an experiment. If previous research was correct, then programmers' responses to different types of comprehension questions should reflect the differences in comprehension process. Unfortunately, comprehension process was not reflected in the responses to the questions. Hence, this research confirms that process tracing methods, such as verbal protocols, are a more appropriate method by which to investigate program comprehension processes.