The most significant direction to pursue in predicting students’ ability to learn computer programming skills seems to come from the area of assessing problem solving ability. However, not every logical skill or ability is an equally reliable predictor. The pre-study survey included 12 questions related to logical problem solving and critical thinking ability. These tests can be grouped into a number of categories:
Gaming Problems |
These are questions in which the thinker is asked to calculate the probability or optimum cost or benefit from a course of action or to be consistent in their calculation of this benefit. |
Problems 1, 3, 5, and 8 |
Decision Trees |
The thinker constructs and works through a decision tree that allows the participant to pose the necessary questions to determine the truth of the entire network. These are the Boolean paradigms discussed by Goodwin and Johnson-Laird (2010) or the disjunctive problems identified by Toplak and Stanovich (2002). |
Problems 2, 7, and 10 |
Rule Based Deduction |
The thinker applies formal inferential logic, rule based analysis or what Newstead et al. call “Analytical Reasoning” (2006), or basic algebra to solve the problem. |
Problems 4, 6 and 12 |
Problem Modeling |
The thinker takes the problem text and envisioning the problem in such a way that a mental model or schema forms that allows them to envision the problem in a new way. They then find the solution almost trivial to solve. |
Problems 9 and 11 |
Among these groups, gaming problems showed no correlation with academic success leading to the conclusion that this is not a type of critical thinking useful to students learning to write computer software.
Decision tree problems show more promise. These problems require the thinker to pursue multiple independent analyses of the problem and then see if there is any commonality in the conclusions. For example, the knight vs. knave problem requires the thinker to begin by allowing inhabitant A to be both a knight and a knave (Figure 2). Only by pursuing both decision branches and determining that they both share the same result is a conclusion to the problem possible. This problem was a significant predictor of academic success.
Problems 2 and 10 are also examples of this type of problem but did not show statistical significance. These problems were replicated from Toplak and Stanovich’s study in which the two deductive conclusions were presented but participants were also given the option to claim that no solution was possible (2002). This option was not presented in Problem 7. As a result, 91% of participants chose this option when confronted with problem 2 and 74% chose it as their solution to problem 10. Stanovich, Toplak and West have since referred to this behaviour as “cognitive miserliness,” the phenomenon that when solving a problem becomes “expensive” in terms of cognitive effort, many people give up rather than work out the possible conclusions (2008).
If this option wasn’t presented in these questions, it is likely that participants would have felt forced to work through the problem to the point where they could decide upon a conclusion instead of choosing the cognitively miserly option.
This group of problems is solved by applying the provided or implied rules to arrive at a solution. Their challenge often stems from the thinker’s ability to parse the semantics of the problem, thereby isolating the relevant facts or rules. For example, in problem 12, the floor allocation problem, the thinker would go through each set of rules against the proposed options and discard an option when and if a rule is found to be violated. Similarly problem 4 can be solved using simple algebra but does involve a two part thought process similar to the disjunctive problems. The card problems in question 6 use the rules of logical inference to identify which cards to turn over however these rules may not be as well known to first year students as the rules of algebra. Problem 12 was strongly correlated with academic performance (p=0.021), and problem 4 had a slightly weaker correlation with p=0.03; problem 6 had a much weaker correlation (0.273, p=0.056).
This group of problems provides insufficient rules or information for the thinker to deduce the solution directly. As a result, the thinker must create a mental schema or model which simplifies the problem before solving it. For example, the thinker must start with a mental picture of a pond completely full of lilies (day 48) and then work backwards to understand that it will be half-full one day earlier. This sort of cognitive hinting is not present in the problem text but is the only way to solve this problem. Similarly, solving the widget problem using algebra is too intensive for many people; the most efficient way to solve this problem is for the thinker to picture a group of five machines each turning out a widget every five minutes, and then realizing that each of a hundred machines would still take five minutes to turn out a widget. Both these problems were very significant (p<0.01) in their correlation with academic performance, indicating that this sort of thinking is critical to success in learning computer programming.
Links
[1] https://niedermayer.ca/user/login?destination=node/117%23comment-form
[2] https://niedermayer.ca/user/login?destination=node/118%23comment-form
[3] https://niedermayer.ca/user/login?destination=node/119%23comment-form
[4] https://niedermayer.ca/user/login?destination=node/120%23comment-form
[5] https://niedermayer.ca/user/login?destination=node/121%23comment-form