3. Results
I wanted to investigate whether or not student performance on the assignments depended on which software package they used. Based on the printed output they turned in with their assignments I could easily tell which package students used and recorded their grades in separate columns of my (Excel) grade book. However, I graded the assignments in a random order, i.e., I didn't sort them into two separate groups prior to grading. The mean scores for the two groups of students are contained in Table 1 below, where each assignment was worth a total of 20 points.
The third row of the table shows that as the semester progressed about one-sixth of the Excel users switched to Minitab, under the impression that it was not only faster and easier to work in Minitab, but that their grade would also improve. However, the first row of the table reveals that mean student scores tended to differ by a single point or less, except for the first assignment.
Table 1: Mean scores among Mintab and Excel users, by assignment, along with 2-sample tests for differences in the mean (at the alpha level of 0.05).
|
Assignment. 1 |
Assignment. 2 |
Assignment. 3 |
Assignment. 4 |
Assignment. 5 |
| Mtb |
Excel |
Mtb |
Excel |
Mtb |
Excel |
Mtb |
Excel |
Mtb |
Excel |
| Sample Mean |
16.65 |
13.10 |
16.84 |
15.97 |
18.36 |
17.58 |
14.71 |
15.00 |
17.22 |
16.14 |
| Sample Variance |
5.87 |
26.60 |
7.79 |
7.97 |
3.38 |
5.88 |
7.62 |
13.39 |
6.97 |
12.48 |
| Number of Students |
42 |
35 |
46 |
33 |
45 |
31 |
45 |
34 |
49 |
29 |
| Hypothesized Mean Diff. |
0 |
0 |
0 |
0 |
0 |
| Degrees of Freedom |
46 |
69 |
53 |
59 |
47 |
| t Statistic |
3.748 |
1.353 |
1.505 |
-0.385 |
1.436 |
| P-Value |
0.0005 |
0.1804 |
0.1382 |
0.7017 |
0.1576 |
| t critical value (2-tail) |
2.013 |
1.995 |
2.006 |
2.001 |
2.012 |
The bottom half of Table 1 shows the results of doing 2-sample tests for no difference in the mean scores received by Minitab and Excel users. On assignments 2-5, there was no significant difference at the alpha level of 0.05 in the mean scores of Minitab and Excel users. The first assignment was difficult for Excel users because it required them to construct and accurately interpret histograms, which is definitely a bit tricky on one's first attempt. By the second assignment, which was similar to the first, Excel users had caught on and essentially pulled even with Minitab users.
While I did try to grade both groups of assignments as consistently as possible, the experiment conducted here was neither double nor single blind. Students were allowed to pick their software package, so the results could reflect some kind of self-selection bias, perhaps with the more grade-conscious students electing to use Minitab. However, I know that there were both strong and weak students in each group. Overall, I feel the data indicate that there really was little difference in the mean scores of the two groups of students, except for the first assignment.
Perhaps the more telling statistic is that the variance of the Excel scores on each assignment was higher than that of the Minitab users, reflecting the greater degree of difficulty in getting adjusted to the Excel environment, e.g., accurately reading the variable axis of a histogram. In many settings Excel requires more effort from the user to get the same results easily achieved with Minitab; apparently, some Excel users never quite made the required effort.
|
