Summaries were completed this week of the data logs of the six students from our MV implementation previously identified as potentially interesting cases. Between the short week and the laborious coding of the data logs, we have just begun to generate some descriptive measures. The table below lists the students whose logs have been analyzed, their pre and post test rank, their change in rank between pre and post tests, the total time logged in BioLogica activities, the number of the activities logged, and their calculated level of engagement.
Student pre-test rank post-test rank change in rank total time # activities level of engagement CV 3 21 -18 1:44 11 0.15 MB 1 1 0 5:25 10 0.39 TF 23 20 3 3:44 11 0.40 AA 4 4 0 3:17 12 0.44 GH 20 3 17 4:20 11 0.48 CG 15 9 6 7:22 8 0.86
- test rank is the rank (1 = highest) of the student on the range of pre and post test scores.
- change in rank is pre test rank minus the post test rank.
- total time is the length of time the student used BioLogica, based on the sum of the logs. (This is not an accurate measure as some of the activities did not log accurate start and stop times.)
- # activities indicates how many of the 12 BioLogica activities were logged for that student. (The existence of a log does not guarantee that the activity was completed.)
- level of engagement is calculated by dividing length of time (LOT) for each log by the length of the record (LOR) generated. Mindless clicking and guessing generates long records in a short period of time, while carefully pondering the representations and thinking them through generates a short record in a long period of time. Due to the inconsistencies in data logging, many activities do not have accurate start and end times. Thus, these are as yet very crude measures as we explore ways of describing and quantifying students' interactions with BioLogica.
We cannot draw any conclusions from the analysis done thus far. We can see strong contrasts that require further exploration. For instance, looking at the extremes of the level of engagement, we can ask why there is such a stark difference between CV and CG. Because we were observers during this implementation we know that CV was not engaged in learning but in 'playing' BioLogica as if it were a game. CG, in contrast, spent considerable time reading the text, or having it read aloud, and thinking about the tasks. One of the questions for future research design is seeking measures other than observation that we can monitor or assess as both formative and summative measures of engagement. The formative measure could potentially be used to manipulate the scaffolding of the activities.
Next steps include more detailed analysis of these students' logs and attempts to correlate them with their learning as demonstrated on the pre- and post-tests. In addition, analysis across the students for each activity will be undertaken in order to describe the relevant characteristics of the activities and necessary enhancements to data logging.