Data Teach Data Teach
Its Friday afternoon and I spent most of today combing over a combination of Google spreadsheets and Khan Academy. I met with one of the teachers I work with, and his BTSA coach. We determined that it was time to complete our cycle of inquiry, look at the data, and determine next steps. This process revealed some interesting things in relationship to the current question of how teachers and school staff engage with, interpret, digest, and respond to the ubiquitous numbers that are both inputs and outputs of the students we serve.
In early November we determined that we wanted to see if of our most struggling, and those who complete the least amount of homework, could both increase the amount of time that they spend doing homework and see if that ultimately changed their scores on the end of unit test. The first of these data points has a direct relationship with the intervention. Meaning that we could see an immediate change in behavior by tracking how much time students spend on Khan Academy and contrasting this with whether they had turned in homework in the past. The main thing that changed, besides simply using Khan Academy, we wanted to ensure that students were able to work on the math at home, so we assigned them basic skills instead of class content. The second data point, scores on the end of unit tests, has a less direct relationship and is more inferential. Our theory being, if students spend more time doing math at home and if students work to build their basic skills, their overall math skills will improve.
As we started to gather this information we saw some interesting things in the data and this spurred us to answer some specific questions. We noticed that some students who spent a lot of time on Khan Academy doing homework improved their Unit quiz scores and some did not, about 50-50. We began calculating how much time students where spending on Khan Academy. This made us curious about how time is calculated by their interface. After a short experiment we determined that it calculates the amount of time that a student is on a particular question, but it has no way of knowing if the student is working, thinking. Since we all know how savvy high schooler’s can be we know that the very possible scenario can be: the student logs in and completes a problem, then opens facebook and youtube and catches up on social events for 30 minutes, then goes back to Khan Academy and complete another problem. This will report to the teacher as 30 minutes of work. It looks like this:
So, what have we learned?
It is important to drill down and compare the amount of time a student spend on a specific skill, with the number of problems attempted in order to determine if the time was well spent. For example, when compared to the image above, this student looks as though she has a balanced diet of skill consumption. However, upon closer inspection, she spent her time in the following ways.
Recognizing Fractions – Got Proficient
17 Minutes total
30 total problems
25 correct without hint
Identifying the slope of a line
2 correct without hint
This paints a very different picture of how her time was spent and where the teacher needs to intervene and provide instruction. We then compared these results to her last 3 unit tests and specifically the standards that related to Identifying Slope, Solving 1 Step Equations, and the Distributive Property. This again revealed exactly how shaky her understanding is as her performance fluctuated, but always remained below proficient.
This deeper data dive was made possible because we are using a tool that can track all this information (Khan Academy), because the teacher keeps amazing records of student’s growth over time on specific standards, because we had the time, and because we had a focus.
Data is an integral part of the teaching and learning process. Indeed, there is no way for a teacher to know whether his or her instruction is effective without any data. Yet, because of this it can be very overwhelming. For teachers entering the field, and even those who have taught for some time, it is not always evident which data to look at, or how to best collect the data. Even though the field has emphasized data based instruction for some time, many credentialing programs are not adequately training young professional with these skills (this is opinion based – not sure if there is research to support my hunch).
So we all must collect data and teach and collect more data and teach-teachers to collect data, and teach.