I first clued into the power of what simple data can do to a class my first year teaching. One day I tallied up the scores to a test and wrote onto the whiteboard how many students received an: A, B, C, D, F or below. This suddenly took each section of science I was teaching to near silence. This was uncommon in my class that first year teaching! Immediately following were questions from many students as they wanted to know why some achieved better than others, what a good or bad test grade would mean for their overall class grade and so on. Also insults and teasing were flung into the classroom environment that took aim at either the high achievers or the low achievers. 

I wanted to provide some simple feedback on the test just as I had seen my high school teachers and college professors do from time to time. What I didn’t expect is such great focus in that very simple data. I didn’t expect all the questions and curiosity and I certainly didn’t expect the vicious undercurrent between my students to rear its ugly head at that moment. It was a lightning rod for attention and I could see that, but I didn’t know what to do about it or how to sculpt it to  my benefit.

Over the next several years I continued to present data in a variety of ways and tried hard to capture that energy, harness it, and use it to my advantage. The result shaped my assessments and I created the simple graphs (found beginning on page 56), along with a keen sense about how I wanted to lead conversations about this information.

Those years have led me to believe a few things about data:

• Data needs to be in the hands of the student in a simple and clear way that they can understand.
• There is great power in holding class and individual conversations about their data. 
• Students will universally and naturally experience important self-reflection when presented with data.
• If I create the right class environment students will be able to handle the data with maturity.
• Student’s are hungry for accurate and impartial information about their progress.
• Students want the teacher on their side to work with them.

Data Disappointment 

Too often teachers are directed toward standardized testing and told that this will be their primary source of data. Then teachers are incorrectly told that this is the path towards success in the classroom. The pressure to improve is on and we just might be starting with the wrong information.

Standardized testing results are too infrequent and difficult to decode. Testing once or twice per year doesn’t generate enough data points to drive instruction in a ten month long course where students’ understanding is changing rapidly. Furthermore, the power of the data is instantly diluted in the process of decoding. A student’s score on a standardized test must be dissected to fit into the existing class curriculum, which is often tedious work.

One example of this comes from a standardized test my current school is using. This test gives me a single number about a student in math then gives me about five subcategories such as, “Numbers: Classify, Compare and Represent.” Then, each subcategory has more subcategories. This time I find a category named “calculate with rational numbers.” Each of these lowest level categories comes in a range of scores such as 300-325. I can look up my student’s number and find their corresponding range on the testing services “A la Carte” sheet to find a list of skills of what someone with this number has likely mastered, is likely ready to learn, and what is likely next for that student. In this case, I found a skill simply named,  “Uses estimation to solve problems involving fractions.” Confused? Well, we are not yet half way done decoding.

When commenting on others’ data be sensitive and constructive. Hold back when you think you’ve begun to overwhelm your peers and be sure to always point out successes. The processes of providing peers feedback is quite similar in these respects to giving feedback to students. When you strike the right chord with a peer you will likely see them return again and again. Above all else, don’t stop sharing your work.

I now need to dig even deeper and see if I can make sense of my student’s score. My school’s chosen standardized test doesn’t provide problem examples nor are the teachers permitted to examine copies of the online test so some detective work is required. Were the fractions all single digit? Were the operations addition, subtraction? Division and multiplication? And so on.  

Once I’ve determined all of these factors I now need to match what I’ve learned about this student and the standardized test and match it to my curriculum. Currently “estimation to solve problems using fractions” happens to be spread across two different chapters and about five different lessons. This means the score’s connection to my curriculum highly diluted at best. It also means that the single number that I fleshed out from my student’s score applies to a good sized chunk of my course, and the information is old by the time I reach these lessons. 

Given that so much work must be done to obtain so little benefit it’s easy to see why teachers are resistant to using data. Teachers are right after all; this diluted and stale information that is only generally connected to their teaching is the wrong choice. The more local, fresh and specifically tied to what’s happening in the class, the more powerful the information will be.

Data Overview

There are three main phases to the use of data; the first being the pre-assessment, disaggregated data, and goal making. The second being the teaching and learning. Finally, the third being the post-test and disaggregated data to reflect on progress and goal achievement.

The first phase of the system begins with the teacher administering a pretest, then the teacher grades and graphs the data that comes from that pretest. The data collected is then presented to the class for a ‘data conversation’ and an outlining of the strategies that the teacher intends to employ throughout the unit represented by the pretest. The data is posted in a visible location in the room for student’s perusal. Then goals are finally made by the students given the information they now have about themselves and the group. 

The second phase has the students engaged in their learning in a new way throughout the unit as taught by the teacher. While the teaching style may be no different as a result of using this data, both the teacher and the student know where they are and what learning needs to occur in order for success to happen. This changes how the student and teacher interact with each other as a group or one-on-one. The data remains alive during the unit with frequent references and use of pretest data. 

The third and final phase comes with the post-test or summative assessment. After the assessment is completed the teacher once again grades the assessment and disaggregates the data and creates graphs representing the data. Yet again, a ‘data conversation’ is held with the class (with both the pre and post-test graphs used here) where both the teacher and the students put forth ideas as to what they did well and what they can improve on into the next unit. It is important to emphasize that the teacher also will offer up ideas to the class on what they personally think they did well and didn’t do well. This humility is critical to the kind of class environment that is needed to make the data successful. Furthermore, the conversation between the student and the teacher is heightened dramatically. Where previously a conversation about a ‘C’ grade on a post-test may have not shed any light on what the causes were for that grade, the teacher and student now have very specific products that are easily comparable from which a substantive conversation can take place much more easily. 

Data and Graphs

When a pre or post assessment is completed by a class the teacher will first grade that assessment as they would always have done. Then the teacher will take the stack of assessments as the raw data. To disaggregate the data the teacher will need to select a few different aspects to asses. The most common and usable trends would be to look at the overall scores by the students and the correct responses on individual problems. Once the aspects have been chosen the data will need to be tallied up. Using total score as an example the student’s total score will all be tallied and then entered into a spreadsheet program such as Excel or Numbers where a bar graph can easily be made. 

Assessments

The bookends of this system are the pre and posttests. These two snapshots of the students serve as a way for both the teacher and the student to quantify what the individual can currently do at the beginning and at the end. That also makes the growth quantifiable to both the student and the teacher. This works when the pre and posttests are nearly identical. Between the two assessments the number of problems and questions should be the same. The skills tested should fall in the same order (e.g. number eight in the pretest should be the same skill tested as in the posttest). Finally the difficulty of the tests and their actual questions should be as similar as possible. 

Traditionally the teacher introduces a new unit with no pretest or pre-assessment issued. The teacher then teaches lessons and the students do the assigned work. Finally a summative assessment or posttest is given. The student evaluates his or her success with a letter grade, a percent, or a fraction written on the page. This situation does not show True Growth, success, or failure. Possibly a general sense of growth can be assumed with a good grade, but it does not show quantifiably by objective or skill the level and extent of the student’s growth. 

When using pre and posttests students can look over their pre and post assessments and put them side by side to compare and contrast. They can visually see their own growth. It is here a tremendous experience can begin to take place. 

The following real quotes were recorded in January 2010 with some students who where now used to my system. They were asked about taking both pre and posttests.

• “I like this because I can see my actual improvement.”
• “I’m relieved when I see that I learned.”
• “I see where I could have done better.”
• “It’s was real hard in the beginning (the pretest) and now that I know how to do it. It (the posttest) was easy.”
• “I don’t like it because I already hated tests anyway, but a good thing is that I know how to work on the next chapter so I don’t get F’s like I did last year.”

To design your assessments you need to ask yourself: “How do I write an assessment for my skill that will inform me and inform the student as to their progress?”, “Does the assessment clearly match up with what they have been working on?”, and “Will that student be able to draw a relationship between what he or she is studying and this measurement of progress?”

In the next two pages you will find a graphic and an example that show how I design my assessments around suggestions from Marzano’s book, Classroom Instruction That Works. In the book he suggests that a rubric be set up to evaluate work with points ranging from one to four. The lowest score being one, three is ‘completing the standard’ and four is ‘exceeding the taught standard.’  

I followed this concept because I thought about how I like the idea of the test conveying different levels of difficulty. To do this I simply gave each standard or skill that I was testing it’s own horizontal row. Four questions are in the row and from the left to right the questions get increasingly more difficult.  The fourth question is beyond what I taught in class. It is not numbered with the rest of the test and is not graphed on my pre and posttest graphs as the most difficult questions don’t reflect what we worked on in class. 

When the pre or posttest is returned we will often fold the paper by its columns to see what level one, two, three or four questions a student got correct. Quickly and visually a student can look at how many from each column they are getting correct.

Graphing Examples

I still make my graphs by first using hand created charts to put in tally marks. Then I transfer that information to a spreadsheet program. The entire process takes me less than 30 minutes.

Below are examples from my middle school math class where bar graphs represent several sections of the class, with 55 total students.

Graphs one and two reflect total score pre and posttest. Graphs three and four reflect correct responses to the specific problems. The data is real class data from my seventh grade math classes in the fall of 2010 at the American International School of Vienna, Vienna Austria. 

Data Conversations

The graphs provide a powerful platform to have conversations about what is going on in your classroom. The conversations can be with an individual student about his or her progress, with the whole class, fellow teachers, and with parents. The communication is clearer, more transparent, and more fruitful when working with any of these groups because the evidence is right in front of you about the argument that you will make. 

With students the interest level will genuinely be high. Students are naturally curious about where they fall in the class and school grade with their progress and they are curious about who the ‘smart’ and ‘not so smart’ kids are in class. Without data graphed like this students will make up their own minds about who those people are and apply their ideas to the social culture of the room. Adding these graphs to a classroom environment doesn’t begin this behavior, rather it brings it to the surface. With good leadership from the teacher the graphs can deal with the class culture and to shape how students think about their own and their peers’ achievement. It is important to say that no student is ever singled out, good or bad, rather a range of achievement is acknowledged and more importantly, all members of the class are learners who can achieve. 

Student Class Conversations

When leading class conversations the experiences and terminology from team building come in very useful to the teacher. Using the data as a calling for all students to play a role in their own learning and the learning of their neighbors flat out works. The teacher needs to come off as genuine, humble, honest, and someone who is acting in the student’s best interest. As much as we would all like to believe that it’s evident to each of our students, it’s a tough sell to middle school kids. The idea that we are acting in their best interest needs to be constantly reinforced. 

Linking Team Building With Data Graphs 

I lead my data conversations with these components almost every time:

1. “We achieve as a team, not at individuals” – this relates to team building clearly, but to sell it as a calling to achievement in the class. I suggest showing the overall score graph and suggesting that it’s everyone’s job (including the teachers) to help push the entire bell curve along. Give an impassioned speech that the individual ‘A’ grades in class mean nothing if there are ‘F’ grades in class. I suggest that small groups that have been working together for the entire unit where there are two ‘A’ grades are and also D and F grades would be a failed group (don’t use a specific example, let them figure it out). Tell them that those ‘A’ students in the group have let their teammates down as much as the teacher has. Liken it to a team building activity where one person didn’t accomplish the goal that could only have been done through teamwork. Lean into this theme as much as you think is useful.
2. “Everyone is Making Growth” On the overall score pre and posttest graph it can be seen that even if not everyone scored that well everyone did learn and that everyone is capable of learning in your class. This message is directed at the lower achievers who need to feel some sort of genuine ‘win’ with the data, no matter how small. This can help give momentum to students who really need it.
3. “Number ___ is my fault and I am sorry.” Pick a number on the ‘by problem’ posttest graph that you think you didn’t connect well with the students or you should have spent more time on. Then tell the students just that. You are at fault because you could have done better in some way and you hope to improve that into the future. The value here in showing humility in tremendous.  When you honestly convey humility you flatten the invisible, rigid hierarchical pecking order that students have created in their own minds. You are, in effect, saying that their hierarchy partially based on who is making mistakes is invalid and that making mistakes is okay. They can see clearly in impartial data that you as the teacher have made a mistake and you are okay with that. Furthermore, you will work towards improvement. The attitude towards their peers’ mistakes will begin to quickly thaw.
4. “What comments or suggestions do you have for me?” If you are going to stand up at the front and call these young or adolescent students to task you will need to allow them to do the same to you, without retribution. If students actually feel safe enough to provide input then you will receive feedback that is constructive. To be honest, some of it will be silly, insensitive, rude, or even flat-out wrong. But you will need to keep your emotions in check, smile, laugh with the students and remember that all day long they are being told what to do. They are not adults and don’t understand the finer points of constructive dialogue and you are here to help them learn that as well. If no suggestions come up I suggest that you be your own worst critic out loud, yet again. Continue to show students your humble side and your own desire for genuine improvement in your class. 
5. “I’m truly surprised at how much you have learned!” I like this statement when I show the ‘By Problem’ pretest bar graph, followed immediately by the posttest bar graph. The change is usually dramatic and stuns the students each time. It’s an instant feeling of group satisfaction and pride. Likely, you will be surprised right along with the students the first time you graph a pre and posttest and see the amount of measurable improvement. Let them know that and let them see you satisfied with their effort and achievement.
6. “Number ___ is unacceptable and we need to work on this.”  Likely they see this as well so avoid dwelling on it. If you know what caused failure to happen tell them what you honestly think occurred and what you believe you all can do about it. Don’t forget yourself, assign yourself a role in that improvement even if it is nothing they will enjoy. I try to save this for the end of the conversation if it makes sense. Try to allow students to enjoy what went well, to absorb what you have to say about yourself as the teacher, then begin to lean into the class about their failure in obligation to their own learning and that of their neighbors as a team of learners. 

With these ideas and some improvisation a good ‘data conversation’ will take place. Now students are ready to make or reflect on goals which will be covered in the next chapter.

Parent Conversations

Just as data can transform your conversations with students, it too will transform what your interactions with parents can be. Armed with simple, clear and useful graphs you can point directly to any student and his or her achievement. Graphs can also quickly show how a particular student might need more attention.  

Showing unequivocal growth from pre to posttests will always make parents a little less on edge. As we all know, sometimes parents need to be reassured that their student is learning in class regardless of their struggles or successes. A genuine way to show that might be using the pre and posttests to put their child’s individual progress into context as it relates to their peers. 

Showing little growth of a student in respect to their peers may do far more than your words to get a parent to understand the critical nature of what is not going right with their child’s learning. Showing this also requires you to bring solutions to the meeting because simply highlighting a problem to a parent is not good enough. 

What I don’t recommend is sending these graphs home in any form. Without context too much could be confused and too little actual good could come from it. 

With clear and impartial data, focus is easy to have and maintain in a meeting about the student’s success. It can change the common attitudes that parents are known for: ‘Parent vs. Teacher’ or ‘Not my Child’ into a team attitude where everyone understands what is going on and what is being done to deal with it. 

Teacher-to-Teacher Conversations

Peer conversations may be the trickiest of them all, but once a group of teachers gets used to making and creating their own data a higher level of teacher teamwork can take place. This type of interaction has been written about in many books dealing with data and is far from a new idea. However, vertical tracking with students is rarely seen.

Teacher-to-teacher conversations about practice and the results they are observing flow easily when the teachers work well together. However, most teacher to teacher conversations don’t add to teacher effectiveness. Many of these conversations occur in hallways, teacher lounges, or even prescribed meetings. These formal or informal conversations rely on a code of collegiality and the understanding that no one pokes around in each other’s classes. This means those in conversations will not actually know what is going on each other’s classes.  

This is a fundamentally broken approach and a difficult culture to break. To break the cycle, team building activities for teachers are needed, just as they need to begin for a new class of students. Then, when some trust has been fostered it needs to be maintained and expanded with meaningful interactions where class data is used to discuss practice or a particular student. When discussing a particular student the grade tracker can be quite useful. You and your peers can look over all of this information and provide feedback. Or perhaps you can provide suggestions using your data as evidence your successes. 

As with student groups in the moments after some team building has occurred, the teacher groups will be eager to see if this new and positive group identity will last or if older habits will prevail, so timing is important. If you wish to foster good conversation with your peers you may need to first present your data to a peer or group of peers and ask them for advice. Be humble and open to suggestion as you did with your students. Listen to what your peers have to say and consider what you can actually use that they are sharing with you. After you feel a positive dialogue has begun, offer to help them take existing and upcoming assessments and to turn them into the pre and post data that you have. This may be a big hurdle, but once overcome they will be on the right path to more fruitful teamwork.

Another way to work with peers is to share vertically across time. For example, you teach science one year to the entire sixth grade. The following year, you will have a full year’s set of data to give to and discuss with the seventh grade teacher about these same students you already had. Then that same group’s 8th grade year could yield two full years of observations and greater solutions can be created. 

This is a tricky conversation to have regardless of what professionals meet to discuss class data. So much of our time as teachers is spent in our classrooms isolated from peers that when the time comes to work together, evaluate, or assist at a deep level, we are often hesitant or afraid of how we may be judged by our own peers. To combat this feeling long before groups ever sit down to look at each other’s data and manor of practice, team building activities with the faculty should take place to help create common language, understanding and trust for one another. Peers understanding and trusting each other is the essential ingredient.