homogeneity and class size

I was speaking with a fellow educator the other day about class sizes and homogeneity. Some thoughts I shared with him....

Class size reduction is an area of thought close to my heart. From my summers working with smaller numbers I have seen that it can have a great effect on my ability to keep in touch with each individual student. Recent studies on the effect of class size reduction do seem to bear this out. In the table below you see the research of three groups[1]. They all compare the increase in learning (in months) to the cost. (It is difficult to measure cost for the second one—thus the ?.)

Interventions

1. Class size reduction from 30-20. 3 month increase in learning in a year at an estimated cost of $30,000.
2. Increase in teacher knowlege from 50th percentile to 94th percentile. 1.5 month increase in learning at a cost of ?.
3. Effective formative assessment. 6-9 months increase in learning per year at an estimated cost of $3000 per year.

Your second point about heterogeneity is one that interests me as well. I went to a school with all homogeneous classes so I don’t have the perspective of learning in a heterogeneous classroom. I do appreciate many aspects of homogeneity and think that combining it with reduced class sizes in specific classes might be the way to go. If 20% of students are failing English 9 then it would seem that one of the possible steps might be to take action by reducing class size to 20-22.

Later in the day I happened to be calling Grant Wiggins to set up a web conference and he talked about the idea whereby competencies might actually lead to “smart” homogeneity. He cautioned that he supports heterogeneity just maybe not in all cases. If I were to extrapolate his thinking I would say that at times homogeneity is the right thing to do and at times heterogeneity is the right thing to do. I know from many of his books and speeches that he is against mindless devotion to any one educational system. He always counsels that as professionals we must thoughtfully decide what is best for students.

My own view is that smart homogeneity is a good idea but one that cannot be done easily. It requires everyone to understand the subtle (all too subtle) differences between homogeneity and tracking. Poorly done homogeneity is more damaging to students than poorly done heterogeneity. That is too sweeping but I feel that there is some truth to it. I support the use of good formative assessment to teach a heterogeneous group in a one room school house style. With good formative assessment you find out who needs what and teach accordingly. Then in the next unit you find out who needs what and teach accordingly. The homogeneous groups within the class are fluid.

On Dec. 3rd I will talk about the third row of the table which is the area where we can take action right now. That said I greatly encourage a push in the community to educate other community member and the school board to the fact that education is an investment NOT an expense. States chose to get into the business of education long ago because they knew it was the right thing to do—even in tough economic times it is important to remember that the best path to future prosperity lies in educating the citizenry.

I had this quote from Dylan Wiliam in another piece I wrote but will add it here as a way of closing:

“If you achieve at a higher level, you live longer, are healthier, and earn more money…In addition, people who earn more money pay more taxes, are less likely to depend on Medicaid or welfare, and are less likely to be in prison. It has been calculated that if a student who drops out of high school would stay to graduate, the benefit to society would be $209,000 (Leve, Belfield, Muenning, & Rousse, 2007). This sum is made up of $139,000 in extra tax revenue, $40,500 savings in public health cost, $26,600 savings in law-enforcement and prison costs and $3000 in welfare savings. Eric Hanushek (2004), a leading economist of education in the United States, has calculated that if we could raise each student’s achievement by one standard deviation (equivalent to raising a student from the 50th to the 84th percentile), over 30 years, the economy would grow by and additional 10%, and just the additional taxes being paid by everyone would more than pay for the whole of K-12 education.”

1. [1] Jepsen and Rivkin (2002)
2. Hill Rowan and Ball (2005
3. Wiliam, Harrison and Black (2004)

Homework

Prompted by reading the latest English Journal from November 2008 some thoughts on homework.


1. My students usually don't do it. The ones who I think "need" to do it are the ones who definitely don't do it.
2. The only clear effect I have seen is a negative effect on grades. This is from when I used to grade homework.
3. Now that I use a policy of "getting homework done" I have found that I still don't see the kids who need to do homework still don't do it.
4. When I gave students choices of penalties for not doing homework they still did not adhere to turning the homework in--my ultimate goal. In this case I gave them 3 choices 1) turn the homework in the next morning with no penalty, 2) I call home and then they turn the homework in, 3) stay after school and get the homework done. I had 3 of the 10 students stay after and 2 of the students turn it in the next day. This rate of 50% doing it in the end is the same as I have always found, no matter what penalty I use.

Interesting ideas from the articles:

1. Choice of homework. Offer students a choice of what they want to do. For example teach a lesson and during the last 10 minutes have them choose what homework they would like to do to prove that they understand the material. Using this approach my wife suggested some scaffolding where we would show examples of effective student work the next day. I plan to try this approach next week and will report on how it goes.
2. It isn't the homework but the right homework. It is really important to give the "right" homework so that it is effective for the stated goals. I have been thinking about this over the years but just as I have been more intentional about the right assessments I need to be more intentional about the right kinds of homework.

Responses to Assessment Conference

Responses:

I loved reading those. The one that most hit home to me was 1-5 (or 1-7) grading/assessment system, rather than percentages. I just could not agree more that percentages alone mean absolutely nothing -- this is clearly a big thing that AP and IB get right.

I also think the emphasis on progress reports is huge --- while teaching, I found this to be the most draining (in terms of both energy and time) aspect of the job, but it's also the way to make the biggest difference. Reading your stuff really makes me long to be back in the classroom.

Another one:
Wow, I had the same thought as [the writer above] as I was reading (about wanting to be back in the classroom). I actually was afraid that he would walk into the office tomorrow and tell [his boss] he was done...

Really interesting stuff and I think the fact that you are posting these things (and I'm assuming making that available to other teachers) is absolutely an incredible way to communicate the stuff you took away from the conference to the staff. So much resonates.

I would definitely encourage you to not be shy about getting this in front of teachers. You know how busy people are. Send them the link 3 times or find some other way to do it. It is important.

Averaging and Zeroes

http://www.teachertube.com/view_video.php?viewkey=dbd43d54b1307129474f

This is a clip of Dr. Douglas Reeves speaking to a Canadian audience about what he calls "toxic" grading practices. Reeves is the author of more than 25 books and countless articles on education.

In the clip Reeves talks about zeroes and averaging. We showed this clip to our high school faculty and there were wide ranging responses:

• Students deserve the average because this helps differentiate the steady performing student from the student that does poorly all the time but well at the end.
• Students are a sum of all of their performances so the average is the correct score for them.
• Zeroes are an essential part of grading.
• I want students to see zeroes and I want them to be calculated in the grade.

Here I want to clarify and expand upon some of what Dr. Reeves was getting at. First in the case of the average.

While in the clip he states provocatively that all averaging must go, what he and other researchers have been preaching for the past decade is to end a mindless devotion to the average as the only way of evaluating students. In fact the average might be the right score for a given student. I, along with Reeves and other researchers, am arguing that the average is not the best evaluation for all students.

It comes back as always to a conversation about standards. Is our goal to get them to the standard (in our parlance competency)? If we work hard as teachers and students work hard at learning and understanding and they make it to the standard what should the grade represent? Why should it be the average in this case? If everyone can meet the standard at the end but then we average scores it not only hurts students but it hurts us. The scores are a poor representation of how we operated as teachers. Grading is always a subjective process. As professionals we strive to minimize the subjectivity but we cannot eliminate it. As professionally trained practitioners we should allow ourselves to award students the score, the evaluation, that most appropriately matches their ability.

As far as zeroes I have posted on this many times but take one more example.

• 100-40=A
• 39-30=B
• 29-20=C
• 19-10=D
• 9-0=F

When you present this example people say that is ridiculous! But this system is as mathematically unsound as the system where the top four categories are 10 or 11 points and the last category is 59 points. So this is the first argument against zeroes--it is simply mathematically unsound.

The second argument is that it does not increase motivation. And has been shown by Dr. Reeves to have a role in whether struggling students stay in school or leave. Zeroes motivate only one type of student--good ones, ones like teachers used to be when they were in the classroom. The students that we worry about the most are not motivated to do work by receiving a zero. To the contrary they are encouraged to give up because when zeros mount the combination of their extra mathematical weight and the increase in a feeling of hopelessness cause students to shut down.

I feel strongly that zeroes should not be used and the average should not be used in all cases. That said, if a teacher still wants to use zeroes and averages as the only way to go I would ask them to continue that practice only after reflecting on exactly why they want to do it that way. As professionals we will always evaluate in different ways--the question is: Is the way you reach the evaluation of a student the best representation of what they can do?

Now What?

OK--so I went to this great conference--see previous post from Nov. 1--now what do I do?

The conference on assessment either confirmed some ideas that I was trying to work on in my teaching or reminded me that there are things that I need to do better on.

Confirmed

• Zeroes--Doug Reeves calls assigning zeroes toxic. I now have more confidence than ever to continue not assigning zeroes.
• Averaging only--I will continue to look at progress and the most recent evidence to compute student scores. This is made more difficult by an electronic grade book that only averages but I will go back to paper if I have to.
• Formative assessment--the most important thing to focus on.

Reminded

• Assess means to "sit beside." I need to work on giving even more timely feedback to my students. I will work on making contact with each student at least once during a 90 minute period. During this contact I will try to really ascertain whether they are getting it or not.
• Randomization. Student participation needs to be random. This lets kids know that they can be called on at any time so they stay more involved. I will finally buy the Popsicle sticks that I have been meaning to get, and put each students name on one of them. This is how I will draw their names randomly.
• Hands down except to ask a question. No one can raise their hands. I will pull the Popsicle stick and call on them.
• Pass on a question. I have always allowed students a pass. But I will change this policy so that my response will be, "OK, but I'll come back to you at the end." This means that they need to stay engaged and listening to the others because I will come back to them and say, "What did you think was the best answer of those three and why?"
• Non-fiction writing. I knew that writing was important but I did not know of its profound effect on all subject areas. Thank you Doug Reeves for this one! Even in physics and math (and science) an increase in non-fiction writing about the subject increases student scores. I do a lot of non-fiction writing but I will improve upon what I do and make it more systematic. The first thing I am going to do is to have students write about the most important thing that they learned in our class during the first semester. They will post these on the wall outside of our room. We will do this on Monday. A second thing that I would like to work on is a blog of student work. This may begin as a paper newsletter but I hope I will be able to morph it into a publication at some point.
• Graphing of student performance. I do a little of this now but I need to improve what I do and make it, again, more systematic. I will create a one page sheet for each competency and begin having students graph their performance.

School Wide

• At the Beyond ABC conference in 2006 we were told that one thing that we should change immediately is progress reports. If they include a number they are not progress reports. Reports with a number are fine by the way--they just aren't progress reports. I will work harder than ever to finally make real progress reports a reality again at Concord High School. I have great respect for the progress report committee of last year (I have to my wife was on it) but their findings are in opposition to the research. I feel that we do not need to discuss this one. A leader should simply explain the following: 1. We give students an achievement report every day through our online system Parent Connect, 2. This means that we do not need to print off a special "paper" achievement report at a given time, 3. What we do need is an official day where teachers are encouraged to sit beside students and discuss progress. NOT the grade, but progress. How is the student doing in relation to meeting the competencies? How is the student doing in relation to the Habits of Mind?, 4. We value this progress conversation so much that we are setting aside 4 days per year and we want you to use these days to sit and converse with students about how they are doing.
• 101 point grading system. I will work to continue to educate teachers, students and parents about the fallacy of the reliability of the 101 point system. If the AP board thinks that 5 levels are OK then we don't need to continue to do 101.
• 1-5 scale. I will work to continue to educate people on the logic and the reduced subjectivity of the 1-5 scale.

I should have this all accomplished by the end of the week. Maybe 2 weeks.

Ahead of the Curve Conference Report

Teaching is the most cognitively difficult thing I have ever done in my life. Compared to classroom teaching, running a university is easy.

-Dylan Wiliam—Interim President University of London, and author of more than 250 books and articles on education.

Last week I was honored to be sent to a conference called Ahead of the Curve. This conference brought together many of the best minds and greatest researchers in the field of education. The list: Tom Guskey, Bob Marzano, Anne Davies, Stephen White, Dylan Wiliam, Ken O’Connor and Rick Stiggins presented on a wide range of assessment issues. From nearly 20 hours of presentations and conversations with colleagues from around the country I have culled the top 13 things that I have learned or had reinforced for me as a result of this experience. All 13 are based on, not one or two studies but dozens and in cases hundreds of researched studies. The 13 items do not represent fringe ideas or controversial issues—they are as close as we get in this business to accepted ideas.

Top 13 (It is Halloween Weekend) and they are in no particular order
1. Whatever the test, the scores should mean the same thing.
2. Never rely on a single assessment to determine a student score.
3. Graphing results of student performance. Has a big effect size. Need to be doing this.
4. Explaining what stays and what does not is critical.
5. Standards were created in part to reject the bell curve. If we are doing standards we should not see a bell curve at the end.
6. Format doesn’t matter.
7. Impact of the Jills.
8. Feedback should be like PE and band and sports.
9. Dropout prevention.
10. The myth of the 101 point system.
11. Progress Reports.
12. Teachers work differently, students work harder.
13. Minute by minute corrections.

From Bob Marzano
1. Whatever the test, the scores should mean the same thing.
When a student takes a test in any area they should know that the score in one class means the same thing as a score in another class. This means that an A in one class should mean the same thing as an A in another class. Or for our world—we should agree that 3 means competent and 2 means nearly there no matter what the class. Even though I have been a leading advocate of a point system with fewer categories I would even settle for A, B, C, D and F as long as we refocused it to have a standards based meaning rather than a norm based meaning.

2. Never rely on a single assessment to determine a student score.
Statistics show that even the very best single classroom assessment cannot measure a student’s true ability accurately. It will give a score but fairly simple statistics show that the student’s true score (true ability) can be plus or minus 15 percentage points of that value. To combat this teachers need to base a student grade on many pieces of evidence.

3. Graphing results of student performance. Has a big effect size. Need to be doing this.
Students need to be creating graphs of their learning. The research shows that this has an incredible effect on student learning. So much so that Marzano said that if we aren’t doing it we should start immediately.



From Doug Reeves
4. Explaining what stays and what does not is critical.
It is crucial to explain to teachers, students and parents what will not change. For example in our current work—teacher autonomy will not change, it never will, it can’t. The competencies will be the same and the assessments will be common but how you teach the material will be up to you. This is an area I am going to improve upon as we move forward with our work together.

5. Standards were created in part to reject the bell curve. If we are doing standards we should not see a bell curve at the end.
Benjamin Bloom said, “There is nothing sacred about the normal curve. It is the distribution most appropriate to chance and random activity. Education is a purposeful activity, and we seek to have students learn what we have to teach. If we are effective in our instruction, the distribution of achievement should be very different from a normal curve. In fact we may even insist that our educational efforts have been unsuccessful to the extent that the distribution of achievement approximates the normal distribution.” I won’t pretend to think that I can better Bloom’s quote but I will add that it is from 1981.

6. Format doesn’t matter.
Doug Reeves actually did a study on this. He looked at plans that he deemed ugly and plans that he deemed pretty. (He is a great researcher so he did have parameters but this is about how he explained it.) Do not focus on creating a beautiful binder that sits on a shelf. Focus on the work and creating something that will work in class and with students. The simple competency template, for example, was just a guide—it works for some it does not for others—awesome. Some folks are using the traditional UbD template, others are coming up with something new—do what works for you within the bounds of goal, assessment, learning plan.

7. Impact of the Jills.
“Jill” is the awesome teacher in the classroom next door. More than any other person (like an assessment coordinator, for example) this teacher leader is the key person and the most influential person on practice. We need to go out and see the good things that other teachers are doing. In our district we are lucky to have many of these—find them and go see what they are doing. And aspire to become more of one yourself. Corollary to Impact of the Jills: We have learned from copious studies and the experience right in front of our faces that students need to learn by doing. Teachers need to learn by doing as well. Seeing what Jill is doing and then trying it out is good practice.

8. Feedback should be like PE and band and sports.
The goalie in a soccer game comes too far out of the net and the opposing team kicks the ball in a loop over his head for an easy score. In order to correct this, the coach makes a note with a red pen in an improvement journal and when the season is over lets the goalie know that he should only come that far out of the goal in certain situations. Sound right? NO. Give lots of accurate and useful feedback—feedback that students can use to correct what they are doing right now. Think about how much feedback an orchestra conductor or a PE teacher give while a lesson is going on. It isn’t always easy but we need to try to emulate these examples in other areas of teaching too.

9. Dropout prevention. Again these aren’t random suggestions they are research based statements from Doug Reeves. He said that these are the things that he would do immediately to combat dropouts.
Literacy—double time—everyone gets double the literacy time—reading is that statistically important
Time management assignment notebook—need to teach all students what they need to do organizationally to be successful.
Immediate intervention—before failure
Engagement—extracurricular policy
o 3-4 is the ideal—this is the range of extras that have a positive effect on learning.
o Need to work on kids who have zero
o Evidence says that if they are more engaged they have better attendance and
o Private schools are always in extracurriculars even when students are not doing well.
o There are consequences but the consequences are not removing them from extras.
Price of freedom is proficiency
Homework may not be about home
Outlaw zeroes--
Early final exam—do early final exams
o Give exam 2 weeks early. Those who are competent are done.
o Those who are not stay and work more.
o A district that did this went to 90% passing.
Credit recovery—ID reason for F and figure out what to do.

10. The myth of the 101 point system.
We have been fooling ourselves. With the advent of calculators and then computers and then computer grading programs teachers moved from A, B, C grading to percentage grading. While crunching numbers into an average does yield a percentage it clouds what the student actually knows and provides little information about what, specifically, needs to be done to improve. It is not a reliable measure. While one teacher may claim to be able to tell the difference, with her students, between a 71 and a 72, there is no way that separate teachers evaluating independently would ever come to this fine of a separation. Corollary myth—the public expects percentages (parents, colleges, students). In truth parents already clamor for and colleges love AP classes which in their end result award one of 5 categories. (The International Baccalaureate offers 7.) If a student takes a practice AP test and scores a 3 they know in very specific terms what they need to work on to move up to the 4 category. This type of conversation can happened with percentages—it is just not as easy. Why make communication harder for students?

From Ken O’Connor
11. Progress Reports.
A report with a grade is an achievement report not a progress report. At the high school Parent Connect now allows parents and students to see a daily achievement report. The need for us to print out one more at a specific point in time is now zero. The high school now has no official progress report and in light of competencies and the needs of students it would be good if one was reincorporated. Since the students now have constant access to their achievement level what is needed more than ever is a conversation about progress. Ok Sweta, you see that you have a C. Let’s talk about what that means and what we can do together to improve that grade over the next few weeks. Especially with competencies it might be appropriate to say, “A C is a perfect place to be right now—your progress thus far in the course is on target. We will of course work together to improve your understanding and thus your grade but right now you are progressing wonderfully.” Alternatively a conversation might happen this way, “Vesper, you have a C and at this point in the year that is evidence that you aren’t making sufficient progress. Let’s talk about what we can do together to improve your understanding.

From Anne Davies
12. Teachers work differently, students work harder.
The person who is tired and busy is the person who has done the learning. Teachers will continue to tire themselves outside of class but in class they should be the ones who are guiding not relearning the material they already know. Students should be stuck, confused and challenged—this is when they learn. One teacher that we heard about has a sign in their class that says, “Stuck? Good. It was worth coming in today.” There are many ways of doing this but the big idea is to put the onus on students to demonstrate what they can do.

From Dylan Wiliam
13. Minute by minute corrections.
There are many solutions that we as educators have tried. Most of those solutions have been superficial and easy to implement—the problem is they have not worked. Wiliam says that now is the time to get down to the difficult solutions. The number one effect on student learning is the teacher that they have. The number one strategy that affects learning is formative assessment. Formative assessment means assessing students on a minute by minute or second by second basis so that one can make changes in a nimble an effective way.

One more from Dylan Wiliam
Education is not an expense, it is an investment.
“If you achieve at a higher level, you live longer, are healthier, and earn more money…In addition, people who earn more money pay more taxes, are less likely to depend on Medicaid or welfare, and are less likely to be in prison. It has been calculated that if a student who dropes out of high school would stay to graduate, the benefit to society would be $209,000 (Leve, Belfield, Muenning, & Rousse, 2007). This sum is made up of $139,000 in extra tax revenue, $40,500 savings in public health cost, $26,600 savings in law-enforcement and prison costs and $3000 in welfare savings. Eric Hanushek (2004), a leading economist of education in the United States, has calculated that if we could raise each student’s achievement by one standard deviation (equivalent to raising a student from the 50th to the 84th percentile), over 30 years, the economy would grow by and additional 10%, and just the additional taxes being paid by everyone would more than pay for the whole of K-12 education.”


Want to learn more?

A little bit of everyone
Ahead of the Curve—edited by Doug Reeves—this book has a chapter written by each of the people I mentioned.

Bob Marzano
What Works in Schools
School Leadership that Works

Doug Reeves
Making Standards Work
The Learning Leader: How to Focus School Improvement for Better Results
Many articles online and in Educational Leadership

Tom Guskey
How’s My Kid Doing?
Designing Grading and Reporting Systems
Many articles available online and in Educational Leadership

Dylan Wiliam
Inside the Black Box—Phi Delta Kappan 1998 (Let me know and I’ll make you a copy.) This is the study that began the modern push for formative assessment.

Ken O’Connor
How to Grade for Learning

Anne Davies
Making Classroom Assessment Work