Daniel Koretz on Testing

Question: Is testing an adequate measurement of student achievement?


Daniel Koretz:    Well, I think, standardized tests as we currently have them are enormously useful.  So, for example, we all know anyone who follows education that there are enormous inequities in American schooling.  One of the ways we know this is because of standardized tests.  That’s what tells us, not only that the inequities are there but how big they are.  And I personally think that it push for accountability in schools is not going away anytime soon.  The public clearly wants schools to be held accountable for increasing the level of their productivity, which means teaching kids more.  So I see no way to have a sensible accountability system without testing.  I just think we have to use it more rationally.  It’s very much like… you know, this is perhaps a weak analogy, but it’s like overprescribing a very strong medication.  The problem is not that the testing shouldn’t be used.  I wouldn’t argue that testing shouldn’t be used, we’re just using it in the wrong way, putting far too much pressure on increases in scores.      

Question: What other standards could we use to measure intelligence?

Daniel Koretz:   

Well, there are new ideas about how to measure students’ achievement.  Some of them are very hard to use on a large scale.  That one of the difficulties we have is that there’s a lot of pressure now and should be a lot of pressure to test high [order] thinking skills, complex problem solving what people like to call 21st century skills.  Some of that is very hard to do with the test that you drop in from a state capital.  I won’t give you a long explanation of the reasons why but one of them is that to measure whether a student has good problem solving skills, you have to give the student a test that somewhat [novel].  If you give the student something that’s routine for them, they don’t have to use real problem solving skills, they can just redo what they’ve done before.  It’s very hard sitting in the state capital to know what things are [novel] for you and what things are [novel] for me, what things are [novel] for children in this school over here and for children in another school.  So to some extent, we have to rely on more local measures of performance, which is what testing experts long ago, two generations ago, told educators to do.  They said over and over use test as a supplementary source of information.  Never use them alone to measure the quality of education. 
   

Daniel Koretz:    There are old techniques but they are not well evaluated.  For example, in countries such as the Netherlands, it’s routine for schools to be visited by outside inspectors who look at far more than test scores.  And, in fact, the Dutch public often hears about test scores in the context of an inspector’s report.  The problem is that we don’t have a real research and development approach to this.  We don’t do what engineers would call systems engineering.  The logical thing to do would be to say, well, we have a complicated problem.  Teachers have a very complex role.  So, therefore, we have a complicated problem in designing an accountability system.  As a result, we’ll try different things.  We’ll measure constantly how they affect schooling and will tweak the system as engineers would say we’ll bury the parameters.  We don’t do that.  We drop these things in from the sky.  We get a tiny bit of evaluation some years later and say, oh that one didn’t work.  But now we know and we drew up another one in.  And the result is that kids are basically left behind.  So what we have is a lot of interesting ideas about better ways of holding schools accountable and very little hard research.  And I would say that that’s really an ethical problem, not just a political problem.  It’s a political problem because we lack information that we could use to better serve children.  It’s an ethical problem because children are not consenting adults.  When we drop into schools these very high powered policies that clearly change teacher’s behavior in dramatic ways, we have an obligation, in my view, to monitor what happens.

Some measures of intelligence cannot be executed through testing alone, Daniel Koretz says.

Why a great education means engaging with controversy

Jonathan Zimmerman explains why teachers should invite, not censor, tough classroom debates.

Sponsored by the Institute for Humane Studies
  • During times of war or national crisis in the U.S., school boards and officials are much more wary about allowing teachers and kids to say what they think.
  • If our teachers avoid controversial questions in the classroom, kids won't get the experience they need to know how to engage with difficult questions and with criticism.
  • Jonathan Zimmerman argues that controversial issues should be taught in schools as they naturally arise. Otherwise kids will learn from TV news what politics looks like – which is more often a rant than a healthy debate.
Keep reading Show less

Are these 100 people killing the planet?

Controversial map names CEOs of 100 companies producing 71 percent of the world's greenhouse gas emissions.

Image: Jordan Engel, reused via Decolonial Media License 0.1
Strange Maps
  • Just 100 companies produce 71 percent of the world's greenhouse gases.
  • This map lists their names and locations, and their CEOs.
  • The climate crisis may be too complex for these 100 people to solve, but naming and shaming them is a good start.
Keep reading Show less

SpaceX catches Falcon Heavy nosecone with net-outfitted boat

It marks another milestone in SpaceX's long-standing effort to make spaceflight cheaper.

Technology & Innovation
  • SpaceX launched Falcon Heavy into space early Tuesday morning.
  • A part of its nosecone – known as a fairing – descended back to Earth using special parachutes.
  • A net-outfitted boat in the Atlantic Ocean successfully caught the reusable fairing, likely saving the company millions of dollars.
Keep reading Show less