Daniel Koretz on Testing

Question: Is testing an adequate measurement of student achievement?


Daniel Koretz:    Well, I think, standardized tests as we currently have them are enormously useful.  So, for example, we all know anyone who follows education that there are enormous inequities in American schooling.  One of the ways we know this is because of standardized tests.  That’s what tells us, not only that the inequities are there but how big they are.  And I personally think that it push for accountability in schools is not going away anytime soon.  The public clearly wants schools to be held accountable for increasing the level of their productivity, which means teaching kids more.  So I see no way to have a sensible accountability system without testing.  I just think we have to use it more rationally.  It’s very much like… you know, this is perhaps a weak analogy, but it’s like overprescribing a very strong medication.  The problem is not that the testing shouldn’t be used.  I wouldn’t argue that testing shouldn’t be used, we’re just using it in the wrong way, putting far too much pressure on increases in scores.      

Question: What other standards could we use to measure intelligence?

Daniel Koretz:   

Well, there are new ideas about how to measure students’ achievement.  Some of them are very hard to use on a large scale.  That one of the difficulties we have is that there’s a lot of pressure now and should be a lot of pressure to test high [order] thinking skills, complex problem solving what people like to call 21st century skills.  Some of that is very hard to do with the test that you drop in from a state capital.  I won’t give you a long explanation of the reasons why but one of them is that to measure whether a student has good problem solving skills, you have to give the student a test that somewhat [novel].  If you give the student something that’s routine for them, they don’t have to use real problem solving skills, they can just redo what they’ve done before.  It’s very hard sitting in the state capital to know what things are [novel] for you and what things are [novel] for me, what things are [novel] for children in this school over here and for children in another school.  So to some extent, we have to rely on more local measures of performance, which is what testing experts long ago, two generations ago, told educators to do.  They said over and over use test as a supplementary source of information.  Never use them alone to measure the quality of education. 
   

Daniel Koretz:    There are old techniques but they are not well evaluated.  For example, in countries such as the Netherlands, it’s routine for schools to be visited by outside inspectors who look at far more than test scores.  And, in fact, the Dutch public often hears about test scores in the context of an inspector’s report.  The problem is that we don’t have a real research and development approach to this.  We don’t do what engineers would call systems engineering.  The logical thing to do would be to say, well, we have a complicated problem.  Teachers have a very complex role.  So, therefore, we have a complicated problem in designing an accountability system.  As a result, we’ll try different things.  We’ll measure constantly how they affect schooling and will tweak the system as engineers would say we’ll bury the parameters.  We don’t do that.  We drop these things in from the sky.  We get a tiny bit of evaluation some years later and say, oh that one didn’t work.  But now we know and we drew up another one in.  And the result is that kids are basically left behind.  So what we have is a lot of interesting ideas about better ways of holding schools accountable and very little hard research.  And I would say that that’s really an ethical problem, not just a political problem.  It’s a political problem because we lack information that we could use to better serve children.  It’s an ethical problem because children are not consenting adults.  When we drop into schools these very high powered policies that clearly change teacher’s behavior in dramatic ways, we have an obligation, in my view, to monitor what happens.

Some measures of intelligence cannot be executed through testing alone, Daniel Koretz says.

This is the training any graphic designer should have. It’s on sale now for under $40.

Come to grips with the fundamentals of graphic design and master the field's top tools.

Gear
  • The School of Graphic Design Mastery Bundle covers basic industry philosophies and tools.
  • Special courses dig into branding techniques as well as using the Adobe Creative Cloud apps.
  • The $1,400 training course is available now for only $39.
Keep reading

What the world will look like 4°C warmer

Will your grandchildren live in cities on Antarctica?

Surprising Science

Micronesia is gone – sunk beneath the waves. Pakistan and South India have been abandoned. And Europe is slowly turning into a desert. This is the world, 4°C warmer than it is now. 

Keep reading

Why vaccines are absolutely necessary

Vaccines have done their job so well that anti-vax parents have forgotten the horror of contagious disease.

Videos
  • "Autism is caused by a lot of factors that we don't fully understand," says epidemiologist Dr Larry Brilliant, "but vaccines are not one of those factors."
  • Vaccines have saved hundreds of millions of children's lives—they have eradicated smallpox, nearly eradicated polio, and they have reduced the population explosion. How? Thanks to vaccinations, parents no longer expect 50% of their children to die from disease, so they have less children.
  • Vaccines have protected the lives of children so effectively that anti-vax parents—who only have their children's best interests at heart—have lost sight of how critical vaccines are. When polio was rampant in the U.S., parents waited in line for hours and hours to have their children vaccinated. Safety changes our mental calculus, but vaccinations must continue to ensure that safety lasts.
Keep reading