Goodhart's Law in education and religion

.

I just heard of Goodhart's Law this week thanks to xkcd. This xkcd required a bit more supplementary reading than usual. The comic itself is sufficient to get the joke but not to really get the joke, if you know what I mean.

Goodhart's Law: "When a measure becomes a target, it ceases to be a good measure"

In short, some things we deeply care about cannot be measured. But since related things can be measured, we just measure those instead. But, sometimes, we then start to care about the thing we’re measuring more than the thing we care about.

Incidentally, the images in this post will all link back to where I took them from. Starting with these from something called Jascha’s blog:

Never have I seen something that so well explains so much of the crap happening in education. One example that’s been in the news lately is the way Princeton Review and U.S. News and World Report rank colleges.

1. We care about what colleges are good.

2. No one knows what that means.

3. So we measure other things instead. Such as how many more people apply to a college than the college is able to accept.

4. Thus, the University of Chicago spends tons of money each year trying to get kids to apply whom they fully anticipate rejecting just to get their numbers up.

But this also applies to a lot of the postgrads trying to study what makes teachers good at their job or the weird assessments schools and teachers give kids in order to see if they’re “learning.”

Teaching to the test is an example of Goodhart's Law. No Child Left Behind is an example of Goodhart's Law. As is Race to the Top. Some of those may be results of Goodhart's Law and some may be examples from birth but all of them result in looking at the wrong things rather than what we care about: happy, intelligent, well balanced, educated kids.

Education needs to “[end] the myth that standardized data is a perfect, neutral arbiter.”

Then, as I was sitting in the temple this afternoon, I realized a lot of the same crap is happening in religion too. Or, at least, my own religion.

Crap:

Take for instance the Church’s recent decision to remove women from the stand in Bay Area congregations. This decision has caused a great deal of pain and upset. And why? Well. Goodhart's Law. That’s why. Here are three paragraphs from that article:

The practice was abruptly discontinued last month, according to church spokesperson Doug Andersen, at the order of the North America West Area president, whose jurisdiction includes California.

The Utah-based faith “has a long-established practice when it comes to worship services,” Andersen says. “The general pattern includes presiding authorities sitting on the stand along with other women, men, youth and children based on their invitation to participate in the service.”

Local leaders, Andersen says, “were recently reminded of this practice.”

This has to be the weakest reasoning for official action I’ve ever read. Certainly it’s the least Joseph Smithy reason I can fathom. But if you want to hear me complain more about the dumbness of it all, you got this:

But here’s what I think happened. And thank you xkcd for helping me figure this out.

1. We want to achieve Zion, to be of one heart and one mind.

2. No matter where you attend church across the world, the Saints are (more or less) studying the same lessons at the same time.

3. All surface things must be the same!

4. Oh no! Women on the stand in Lafayette? That’s not happening in St. George!

5. Shut it down.

The Church’s unwillingness to kick women off the stand in writing. The fact that not only is this not a canonical doctrine—not even a policy!—just a “general pattern” or “practice.” These facts show that it’s not really defensible.

Having women on the stand led to area women feeling seen and appreciated. It helped us all come closer to the ideal of one heart and one mind.

And the inelegant way this was torn away from the women specifically and Zion generally threw hearts and minds into chaos and pain.

And that’s the danger described by Goodhart's Law.

Zion is, perhaps, an impossible thing to measure.

So better be careful what we do measure.

And we’d better be careful not to let that measurement turn into an idol.

Souls are on the line.

Subscribe to Thubstack.