Thursday, March 10, 2011

Costly Signaling for Lay Skeptics

Let's say you're an average person, of average intelligence, average education, with an average job, and you've run across several news articles.

One says that an asteroid has just been detected that will hit the earth in 2015. Another says that taking vitamin B3 daily can improve your cholesterol levels. A third says that increasing defense spending will help balance the budget. Another says that evidence of extraterrestrial life has been found in an Antarctic meteorite. A fifth one says that the Gospel of Mark has been dated as having been written between 40 and 50 CE. And finally, a story that people who prayed to a statue of Krishna have been cured of cancer and blindness.

How do you, as a lay person with a full-time day job, determine which ones to believe, and which ones to disregard?

I don't have a good answer, by the way. I'm hoping you can suggest something in the comments.

All such articles are trying to "sell" you an idea, in a broad, general sense. Sometimes the selling is literal, as when a company tries to convince you that you're a pathetic malodorous loser who'll never be accepted by the in-crowd or find true love unless you buy their product. Other times, it's metaphorical: "I want you to know this, because..." well, that's the question, isn't it? "Because we'll all benefit if people who will implement these ideas get elected." "Because I'll make a ton of money if you help elect people who'll implement these ideas." "Because I care about you and your health." "Because this will help save your soul from eternal damnation." "Because this idea, while bland, is true, and I think it's better if we know the truth."

It would be great if there were a single source to which one could turn to to get the truth, or if news articles came with a little checkmark, the way Twitter shows that "neilhimself" is the famous Neil Gaiman, while "NeilGaiman" is someone else. Unfortunately, that's not the case. The problem is that true ideas and false ideas can look an awful lot like each other.

But it occurs to me that nature has come up with a solution to this problem. In sexual species, males often try to communicate that "you should mate with me; I'll provide our offspring with plenty of food, and they'll be resistant to parasites and predation." In such cases, it's often advantageous to lie: a male who convinces a female he's in it for the long haul can impregnate her, then ditch her to impregnate someone else. Preferably while some other male sucker gets stuck caring for the liar's offspring.

So what's a female to do? How does she figure out who's serious about helping to feed the kids, and who's just trying to get inside her cloaca? One solution is known as costly signaling. "Signaling" refers to the "I've got great genes" message, above. The "costly" part means that the signal should be sent in a way that's difficult or expensive (in time, effort, ability, etc.) to fake. The usual example is that of the peacock, who demonstrates his worth by the fact that he's managed to survive despite having a huge, flashy tail that prevents him from flying, and hinders escape from predators. If he's managed to overcome such a handicap, he must have superior genes indeed.

The idea of costly signaling is more general than that: it basically means that the signaler has to invest enough effort or resources into the communication to be taken seriously, that cheating isn't worth it.

(As an aside, I can think of a few possible instances in human society: an engagement ring sends the message that "I'm willing to spend a pile of money on a small rock; so I'm in this for the long haul, not just for a quick fling". Taking a prospective client to dinner or to a ball game says "We don't do this for just anyone; but we're willing to do what it takes to get your business." And an Italian sports car and designer clothes say "I have so much money that I can afford to waste it on an expensive logo. Of course I'll be able to feed our family and send our kids to college.")

So getting back to my original point, it might be possible to identify costly signals to distinguish trustworth news sources from untrustworthy ones.

For instance, was the article published by a major news outlet, or by some local paper you've never heard of? In principle, the greater the reputation of the publication, the more editors and fact-checkers it has had to pass through to get published. Unfortunately, given the state of American journalism, this may not be as safe an assumption as one might hope.

A related criterion might be: do they have a fancy web site, or does it look like it was slapped together by someone's kid in the 1990s? Unfortunately, this doesn't work at all, since organizations like Americans for Prosperity, BP, and Answers in Genesis can easily afford good web designers.

Do the authors have letters after their name? An article on medicine written by an MD, or an article on science written by a Ph.D. is probably more trustworthy than one written by a beat reporter. The time and effort required to go through grad school or med school to obtain those letters should weed out the fakers.

Of course, the competence has to be in a relevant field: I tend to trust what Paul Krugman writes about the economy, because he has a degree and a Nobel prize in economics, but not if he writes about, say, medicine or geology.

And, of course, it's very easy to just say that one has a Ph.D., or to buy a degree from a diploma mill, without putting in the effort to learn a subject well enough to speak authoritatively about it. To combat this, there accreditation institutes that investigate schools and give their stamp of approval to the ones that require students to learn something before graduating. Of course, now that a lot of people have learned to ask "is your degree from an accredited school?", there are accreditation mills, which will accredit any diploma mill for a fee.

Has the author published any peer-reviewed research? Peer review is intended as a filter to make sure that research journals don't publish any old garbage. This criterion is probably pretty good, though not flawless. For one thing, it usually requires effort on the reader's part to seek out the author's publication record. For another, various creationist organizations publish cargo-cult "peer-reviewed" journals where articles are reviewed by a panel of fellow creationist before publication.

Trusted endorsements: this might be called the poor man's peer review. When Phil Plait, an astromer, writes a blog post that links to a post on astronomy, that's a good sign. It means that the article on the other end of the link hasn't raised Phil's baloney-meter. That tends to make me trust the article more, because Phil would notice errors that I wouldn't.

Does the site link to contrary views? In its heyday in the 1990s, one notable difference between the pro-evolution site and anti-evolution sites was that usually linked to the creationist sources they were discussing, and to creationist rebuttals of their articles. To me, this said "we're going to make it easy for you to read the other side's rebuttal, because we're confident that the facts are on our side, and even if you read both sides, you'll agree with us."

Any others? Ideally, the sort of costly signal should be something hard for the writer to produce, and easy for the reader to verify, without requiring too much effort (because we want to dismiss bogus claims quickly) and without requiring special knowledge. And if the criterion fits on a bumper sticker, so much the better.


lucette said...

Possibility to reproduce the results.

Don Wharton said...

Reproduced results is the essential confirmation of good science. The post was talking about average people determining the accuracy of claims in news stories. Those would seldom have a careful examination of other science results but it certainly should give the reader more confidence if confirming science is cited.

Gary Berg-Cross said...


This discussion of regular folks news analysis is a timely, important topic that I think about. So thank you for the posting. I imagine we could have extended discussions on it.

About 10 years ago I volunteered to help Stephen Fromm on his web project to act as a clearinghouse for knowledge: as

"an attempt to achieve economies of scale in the dissemination and organization of information, both current and historical, relevant to politics and public policy. The project’s primary long-term goal is to help individuals access the current sphere of knowledge more efficiently and avoid needless duplication of effort. Furthermore, describing the current and past state of the world will be emphasized over normative statements of how the world should be. "

You can still see the site, which stopped in 2004, and some analysis at

One of the ideas of what we were doing was to use analysis try to converge on ground truth/closer to reality when complex topics are discussed. This is not simple, but we were hoping that with more things being published on the Internet we might be able to assemble enough facts etc. to make cases clearer. We never had enough resources to do much of this and Stephen couldn't attract $$ to fund the effort further.

I think that the situation, of getting to ground truth, has even gotten harder in that there are most costly signals being thrown. That is, we have funded efforts to put disinformation out and advocates use viral techniques to manipulate people's opinion through their hot buttons. It's costly to create such signals, but that is what the wealthy can do, so it may be a reasonable strategy to be skeptical of these costly signals.
We are victims of our limited rationality and increasing limited time to sift through the info glut that now is the state we find ourselves in. To simplify things we sometimes resort to some naive version of balance as if there were 2 equal sides to all issues. This is often a false dichotomy.

One thing that I might suggest is to think of the situation organically. As info consumers we have to develop and validate "filters" for things coming in our environment. A filter might be a trusted source such as Paul Krugman, who has credentials and a history of prediction on things such as the financial meltdown. So follow him over time and assess how well he does. Some predictions don't hold up. (A funny example of this was done some time ago asking Prophets to predict what things would happen in the first 10 years of the new century. Other examples have chimps picking sports winners to see if they are as good as experts.)
I would say this converging evidence approach is an important aspect of Climate Change. Here well funded efforts create messages that make it hard to hear the growing, supportive data. Did we have a colder than normal winter. It is hard for a regular person to know, but even in this area WAPO keeps track of warming degree days for the season and if you look at the back of the Metro section you will see that we needed slightly less heat so far this season than last year or the average (about 15 out of 3000 degrees days less, while this summer we needed about 1000 out of 4000 more cooling that the average, so summer was hotter and winter is about average this year).

In the Truth and Politics effort we were trying to assemble such info as sort of a meta-filer for people to look at ad use. I think that we all have filters of some kind but even here we often don't have time to validate them. We need help....