On Biased Research and Enlightenment

Last spring, researchers published the results of a study in which 5,000 Americans were asked to choose one of five responses—strongly agree, somewhat agree, strongly disagree, somewhat disagree, or not sure—to the following statements:

1. Restrictions on housing development make housing less affordable.
2. Mandatory licensing of professional services increases the prices of those services.
3. Overall, the standard of living is higher today than it was 30 years ago.
4. Rent control leads to housing shortages.
5. A company with the largest market share is a monopoly.
6. Third-world workers working for American companies overseas are being exploited.
7. Free trade leads to unemployment.
8. Minimum wage laws raise unemployment.

Every "correct" answer was taken as a sign of "economic enlightenment," which, I'm sorry, is a ludicrous term, and I'm going to insist on using scare quotes. Even if we go along with the premise that each of these statements is either undeniably true or undeniably false, there's nothing especially enlightened about acing a multiple-choice quiz where three of the five possibilities—including, in all cases, "not sure"—are considered correct.^[1]

The researchers were caught entirely by surprise, I'm sure, to find that respondents who claimed to be conservative or libertarian were more "enlightened" than those who claimed to be liberal or progressive, with moderates, as is their wont, landing somewhere in between. They also found no correlation between attending college and being "enlightened," for which four possible explanations are suggested:

1. The liberal-dominated academic environment makes students "not only unreceptive to economic enlightenment, but actually unfriendly to it."
2. The college experience generally shelters students from economic realities.
3. The college admissions process, with its emphasis on abominable Marxist concepts like community service, is inherently biased in favor of liberals.
4. The "enlightened" are less likely to go to college in the first place, possibly due to concerns about the factors above.

I have no major complaints with the theories (which isn't to say I agree or disagree, just that they do seem logically connected to the data). In fact, for a study clearly designed to reach a pre-determined, ideologically-driven outcome, the authors do a pretty good job of acknowledging the flaws. Among other caveats, right at the top they concede the "asymmetry in sometimes challenging leftist mentalities without ever specifically challenging conservative and libertarian mentalities."

Still, I'd like to posit a hypothesis of my own: People are disinclined to acknowledge the negative aspects of views they support, or the positive aspects of views they oppose.

I say this, in part, because if I had taken the quiz myself I would've gotten seven of eight "correct,"^[2] making me more economically "enlightened" than even the average conservative or libertarian, and more than twice as "enlightened" as the average liberal or progressive. So, great for me, except that I don't know anything about economics. On the comprehensive list of my areas of expertise, economics ranks somewhere above particle physics and below Australian rules football, which I believe is played on a field shaped like an oval.

On the "rent control leads to housing shortages" question, for example, my thought process went something like this: "Rent control restricts natural market forces; natural market forces are generally good—therefore, rent control is bad. Housing shortages are also bad. Is the latter causally related to the former? I don't know, probably." And with that, I patiently await my letter from the Nobel Committee.

———————

Everything above this line was written several months ago. The plan was to illustrate my point by creating a similar quiz where the "correct" answers favored liberal views, but I never got around to it. I should've known if I waited long enough someone would do it for me. Specifically, I'm thinking of that study I ranted about back in January—the one that found Fox News viewers to be "misinformed" by asking questions like this:

• Is it your impression that economists believe the economic stimulus (a) caused job losses, (b) saved or created a few jobs, or (c) saved or created several million jobs?
• Is it your impression that among economists who have estimated the effect of the health reform law on the federal budget deficit over the next ten years, (a) more think it will increase the deficit, (b) more think it will not increase the deficit, or (c) views are evenly divided?
• Do you think that most scientists believe that (a) climate change is occurring, (b) climate change is not occurring, or (c) views are evenly divided?

Asking for the opinions of experts, rather than the respondents themselves, allowed the researchers to plausibly (but not indisputably) say that the questions have objective answers, but this study has exactly the same problem as the other one, with the ideologies reversed.

For both questionnaires, there are two groups of people likely to give the "correct" answers—those who actually know the answers, and those whose thought processes haven't evolved past "MY SIDE RIGHT, OTHER SIDE WRONG." When no attempt is made to distinguish one group from the other, the data probably won't tell us anything we don't already know, and it certainly won't support a conclusion that conservatives are in some way superior to liberals, or vice versa.

Being enlightened, or well-informed, or whatever you want to call it, and being a close-minded ideologue aren't the same thing—in fact, they're diametrically opposed. It really seems like that should go without saying, but apparently it doesn't.

1. That's not to disparage those who answered "not sure." In fact, I'd argue "not sure" is the only truly enlightened answer. In the translated words of Socrates:

And how is not this the most reprehensible ignorance, to think that one knows what one does not know?

2. The exception? Number six: "Third-world workers working for American companies overseas are being exploited."
    Dictionary.com defines "exploit" as (1) to utilize, esp. for profit; (2) to use selfishly for one's own ends. How it can be said that overseas workers are not utilized as part of a larger scheme to generate profit, or that the managers of such a scheme are not motivated at least in part by their own selfish interests (providing for their family, buying a nicer car, funding an army to overthrow the local dictator, etc.), is beyond me. But I guess that's why I'm unenlightened.

Survey Says: Fox News Viewers Bad at Reading Comprehension

Hey, a study! A new study! Gather 'round everyone, there's a new study out!

It comes from the University of Maryland's Program on International Something-or-Other, where researchers have used biased methodology to reach a pre-conceived result made a number of fascinating discoveries. One finding, in particular, is ideally suited to the sort of vapid, substanceless news reporting people tolerate for some reason very interesting: Apparently, regular viewers of Fox News are more likely than those who get their news elsewhere to be "misinformed" misinformed. Of course, research like this frequently raises serious questions about the integrity and/or competence of everyone involved is rarely met with universal acceptance. This study would, if printed out, actually devalue the paper it's printed on is no exception.

Alright, enough with the strikethroughs—I think I've made it clear how little respect I have for this kind of counterproductive bullshit flawed research.

Why is it flawed? A bunch of reasons, but this article from Right Wing News nicely summarizes the one I want to talk about:

[S]ome of the questions asked by the World Public Opinion Dot Org are framed around "what the experts think." So there is another obvious point to be made about the flaw in their research: It is inclined to bestow the title of "well informed" upon people who place great weight on what "experts" think, and then just slavishly mimic them. I suppose there is a certain fairness to that, along with a worthy fidelity to the intent of the study—you need to track down some information in order to find out what the so-called experts are thinking, so you can copy it—but it occurs to me. When we seek to establish and maintain an "informed" democratic republic of participating voters, this seems somewhat far-flung and distant from what we should be trying to build.

What good is information if you aren't thinking independently about it means?

Along the same lines, here's Townhall.com's Brent Bozell:

Fox viewers were more likely to believe "Among economists who have estimated the effect of the health reform law on the federal budget deficit over the next ten years, more think it will increase the deficit."

That is misinformation? This question is not about facts at all. It's about the opinions of economists looking into a crystal ball, and PIPA's "economists" estimate that herding 35 million uninsured Americans into a new federal entitlement program is going to reduce the deficit. This assertion by liberals that ObamaCare would cut deficits isn't technically a "lie"—yet. It is merely a patently ridiculous claim that doesn't acknowledge the real world. But somehow, Fox News viewers are tagged as the "misinformed" dummies, because their opinions are grounded in logic.

Those devious pollsters! It seems people weren't asked if, for example, healthcare reform will increase the deficit, they were asked if economists believe healthcare reform will increase the deficit. Several other questions were similarly phrased.

That the researchers used biased methodology is so obvious it's not worth talking about up here in the main text,^[1] so I'm setting that aside for now. At the same time, let's not treat these Fox News-viewing respondents as paragons of enlightenment. Bozell, like most conservatives who've weighed in on the study, acts like they were tricked into giving "wrong" answers, because God forbid Fox News viewers be expected to know what the so-called experts think about anything.^[2]

Conservatives have gotten into the habit of dismissing "experts" as out-of-touch elitists who care more about promoting an ideology than objective truth-seeking. (Note: Does not apply to experts who endorse conservative views.) As usual, they're kind of right and kind of wrong, but Bozell is taking it a step further. Later in the same article, he asks:

Is it fair—whether the pollsters are liberals or conservatives—to expect the American people to identify correctly the estimates made by a panel of economists organized by news editors of The Wall Street Journal?

Is it fair to ask a difficult question? I'd like to think so, especially if your goal is to determine whether the answers to such questions have reached the public, which, nominally (if not actually), is what the study was about. Nobody's trying to deprive anyone of their fundamental right to disagree with the experts as obstinately and as unjustifiably as they see fit. Besides, I'm not sure it's even logically possible to disagree with the experts if you aren't, you know, informed about what the experts think.

So let's criticize the hell out of this study for all sorts of reasons, but not that one. I guess Bozell would be happier if this were just another opinion poll, but if you're not informed enough to answer at least a few questions about what the experts think, I'd rather not hear your opinions anyway.

1. Basically, if you went in with the attitude that everything liberals do is right and everything conservatives do is wrong, you would've given the "informed" answer to almost every question.
    The content of a lot of the questions was suspect too. This video probably isn't flawless, but it's a pretty good overview of some of the problems. (That's twice in just over a month I've praised something on one of the Breitbart sites, so…haha, [snarky comment about the frigid conditions in Hell, maybe a Michael Bloomberg reference and something about snow plows, etc., etc.])

2. Or, on the chance some thought they were being asked for their opinions, God forbid they be expected to read an entire sentence and evaluate the semantic contribution made by every single word.
    All that said, this isn't really about Fox News, because I have no doubt that, given an ideologically-neutral set of questions, those who get their news from other sources wouldn't fare much better. A follow-up article elaborating on that is in the works, but, for now, I'll just point out that the researchers threw in a token question for conservatives (about whether there was proof the U.S. Chamber of Commerce had been raising foreign money for Republican candidates), and, shockingly enough, on that one MSNBC and NPR viewers/listeners were the most "misinformed." That completely undermines the reported findings, doesn't it? How they could just shrug it off and proceed with their "Fox News viewers are misinformed!" publicity grab is beyond me, but I guess that's why I'll never climb to the top of academia.