Lizard people control public opinion research

As reported by Atlantic Monthly, 12 million Americans believe that lizard people run our country.  This finding is based on the survey question:  "Do you believe that shape-shifting reptilian people control our world by taking on human form and gaining political power to manipulate our societies, or not?"  As a friend put it, someone went to graduate school to learn to write that question.  

The lizard people finding is a variation on findings reported in David Hemenway's article "the myth of millions of self-defense gun uses", where he reports that about 4% of people report having personally met an alien from outer space.  Even in a nationally representative sample chosen perfectly, small measurement error can be magnified when extrapolated to the nation.

The different conspiracy theories vary in prevalence, from the 4% believing in the lizard people to 51% believing that JFK was killed by a conspiracy, so clearly they are not all due to random error.  

Some of the questions are obscure or old, such as about the government controlling minds with TV (15%), the CIA invented crack cocaine (14%), and fluoride in water is dangerous (9%), which could be from non-attitudes: poll respondents' expressing opinions about matters that they have no opinion about.  Pollsters didn't understand why respondents showed different levels of support for specific issues in repeated polling.  To test whether respondents were reporting opinions about issues that they didn't truly care about, they asked respondents about a fictional bill, and 30% of respondents gave an opinion about it rather than stating no opinion.  The phenomenon was called non-attitudes.

An issue where I'm not aware of research is how non-attitudes change with time due to exposure.  If the lizard people issue gets reported widely, it could gain more exposure and credibility, and more people may endorse the statement in the future.  Can public opinion polling change opinions and give credibility to fringe opinions? 

(After some googling, I discovered that the lizard people theory is not new: it was promulgated in 1999 by a former BBC sports reporter who also included ideas about how the Jews caused 9/11 and control the media, variations on the oldest conspiracy theory around.  And Wired reported it in 2007 in a list of conspiracy theories.) 

The question on the list with the most additional poll data is probably global warming:  37% in this poll said it's a hoax (the exact question was "Do you believe global warming is a hoax, or not?")  That seems on the high side.  Gallup polls has found that 15% say that global warming will never happen, but 40% of people say that the importance of global warming has been exaggerated, which fits in with dozens of other polls.  The word "hoax" is strong and implies that global warming won't happen, and yet the percent endorsing the question is more similar to the 40% on Gallup's question about whether the importance has been exaggerated, rather than the 15% saying that it will never happen.

Context effects could have increased the prevalence of extreme opinions:  when the survey is entirely made up of kooky questions with extreme wording ("hoax" versus "exaggerated"), people may be more likely to give kooky or extreme answers.  If just one of these questions had been included on an otherwise straight-up opinion poll, a lower percentage may have endorsed it.  So maybe 4% belief in the the lizard people isn't so high, after all.  

Turning the tables on survey research

Just got off the phone with the Gallup pollster. I am one of their ~1000 person sample on their current health survey. When they asked my profession, I said that I was a survey researcher with health topics, and we both laughed. The coincidence was uncanny.

They ask important questions like approval of the president towards the beginning, along with the 1 question health assessment. They asked if I felt like I was treated with respect at work yesterday, a couple of other working conditions questions, income, and hours, personal stock market investment, as well as pessimism/optimism about economic growth, stock market, personal investment, employment, inflation. Also asked contextual questions such as satisfaction with neighborhood quality, safety, commute time to work.

The questions were of course cursory, such as giving only 2 or 3 option answers (satisfied or not, sometimes a neutral option). No questions had branching options, so every question was asked to every respondent, so only missing data should be due to people hanging up in the middle. The stock market investment quantity was phrased on a logarithmic scale, so above or below 10k, and then presumably other questions.

A couple of the questions were surprisingly badly worded, such as this double-barreled question, "Are you satisfied with the price and availability of fresh fruits and vegetables in your neighborhood?" Lots of availability but high prices here in Dupont Circle.

Short questions require respondents to define the categories themselves. For instance, they asked how many days out of the last 7 days have I exercised. I don't consider walking to be exercise, even walking for 5 miles. I haven't lifted weights or swam in the past week, so I said that I haven't exercised at all. In retrospect, probably it would have been more accurate to say a few days since I have walked more than 5 miles a few times this week.

Two questions I would have been interested in is whether people have a pet, since that goes with the health theme --- people with pets may have better health situation, and everyone likes to hear about pets. Another question is whether single people have a steady romantic relationship since "single" is a big and diverse category, and having a romantic partner could affect health and health outcomes. Of course I would have also liked to know whether people had sex recently, and whether they have safe sex, but I know that Gallup wouldn't do that.

Altogether, it was a 14 minute survey, and they are gathering a dataset that could furnish many papers on the connection between working conditions, attitudes towards the broader economy, political attitudes, and both mental and physical health. I wonder whether they will use the data for that purpose, or if they will just do the univariate and basic bivariate summaries. For sure, I'd appreciate the chance to analyze the data, if I could ever find the time.

Do people report embarrassing facts?

I've done some work in whether people report facts about themselves that are sensitive or embarrassing, and it seems that generally people report a surprising amount of sensitive information. Something that I've found fascinating is how people will reveal sensitive information when it's not called for on the humor blog "Damn you autocorrect". Usually, the cases are innocuous or not-so-innocuous, but occasionally the autocorrect will falsely "reveal" information (by correcting text to something not meant) that will induce the correspondent to reveal information that they would not have otherwise said.

Here are the examples:

• Person reveals herpes infection.
  
• Girlfriend reveals interest in breaking up.
  
• Mother reveals that parents suspect son is gay.
  
• Person reveals that she's heard her friend is well-endowed.
  

Only 4 examples and presumably it doesn't always "work" (that is, reveal information that would otherwise be hidden and not revealed on a survey), but I wonder if this could be harnessed somehow as a survey device.

HIV prevalence in Chicago

A new Chicago Dept of Public Health (CDPH) study finds sobering results about HIV prevalence among men who have sex with men (MSM): of those sampled, 30% of black men, 12% Latino, and 11% of white tested positive.

There's a statistical twist: the reports of the study only give the sample size 570 which is almost irrelevant compared with how they chose these men. Since the detail wasn't given I would guess that they went to stereotypical places, so results would be more accurately reported as being in terms of MSM who go to the stereotypical hangouts where CDPH knew where to find them. I would expect these men to have more sexual partners, more high risk behavior, and so more likelihood of HIV and other STDs. Such high prevalence among any group is of course shocking, but if some omniscient being did a random sample of all MSM, they could pull the partnered, boring, reclusive, etc., and find lower prevalence.

Further, I wonder if their sampling method is differentially selective for higher risk in some groups than others. Gay black men don't have a neighborhood in Chicago comparable to Lakeview, the predominantly white gay neighborhood in Chicago (my former neighborhood), so the more easily sampled may be the very highest risk. In Lakeview they could have found ice cream shops, laundromats, dog supply stores, and yoga classes where upwards of 30% of the clientele are gay; I doubt there's anywhere else in the city with such a high concentration in non-gay-themed gathering spaces. Even if they limited themselves to bars, it might still be that gay bars in Lakeview attract a much broader cross-section of the Lakeview community than African-American gay bars on the South/West side of the city.

Also there is the phenomenon of small numbers: the smaller the group you are drawing from, the more extreme the numbers get. Just two days ago I was estimating chlamydia prevalence among different subgroups of Evangelical Christians and I found a higher prevalence among a specific evangelical denomination of the more frequent church attenders than among evangelical frequent church attenders in general. Are people who have chlamydia attracted to this denomination? Does this denomination discourage condom use more than others? It's possible that was a real effect, and perhaps it will get its own paper, but it's also possible that the higher prevalence was because I was basing my estimate on only 73 individuals, versus 700 for the highly religiously involved.

Youth Pride Services, an agency that serves the African=American gay community in Chicago writes a letter to the editor of the Windy City Times (gay newspaper) saying that they have little visibility. Sadly, notification of this high prevalence may not bring them visibility either.

When qualitative research is better

I got several phone calls from the same market research company and finally agreed to take the survey for a technology company. They were interested in finding out how companies decide how to purchase technology products, so they asked about 20 screening questions to find out whether I have any role in purchasing and choosing technology, and wanted to follow up with an internet survey about how I choose technology. The screener questions alone took 7 minutes. Who knows how long the internet survey would take, so I turned it down.

Now my role in choosing technology at work is that I use my personal laptop at work and I bought a $150 printer at the beginning of the year and that's it. Clearly not what they have in mind. If they asked me to explain my role, I could explain the printer buying decision in 3 minutes, they could pay someone to code all the responses, and that would be all. They would get better information, and probably get better response rates.