One of the people I follow on Twitter linked to a recent Gallup Poll that claims “Americans’ Views of the TSA More Positive Than Negative.” After clicking through and studying the results of the poll and how the company made that claim, I was appalled at how misleading the results were presented. Stick with me here and you’ll see why.

Here is the first paragraph from the Gallup poll:

They have less positive views of TSA’s effectiveness at preventing terrorism

by Frank Newport and Steve Ander

PRINCETON, NJ — Despite recent negative press, a majority of Americans, 54%, think the U.S. Transportation Security Administration is doing either an excellent or a good job of handling security screening at airports. At the same time, 41% think TSA screening procedures are extremely or very effective at preventing acts of terrorism on U.S. airplanes, with most of the rest saying they are somewhat effective.

If you’re an American over the age of 18 and you are wondering why no one from Gallup called you and asked you a bunch of questions about the TSA, you’re not alone – more than 237,700,000 of your fellow adult Americans didn’t get a call from Gallup either – because only 1,014 of you completed the telephone survey, or 1 out of every 235,400 of you.

Now let’s move on to the questions asked to these 1,014 people during the Gallup telephone survey:

Thinking now about the TSA, the government agency that handles screening at U.S. airports, do you think the TSA is doing an excellent, good, only fair, or poor job?

If someone called me up and asked me that question I’d feel like I had to ask them to define the meaning of “good job.” The question is so subjective and there are so many ways each individual person surveyed would think of something as being a “good job” that the question itself simply invalidates the value of any answer given.

Next question:

How effective do you think the TSA’s screening procedures are at preventing acts of terrorism on U.S. airplanes?

Now there’s a question where our answers and opinions can be backed up by facts… wait a minute, how many terrorism attempts have been thwarted by the TSA’s procedures? Are there any published cases or concrete examples? No? Does this mean terrorists have been deterred by the TSA screening procedures? We don’t know, so our opinions have no basis in fact.

What’s the value of an uninformed opinion, or, an opinion with no basis in fact?

Here are the real facts about this Gallup poll:

1. The headline “Americans’ Views of the TSA More Positive Than Negative” is what’s known to critical thinkers as a Hasty Generalization, or, a generalization based on an unrepresentative sample. Quite simply, the opinions of 1,014 American adults is not a large enough sample size to reflect the opinions of over 230,000,000 of their peers.

2. The statement made in the first paragraph of the analysis at the Gallup site: “Despite recent negative press, a majority of Americans, 54%, think the U.S. Transportation Security Administration is doing either an excellent or a good job of handling security screening at airports.” Is highly misleading and even more of a hasty generalization than the headline itself. It’s designed to appear as a statement of fact, when in actual fact, it’s based on both dubious sample size and…

3. …questions which are so broad in their subjectivity that any data extracted from virtually any sample size will never give conclusive answers. What are the criteria for doing a good job? Have the TSA procedures stopped terrorism? I know it’s called an ‘opinion poll’ for a reason but, what value do highly subjective and uninformed opinions really have, either individually or in aggregate?

4. Question: Who commissioned the poll? It doesn’t say who paid Gallup to formulate the questions, the scoring methods, write the scripts, make the calls and analyze the data. Gallup is a paid service, with clients, so who paid for this survey to be done?

5. A common practice used by mainstream media and propagandists is to use a headline to convey the message they want you to see, reiterate that message in the first paragraph and provide one or two facts, the truth, at the very bottom of the article. Notice how the article at the Gallup website makes the statement they want you to believe in the headline and first few sentences and at the very bottom, in fine print I might add, the sample size, margin of error and other details, and never mentions who paid for the poll to be conducted.

Look, anyone, any company or organization can hire Gallup to call a few people to get their opinions on something. But those poll results can easily be skewed, even by something as seemingly innocent as the tone of the person’s voice asking the questions. These polls, when cited by the media, mentioned by polititions or parroted by a guy talking politics at the local coffeeshop, are essentially worthless.

The next time you see or hear someone citing the results of a poll and implying that the results are the opinion of the majority of Americans, just remember, polling is big business and poll results are based on the uninformed opinions of a ‘majority of Americans’ who would fit into your average school cafeteria.

Think, people. Think. Don’t let others do your thinking for you.

By Paul Short on 11/08/2012

Related Posts

• No Related Posts Found