Facts, Statistics & Bias

Facts, Statistics & Bias

Exploring Statistical Information

"There are three kinds of lies: lies, damned lies and statistics." In his autobiography, Mark Twain attributes this remark to Benjamin Disraeli (quoted in Columbia Dictionary of Quotations, p. 870). To some degree this may be true - people often use the same statistics to try to prove two different sides of the same issue. This week we'll see where statistics come from, how to find them and how to evaluate them.

What are Statistics?

Two Definitions of Statistics:

1) Science that deals with the collection, classification, analyses and interpretation of numerical facts or data.

2) Actual facts or data

Are you a statistic?
There are a lot of data about you out there:

Social Security Number
Zip Code
Area Code
Height
Weight
Library barcode number
Grade point average

Statistics are used to either prove and support research or to refute it.

TWO USES OF STATISTICS:

1) Descriptive statistics is the use of numbers to summarize information that is known about some situation

An Example: A random sample of students was taken in a large university campus in the city of Providence, Of those questioned, 15% used Five Star notebooks for their course note taking. This summary of the data taken explains that of the 140 students asked, 21 used the Five Star brand of notebooks

2) Statistical inference is the use of numbers to give numerical information abut larger groups than those from which the original raw data were taken.

15% of university students in New England capital cities use Five Star Notebooks. This is an example of using the descriptive statistics to IMPLY that students in New England Cities use Five Star notebooks. We would be using this number to INFER something from this sample about a larger population for which we do not have complete information.

Organizations

Professional/trade & industry/special-interest organizations

For almost any topic imaginable, there exists a professional, trade, or special interest organization.

An example of a professional organization would be the American Medical Association, whose members consist of doctors. The AMA provides professional information and networking opportunities to its members, educates the public and policy makers about health issues, and advocates for the interests of the “medical community.”

An example of a trade organization would be the National Restaurant Association. This group is made up of individual restaurant owners and restaurant chains. Their goal is to represent, educate, and promote the restaurant industry and to advocate for governmental policies favorable to the industry.

An example of a special interest organization would be the Surfrider Foundation, which is a “non-profit organization dedicated to protecting our oceans, waves and beaches.” Other examples would be the National Organization for Women, the NAACP, the American Civil Liberties Union (ACLU), Greenpeace, or Save the Bay.

Professional, trade, and special interest organizations are usually excellent sources of information on topics related to their missions, and sometimes they are even the primary source of information on that topic. These organizations are increasingly making valuable information available on their Web sites, although the information they are willing to provide for free to non-members is often limited. It is also important to keep in mind that these organizations, by definition, have as an agenda to promote their profession, industry, or cause, and that the information they make available will likely be compatible with their mission.

We often find associations quoted in the press as experts on a certain industry or topic.

For example:

On November 24, 1999 the Washington Post published an article on the annual survey on toy safety by the U.S. Public Interest Research Group (U.S. PIRG) entitled “Trouble in Toyland.” View the sites listed below to see the PIRG findings and the response by the Toy Manufacturers of America. Each group defends its position armed with facts and statistics.

State PIRG's report "Trouble in Toyland" (read the Summary and Findings)
http://www.toysafety.net/report.html

Toy Manufacturers' response:
http://www.toy-tma.org/industry/news/
statements/hazardous.html

TMA's resources on toy safety:
http://www.toy-tma.org/industry/safety/index.html

Other statistics from TMA
http://www.toy-tma.org/industry/statistics/index.html

Finding an Association

So how do you find an association or organization related to a particular industry or issue?

Encyclopedia of Associations - three print volumes. A directory of orgabizations and associations indexed by title and keyword. Entries show contact informations and the scope of the Organizations' activities.

Online equivalent -Associations Unlimited, enter this database, and do a subject search for “toys” and find the record for Toy Manufacturers of America, Note the variety of other toy-related organizations that come up.

Finally, note how not all associations make information freely available to the public like TMA did. Many limit information to “members only.” An example:
http://www.iaapa.org/ and their section labeled “Information Services.”

Where do statistics come from?

Government agencies (U.S. and foreign)

The Census Bureau
The Bureau of Economic Analysis
The Bureau of Labor Statistics
National Center for Education Statistics
The Bureau of Justice Statistics
National Center for Health Statistics
The Bureau of Transportation Statistics
and so on…

Inter-Governmental Organizations (IGO’s)

The United Nations
the World Bank / International Monetary Fund
OECD (The Organization for Economic Co-operation and Development)

Professional / trade & industry / special-interest organizations

The American Medical Association
The American Library Association
The American Marketing Association
The American Bar Association
The American Psychological Association
The Beer Institute
Toy Manufacturers of America
American Apparel and Footwear Association
Amnesty International
The American Civil Liberties Union (ACLU)
The National Organization for Women
The NAACE
Greenpeace
Save the Bay

Studies, Polls, Surveys, Misc. Research

Tend to focus on specific topics, e.g. attitudes of high school students toward drug use, should the death penalty be abolished, the popularity of the President of the United States, etc.
Usually done at universities or research institutes, or by opinion research firms.
Results are reported in scholarly publications, like journals, and, in some cases, in the popular media.

Strategies for Finding Statistical Information

When looking for statistical information, we should adopt a similar strategy to the one we used when searching for information on the Web: we should ask ourselves, “Who would collect this information?”

For example, if we were looking for statistics on how many people were put to death in the United States last year under the death penalty, where might we look?

First, we should ask ourselves, “Who is involved in this activity?” Well, “state governments” would be a start, as the death penalty is administered at the state level. But we don’t want to check every state.

“Who might keep track of the country as a whole?” might be our next question. “The federal government,” would be a correct answer. Next, we would want to know “what part of the federal government? Is there an agency associated with law enforcement?” The answer to this would be the Department of Justice, and, more specifically, the Bureau of Justice Statistics. While we might not know about this agency off the top of our heads, as long as we’re thinking along the right track, it’s not too difficult to find out the name of the agency.

Another possibility for finding this statistic would be an organization either in support or opposed to the death penalty. They would be sure to have the statistics, although we would want to watch for possible manipulation in how they reported or used them. To identify such an organization, we would use a source like the Encyclopedia of Associations.

Finally, we might be able to find up-to-date statistics in a newspaper or magazine article about the death penalty. We already know how to search library reference databases to find citations.

An excellent place to start…

1. One of the best places to start when you’re looking for statistics is the Statistical Abstract of the United States, published annually since 1879 by the United States government. The Statistical Abstract presents statistics on all different topics from many different sources. Chances are, you will find what you are looking for, or close to it, there. The latest annual issue is located at the Reference Desk in Kingston; older issues are on the Lower Level in the Serial Stacks. It can be found on the Web at http://www.census.gov/statab/www/

2. The Statistical Abstract can also lead you to other statistics sources. At the bottom of each statistical table in the Statistical Abstract, you will find a reference the original source of these statistics. By consulting the original source, you will most likely find additional statistical information on that topic.

3. Another useful source to consult is Statistics Sources [at Ref. Z7551 .S84], published annually. When you look up a topic alphabetically, this book will list different published sources of statistics on that topic.

4. Finally, as noted above, statistics are frequently reported in newspaper and magazine articles. By consulting a reference database such as Research Library or Academic Universe and doing a search for your topic, chances are you will find statistics in at least some of the articles you find.

Some Quick Strategies for Finding Statistics

Use the HELIN Catalog. Search by Subject and add the subdivision, statistics (e.g. women statistics).
Use a periodical index. Search the index by subject (e.g. breast cancer statistics).
Use an index to statistical sources (e.g. American Statistics Index)
Use the World Wide Web
Use a guide or bibliography of suggested resources.

EVALUATING STATISTICAL INFORMATION

Being able to evaluate statistical information is important, as statistics can be manipulated so that they tell the story that the person using them wants to tell. We have all heard how it is possible to “lie” with statistics.

However, evaluating statistical information can be a very difficult task, and in some cases is something that only experts or others with advanced knowledge of statistics know how to do, as it requires an in-depth understanding of the data-gathering process and the mathematical techniques used to generate the statistics. [This is something you’d learn a bit about in an introductory statistics class, which is a really good idea to take at some point, as this is very useful information!]

Nonetheless, with some common sense, everyone can learn to be wary and critical of the numbers they find and can hopefully avoid being “duped” by statistics.

Here are a few general points to keep in mind when working with statistical information:

Who collected the data? Do they have a particular interest in the results?
Do the statistics show any bias?
Are the data timely? Check the time period when the data were collected.
Is the coverage complete? What was the population, or sample size, of the study? For example, the sample size of the decennial U.S. Census of Population and Housing is theoretically the total number of people in the United States. The sample size for a most other studies, surveys, and polls will be much smaller.
Has the data been “re-packaged”? For example, government data published by a private source might not be as complete.
Is the data from a primary source? If it is from a secondary source (for example, the Statistical Abstract), has it been properly documented so that you can find the primary source?

This course was developed by Joanna Burkhardt, Mary MacDonald and Andrée Rathemacher and was adapted for online use by Jim Kinnie as part of the URI Libraries Plan for Information Literacy - http://www.uri.edu/library/instruction_services/infolitplan.html