Evaluating Scientific Research: Separating Fact from Fiction

2.5 avg rating
( 4 ratings by Goodreads )
 
9780130128454: Evaluating Scientific Research: Separating Fact from Fiction

For junior/senior-level courses in Research Methods, in departments of behavioral and social sciences, nursing, law, and public policy. *Acknowledging the profound role science plays in our lives, this text challenges basic assumptions and encourages readers to question all aspects of the research process. It uses non-technical vocabulary and a simple, straightforward style to explain the research process and cover six problem areas-limitations of science, preparing for research, measurement, research designs, data analysis, and philosophical issues. An emphasis on creative and critical thinking coupled with discussions of many controversial issues make this approach inspirational to students planning to conduct original research or anyone wishing to develop the ability to evaluate research. *Discusses a number of issues associated with many types of research. The author explains how to identify improper methods in research and shows the student how to avoid those same mistakes.

"synopsis" may belong to another edition of this title.

From the Publisher:

Titles of related interest also from Waveland Press: Krathwhol, Methods of Educational and Social Science Research: An Integrated Approach, Second Edition (ISBN 9781577663331); Levy, Tools of Critical Thinking: Metathoughts for Psychology (ISBN 9781577663164); and Schweigert, Research Methods in Psychology: A Handbook, Second Edition (ISBN 9781577664154).

From the Inside Flap:

Preface

"Omission and simplification help us to understand-but help us, in many cases, to understand the wrong thing." Aldous Huxley

"Most people would rather die sooner than think. In fact, they do so." George Bernard Shaw

"What good fortune for those in power that people do not think." Adolph Hitler

Writing this book has kept me in a state of perpetual anger. While reviewing the literature in preparation for revising my how-to book on research methodology (Leavitt, 1991), 1 kept coming across books and articles that documented serious misuses of science. They persuaded me to change direction and write a much more critical book than was originally intended. Anger, like misery, loves company, so I'll be disappointed if readers don't come to share my emotion. Only if enough people become angry will changes be made.

I wrote with two groups of readers in mind. The smaller is comprised of college students enrolled in a methodology course who hope to do original research someday. Many texts for such courses present a large technical vocabulary and complex formulas for analyzing data statistically, but knowing such esoterica no more makes people scientists than knowing how to paint by numbers makes them artists. Good scientists think creatively and critically about the literature they read, the questions they ask, the conceptual foundations of various research designs, and proper inferences from those designs. I try to sensitize readers to such issues.

The second group is everybody else. Science plays a profound role in our lives, and decisions with far-reaching consequences are often based on scientific evidence. Yet, the evidence is often controversial or misleading. Medical and social scientists receive years of training in their specialties, but developing the ability to evaluate research is a considerably less formidable undertaking and does not require brilliance.

The fruits of scientific theory and research are most evident in technological developments such as computers and high-resolution television sets, but science has also had a huge impact on social, esthetic, and religious beliefs. Before Galileo, people believed that angels, heavenly bodies, beasts, and elements all had their place in a fixed hierarchy. His proof that the universe is not only much greater than previously supposed, but that it changes continuously, was so threatening that the Catholic Church condemned Galileo to house arrest. For the poet John Milton, Galileo's proof created "a wide gaping void that threatened even the angels with loss of being." Philosophy was changed profoundly, and even the rules of evidence of law were reevaluated.

In the 17th century, the ideas of Galileo, Descartes, Robert Boyle, and Kepler shaped metaphysics, political theory, and theology. Newton's gravitational theory provided a unified explanation of phenomena previously thought unrelated, a triumph that encouraged the belief that humanity's moral and political problems could be similarly solved. Social theorist Saint Simon urged that traditional Christianity be replaced by a Religion of Newton. About 150 years later, Darwin's evolutionary theory repudiated the descent of humans from angels and thus devastated Victorian England. Social Darwinists horribly misused the theory to justify slavery, master race theory, and other tyrannies of the well-placed over the needy. More recently, philosophers invoked quantum theory in debates about free will versus determinism.

The tangible side of biomedical research is most evident in procedures such as organ transplants and cloning, but biomedical research also influences quality of life and, by lengthening life span, changes family dynamics and societal norms. Scientists play a major role in the criminal justice system. They analyze crime scenes for blood, semen, and other physical evidence, and their findings often mean the difference between conviction and acquittal (Neufeld & Colman, 1990). Connors, Lundregan, Miller, and McEwen (1996) documented 28 cases of people convicted of serious crimes who used DNA tests on existing evidence to successfully challenge the jury verdicts. They had been sentenced to long prison terms and served, on average, 7 years in prison.

The outcomes of civil trials, including class action lawsuits involving millions of plaintiffs, often hinge on scientific testimony. Experts testified for and against the claims that silicone gel breast implants cause a variety of medical problems, that exposure to asbestos fibers causes lung disease, that the Dalkon contraceptive shield causes toxic shock syndrome, and that tobacco is both addicting and harmful.

The effects of social science theory, though subtle, are considerable. Freud left a substantial legacy. He introduced terms such as wish fulfillment, frustration, regression, repression, unconscious impulses, sublimation, anxieties, neuroses, and defense mechanisms, all now widely used in character descriptions. He gave people freedom to express interest in sexuality and question Puritan ethics. Freudian theory influenced modern literature, cinema, art, philosophy, and ideas about male/female differences.

Social science research has implications for allocations of taxes; many other aspects of public policy; decisions about potential candidates for public office; voting behaviors; television programming; teaching methods; the self-esteem of whole classes of people; and attitudes about dating, marriage, sexuality, and child rearing. Social scientists played an important role in formulating and assessing public policies on adolescent pregnancy. Information about the social and economic costs encouraged legislators to fund family planning clinics to prevent unintended teenage pregnancies, and evaluations of services for pregnant adolescents led policymakers to provide programs for reducing the negative consequences (Vinovskis,1989).

A social scientist's report that previous criminal convictions could be used to accurately predict whether a person would commit future crimes inspired legislation to sentence convicted criminals based partially on predictions of future criminality (Greenwood, 1982). This finding was relevant for "three-strikes-and-you're-out" laws.

Historian of science Bernard Cohen (1927) wrote, "Scientific ideas have exerted a force on our civilization fully as great as the more tangible practical applications of scientific research." And, in the words of psychologist Gustave Le Bon (cited in Seldes,1977, p. 404):

The sudden political revolutions which strike the historian most forcibly are
often the least important. The great revolutions are those of manners and
thought. The true revolutions, those which transform the destinies of people,
are most frequently accomplished so slowly that the historians can hardly
point to their beginnings. Scientific revolutions are by far the most important.

All that notwithstanding, some artists and musicians proudly proclaim their ignorance of science, and so do many college undergraduates majoring in psychology. They envision careers as psychotherapists, industrial psychologists, testing specialists, and so forth. If they take a course in scientific methodology, it is because their university requires it, not because research intrigues them. Their attitude is not just unfortunate; it is immoral. Literally hundreds of therapeutic techniques are promoted, including ineffective and harmful ones. Many industrial innovations fail to improve morale or increase productivity, and many tests are invalid or biased. People in the helping professions who do not think critically about research have no way to distinguish the good from the bad. They are likely to harm their clients and bring disrespect to their field.

Studying science is not just for the college crowd. Bauer (1992, p. 147) wrote that it

is excellent training for the mind, much better than the classically prescribed study of Latin. When you study science in the right way, you learn about reality therapy; and that is worth applying to other things than science. Science can show that some things are quite definitely wrong; that knowledge is a much better guide than ignorance; and it can teach humility in posing endless questions to which we have no good answers.

Science at its best can be as esthetically and intellectually satisfying as the greatest masterworks of music and art. Greek astronomer Ptolemy wrote, "I know that I am a mortal, a creature of the day; but when I search into the multitudinous revolving spirals of the stars, my feet no longer rest on the earth, but, standing by Zeus himself, I take my fill of ambrosia, the food of the gods." Almost 2,000 years later, Nobel laureate Steven Weinberg (1994) wrote, "The effort to understand the universe is one of the very few things that lifts human life a little above the level of farce and gives it some of the grace of tragedy."

Unfortunately, scientific articles for professional audiences are written in a dry, technical language beyond the comprehension of most laypeople. The language intimidates and has done so for centuries. More than 200 years ago, Empress Catherine of Russia invited atheist philosopher Denis Diderot to visit her. Her advisors worried that he might corrupt his listeners but were unwilling to confront him, so they devised a plan. They told him that the famous mathematician Leonhard Euler had constructed a mathematical proof of the existence of God. Diderot was ignorant of mathematics but agreed to listen. Euler confidently said:

(a + bn)/n = x. Therefore, God exists.

Diderot, confused and embarrassed, asked for and was granted permission to return immediately to France.

Like Diderot, many people today may assume that what they cannot understand is correct. By so doing, they cede power to a coterie of scientists and science writers. Yet a substantial proportion of published research, with far-ranging ramifications for health and happiness, is flawed. Bad research (and improper interpretations of good research) has adversely affected many people's lives. In 1917, prominent psychologist Henry Goddard wrote that a high proportion of Jewish, Italian, and Russian immigrants to the United States were mentally retarded. His report helped inspire the imposition of strict immigration quotas, and a result was that desperate refugees fleeing Nazi Germany or Soviet-dominated territory during World War II were denied entry into the United States.

Evidence, probably misinterpreted, on the high costs of drug abuse provided the rationale for subjecting employees and job applicants to urine tests for illegal drugs (cf. Horgan, 1990). According to spokespeople for the National Cancer Institute, the U.S. Department of Health and Human Services, the National Breast Cancer Coalition, and the American Cancer Society (cf. Melbye, 1997; National Cancer Institute, 1996), frightening reports that abortions increase the risk of breast cancer (cf. Loose, 1996) are at odds with the evidence. Sales of dietary supplements, natural foods, and natural personal care items totaled $65 billion worldwide last year (Nutrition Business journal, 1998), even though critics argue that many of the health claims for the supplements are unfounded or misleading.

The book is divided into six sections, each dealing with a separate type of problem.

Definition and Discussion of Limitations of Science (Chapter 1). Although science transforms society and is the most effective tool ever invented for predicting and controlling events, historians and philosophers do not agree about what it is. Many successful scientists used methods that most of their contemporaries frowned upon. Questions about ethics, the existence of God, and the possibility of an afterlife are not within the realm of science.

Scientists often derive such accurate predictions from their theories that they seem to have penetrated ultimate reality. But modern philosophers assert that theories can never be proven and absolute truth is unattainable. Preparing for Research (Chapters 2 to 4). Reading and Evaluating the Literature. The scientific literature is enormous, rapidly growing, and often overwhelming. Not even the most dedicated expert can keep up with everything published in her field. In addition, many good studies are never published, and many published articles are never cited by reviewers of the literature. Many potentially important connections between published articles are never made. As reports of research make their way from scientific journals to textbooks to popular magazines and newspapers, findings are often distorted to make them appear more significant, glamorous, and indisputable than they really are. Conflicts of Interest. Scientific research is often a high-stakes enterprise, with certain outcomes considered far more desirable than others. But unlike lawyers who insist on the innocence of their clients, or salespeople who proclaim their products the best on the market, scientists may appear to laypeople as impartial seekers after truth.

Finding a Good Problem and Asking Good Questions. Good researchers pick solvable problems and ask good questions about them. Measurement Issues (Chapter 5). Many scientific controversies revolve around disagreements or misunderstandings about what should be measured. For example, clinics that treat alcohol abuse often use abstinence from alcohol as the sole criterion for success. But many "recovering" alcoholics develop problems with relationships, health, work, and other drugs. Research Designs (Chapters 6 to 16). This section, the longest in the book, is divided into subsections describing the rationale behind and appropriate procedures for conducting experiments, comparative and correlational research, case study analysis, descriptive research, survey research, archival research, and research using animal models of human conditions. There are also brief sections on computer simulations and qualitative research. Scientific researchers need specialized knowledge and technical skills, but intelligent nonscientists should have no trouble understanding the conceptual foundations underlying each research design. They should make the effort, because published research is plagued by statistical errors, inappropriate generalizations, and errors in logic. (The discussion of interactions and individual differences will help people evaluate the relevance of research findings to their own lives.) Data Analysis (Chapter 17). Eminent critics have argued for decades that the dominant method for analyzing statistical data, null hypothesis testing, is seriously misguided. The criticisms are presented and alternative methods suggested. Philosophical Issues (Chapter 18). Unsophisticated laypeople, unaware of the importance of control groups and other refinements, frequently misinterpret observations. For example, they commit the cardinal sin of drawing causal conclusions (X caused Y) from correlational data (as X changes, Y also changes). Textbook writers give examples of such errors to emphasize the virtues of the scientific method. Yet the writers generally ignore philosophical challenges to scientific inferences. For example, scientists observe and experiment to determine how one variable changes with another. Then they try to find the equations. But an infinite number of equations yielding an infinite number of different predictions can be found to fit any set of data.

The following reviewers provided useful suggestions: Susan H. Fanzblau, Fayetteville State University; Donna LaVoie, St. Louis University; and Mary McVey, San Jose State University.

Fred Leavitt
fleavitt@bay.csahayward

"About this title" may belong to another edition of this title.

Top Search Results from the AbeBooks Marketplace

1.

Fred Leavitt
Published by Prentice Hall College Div (2000)
ISBN 10: 0130128457 ISBN 13: 9780130128454
New Paperback Quantity Available: 1
Seller:
Ergodebooks
(RICHMOND, TX, U.S.A.)
Rating
[?]

Book Description Prentice Hall College Div, 2000. Paperback. Book Condition: New. Bookseller Inventory # DADAX0130128457

More Information About This Seller | Ask Bookseller a Question

Buy New
11.90
Convert Currency

Add to Basket

Shipping: 3.02
Within U.S.A.
Destination, Rates & Speeds

2.

Fred Leavitt
Published by Prentice Hall College Div (2000)
ISBN 10: 0130128457 ISBN 13: 9780130128454
New Paperback Quantity Available: 1
Seller:
Irish Booksellers
(Rumford, ME, U.S.A.)
Rating
[?]

Book Description Prentice Hall College Div, 2000. Paperback. Book Condition: New. book. Bookseller Inventory # M0130128457

More Information About This Seller | Ask Bookseller a Question

Buy New
28.84
Convert Currency

Add to Basket

Shipping: FREE
Within U.S.A.
Destination, Rates & Speeds

3.

Leavitt, Fred
Published by Prentice Hall College Div (2000)
ISBN 10: 0130128457 ISBN 13: 9780130128454
New Paperback Quantity Available: 2
Seller:
Murray Media
(North Miami Beach, FL, U.S.A.)
Rating
[?]

Book Description Prentice Hall College Div, 2000. Paperback. Book Condition: New. Never used!. Bookseller Inventory # P110130128457

More Information About This Seller | Ask Bookseller a Question

Buy New
29.35
Convert Currency

Add to Basket

Shipping: 1.51
Within U.S.A.
Destination, Rates & Speeds