Notes on Bad Science (or even Pseudoscience)

Based on Lecture by Jack S. Damico

A quick review of the lecture (and power point) provided:

Good Science: a body of knowledge, belief, methodology, or practice based on evidence (observational, theoretical, qualitative, or experimental) organized by logic to reach a conclusion (or hypothesis or theoretical formulation) that is then tested by trying to prove it wrong.

Above all, Conclusions are tentative – open for revision

Hypotheses are framed so they are capable of being disproved

Alternative ideas are confronted by systematic observation and analysis (experimental or

qualitative)

Skeptical scrutiny is not opposed

Conspiracies are not the reason for lack of advancement

Beware:

A tendency toward paranoia

Purveyor considers him/herself a genius and others not capable

Purveyor sees her/himself unjustly persecuted and discriminated against

There is a strong compulsion to focus attacks on the greatest thinkers or

scientists

Science adopts an attitude of appreciation of human imperfections and fallibility. This is

why results and ideas are tentative.

The purveyor works in concert with his/her colleagues (not in almost complete isolation)

Bad science: a body of knowledge, belief, methodology, or practice that claims to be scientific but does not adhere to the tenets of science

·  Does not adhere to the scientific method

·  Lacks supportive evidence

·  Focuses on resemblances rather than cause-effect relations

·  Lacks scientific plausibility

·  Lacks scientific rigor or status.

As a result it often employs vague, exaggerated or untestable claims, over-reliance on confirmation rather than refutation, lack of openness to testing by other experts, and a lack of progress in theory development

Bad science tends to coalesce around the following characteristics:

·  There is a tendency to note that with creators or advocates of the method/idea that “Wishing makes it so”

·  Bad science is embraced in exact proportion as good science is misunderstood

·  Absence of critical thinking

·  There is a tendency toward dogmatism

Note that this demarcation is not a clean duality; we are dealing with a continuum when dealing with knowledge not a binary choice. Also, this continuum of good versus bad science is typically accompanied by another in the form of strong versus weak scientific competence. Consequently, one must strike a balance between the two poles of the continua

Some have employed a TOOLKIT to assist in ferreting out the differences:

When possible there must be independent confirmation of the “facts”

Encourage substantive debate on the evidence by knowledgeable proponents of all points

of view

Arguments from authority carry little weight

Spin more hypotheses, explanations…multiple working hypotheses

If there is a chain or argument, every link in the chain must work (even the premise)

Apply Occam’s Razor: when faced with two alternatives, the simpler explanation is

usually the best (e.g., law of large numbers)

Can this idea be falsified?

______

A CHECKLIST TO DEMARCATE

A “yes” answer to the question moves toward bad science:

·  Does the claim lack the necessary theoretical characteristics?

Remember: Theories have the following characteristics

Arise from and supported by systematic observation

Must be specific enough for falsification

Must make specific testable conditions

Must be tentative

·  Does the claim lack acceptable support?

Remember: Defense by authority is not an acceptable support mechanism

·  Does it sound far-fetched or too good to be true?

Remember: a claim should make sense given your knowledge of conventional science…extraordinary claims need extraordinary evidence

·  Does the claim only (or primarily) come from a source dedicated to supporting it

Remember: Science starts with a null hypothesis and searches for evidence; it does not start with a positive hypothesis and then supports this hypothesis with questionable evidence and anecdotal reasoning

·  Are claimants guarded about evidence or procedures employed?

Remember: Science seeks to be open with methods and data so refutation can

occur.

·  Are the available supportive data of questionable quality?

·  Is the claim typically announced by (or in) the mass media rather than supportive

professional mechanisms?

·  Is there concern by purveyors about suppression by “authority”?

·  Is the claim oriented to, or intended to show there is something wrong with the norm?

______

ASSIGNMENT

For next class please identify and investigate a practice, intervention, issue, or concept that seems suspect in the field of speech-language pathology and determine (using the checklist and “tool kit” discussed in class and on this handout) whether it is a based on good science or bad science (and why).

Prepare a one page summary of your work and be prepared to discuss and defend your answer next week.