9 Hours Overview of Research for Public Administration

9 hours overview of research for public administration

Hour / 1st Session
1 / Why do research?
What is research behavior?
What are the ethical constraints?
2 / What is a scientific knowledge base?
What is a research plan or design?
What are threats to research designs?
What does “cause” mean?
3 / How variables become observable and problems with rules of science?
What are the challenges in understanding how to do research?

Attribute v. variable
Norm for many not behavior of anyone specific
Science relies on observation and measurement
Nominal v. Operational

2nd Session- Methodology
4 / Questionnaires and surveys
5 / Field observations
6 / Records, documents, narratives and content analysis
Writing about methodology e.g. content analysis.
3rd session - Analysis
7 / Measuring and describing
8 / Measuring and inferring
9 / Measuring and predicting
Hour / 1st Session 3 hours
1 / A. Why do research in public administration?

Pure research is done to advance the knowledge of the sciences that public administrators rely on such as anthropology, psychology, sociology, political science and economics.
Applied research is done to improve the implementation of public policies, programs and management practices.

Republics require that elected officials be accountable to citizens for the public authority and money that representatives use.
All human efforts are flawed; therefore, representatives in a republic need reliable information on which to base their support or opposition for public policies and decisions.
a)U.S. universities and research institutes (also known as “think tanks”) are supported by public funds, interest groups, political parties and the media that want to promote or challenge public decisions. Research is one of the tools in that discussion.
b)In a republic the basis for decisions (in theory) is reason. To begin to reason in a legal/political arena there needs to be a standard for determining what will be accepted as “true”. This is similar to the rules in courts of law about what constitutes acceptable testimony and evidence.
c)Historically the standards for “true” in the applied and social sciences have been the same standards as used by the natural sciences of astronomy, biology, physics and chemistry.[1]
B. What is research behavior?
a)Social Science Process

(Adapted from Nachmais and Nachmais 2000 Research Methods in Social Sciences. New York: St. Martin’s Press.)
B. What is research behavior?
2. Policy/Program Evaluation

(Adapted from Posavac and Carey 2001. Program evaluation: methods and case studies. New Jersey: Prentice Hall)
** Evaluations take many forms. An assessment of needs tries to establish if there is an appropriate role for government. Other evaluations audit progress and/or examine outputs/outcomes to determining if funding support will be continued. Evaluations about ending funding are called “summative” evaluations. Regardless of evaluation agenda, setting criteria and developing instruments may occur before or after a plan has been approved by key decision makers.
C. What are the ethical constraints?

Participation of subjects is voluntary. No coercion
No harm to subjects. Researcher needs “informed consent” from participants.
Individual participation is anonymous (persons in study are not identified)
Individual participation is confidential (responses from individuals not provided outside of study).
Avoid deception and/or assure participants are debriefed.
Describe the steps that assure the collection and analysis of data is accurate.
Use appropriate tools and methods of study for issue under study.
Judiciously interpret data and findings. Don’t ignore negative findings, hide methodological limitations, or over state findings.
Be prepared to share methods and analytical approach with public.

2 / What is a scientific knowledge base?
Seeks information that is objective, systematically observed and valid.

Empirical- relies on trained and disciplined observation and experience.
Language of science is about variables. (group of attributes)
Transmissible or “inter-subjective”. Statements verified by others. There is fidelity in understanding among the researcher, fellow academics and/or consumers of the study regarding methods and analysis.
Tasks are to explore, describe, explain probabilistic relationships and make predictions that are reliable. Non-normative because it doesn’t seek to determine if outcome or action is desirable.
Information can be generalized. E.g. more relevant to demonstrate that simple guideline leads to more frequent compliance than to demonstrate that there is more compliance in Wisconsin than in Alabama.
Information leads to theory that explains many empirical generalizations and make predictions.
Theory is shorthand summary of why a relationship between variables exists. Empirical evidence identifies who, what, where, when and how.
Science focus is on patterns not unique non-repetitive phenomena.

What is a research plan or design? Plans include decisions about

How observations will be sampled. Sampling is either random (“R”) or non random
Making observations regarding study variables prior to introducing a stimulus. This pre-stimulus period is called the pretest. (Ox)
Managing the introduction of the stimulus or the experiment (X)
Making observations about variables after the stimulus period or what is called the posttest. (Oy)
Including a control group that participates in the pre and/or posttests but does not participate in the experiment.

The classical research design would be represented as follows:
(Study group) R (O1) X (O2)
(Control group) R (O3) (O4)
There are many research plans other than the classical design noted above. The simplest is the one-shot case study X (O1)
A complex study design is the Solomon 4
R (O1) X (O2)
R (O3) (O4)
R X (O5)
R (O6)
Researchers select a design that best suits their situation but try to avoid designs with too many threats to the validity of the study, such as the one-shot case study.
What are threats to research designs?
A threat to the validity of a research effort is any situation that can introduce a plausible alternative reason for the changes in the outcomes (dependent variable) other than what the researcher identifies as the cause or independent variable.
The classical research design enables the researcher to avoid a number of threats to validity such as:

History – events outside of studying influencing subjects in the study thereby distorting influence of independent variable.
Maturation – outcomes that occur because subjects are getting older, tired, or changing in ways unrelated to the study.
Testing – outcomes that occur because subjects become sensitive to the data collection instruments used by the researcher. (Instrumentation is another threat to the validity of a study but is not resolved by changes to the research design)
Statistical regression- selecting extreme situations or subjects for study.
Selection bias – when subjects in O1 significantly different from O3
Experimental mortality- when researcher loses contact with participants during course of data collection e.g. between pre test and posttest.
Causal time order- not being able to clearly indicate that the cause (independent variable) preceded the effect or dependent variable
Lack of adequate isolation between the control group and the subject group resulting in sharing of information (diffusion), rivalry, compensation and/or demoralization.

What the term cause means in science:

The independent variable precedes the dependent variable
The independent variable co-varies with the dependent.
There is no other explanation for the changes in the dependent but the impact of the independent.

3 / How variables become observable and problems with rules of science?
Part One: Introduction – Define the Problem

Explain the purpose of the study and state the questions to be answered.
Define the terms
Provide background information. This typically includes who thinks there is a problem, when was it first identified, where does it exist and how is the problem defined?
State the hypothesis***
Set the limits of the study.

Part Two: Literature Review

What are the current and “classic” readings one needs to understand the scope of the topic under study.
What have past scholars, practitioners and decision makers said and how have they studied the problem? Where do they agree and where do they disagree?
Given areas of agreement and disagreement what contribution is needed? How does this study contribute to the issues facing decision makers or fellow scholars?

Challenges:

Understanding the difference between an attribute and a variable.

Gender is a variable that is comprised of attributes male and female; Age is a variable and young, middle and senior or10, 30 and 72 are attributes.
Research questions may use attributes but research hypothesis use variables. For example a researcher may be interested in examining why there is a disparity between the wages of men and women.
The hypothesis (H) would test if gender were associated with wages.
The null hypothesis (NH) would be that the relationship between wage differences and gender vary at random.

Research Fallacy :

What is true for a group may not be true of everyone in the group. Women bear children between the ages of 15-45. However not every women has had a child during that period of her life.

Measurement.

If a variable exists it can be defined. If it can be defined it can be observed and if it can be observed it can be measured.
Social science cannot examine a phenomenon that cannot be observed but social scientists cannot always adequately measure the phenomena they are observing.

Sampling is the process of selecting the subjects for study so that every subject has an equal probability of being a participant. Subjects possess attributes that researcher define into variables.

Concepts (e.g. helpful) need to be defined. These explanations are called “conceptual definitions” e.g. identifies where a need or deficiency exists, provides information or resources to address deficiency without draining resources from person/s in need.

Concepts are studied or examined as “operational” and having a way to be measured in the real world.

a) Direct measurement is when researcher applies a measuring tool to a variable.
b) Indirect is when the researcher asks the subjects or observers to report on the variable.
c) Construct variables are a cluster of empirical attributes that collectively represent a variable. E.g. washes dirty dishes and returns them to cupboard, picks up objects from floor and places them in storage area, gives information to persons on request, approaches persons and offers information, direction or resources, reads information and organizes it for quick retrieval, etc., might be operational elements for examining the variable “helpful”. The researcher performs research tasks at the operational level.

Discussion Exercise

Sally Simple works at a local elementary school that has implemented a new reading program that requires 100 additional hours of language exercises each academic year for all grade levels.

A recent study by educational psychologists indicated that language exercises encouraged students to read more books and helped them remember what they read.

Sally randomly samples 200 teachers and parents. She asks, “Is the new language program improving the reading skills of your children?” The survey responses were:

YesNo Don’t know

Sally reports that the new program is improving the skills of children at her school.

Questions:

Has Sally defined the problem?

Does Sally know what to study regarding the new program and how to study it?

Is Sally’s behavior ethical?

In what ways has Sally contributed and/or failed to contribute to our understanding?

What research design has Sally used?

Are there alternative plausible explanations for the perception by 80% of those surveyed that said the new language program improved the reading skills of their children?

Did the new language program cause a change in reading skills?

What does Sally actually know based on her study?

[1] Post modernists and feminists scholars extensively debate these standards. However, for the purposes of this discussion the modern, positivist view will be initially discussed.