Quasi Experimental Research Definition and Designs

3457
Charles McCarthy
Quasi Experimental Research Definition and Designs

Introduction

Quasi-experimental research comes from the educational field, where the investigation of certain phenomena could not be carried out following experimental procedures (Campbell and Stanley, 1966).

In recent decades, they have acquired great prominence in applied research.

Characteristics of Quasi-experimental Designs

Used in situations where subjects cannot be randomly assigned to various conditions.

Less control over the effects of VVEE than in experimental designs.

Whoever investigates can… (Remember the MAX-MIN-CON Principle, topic 3)

-  MAXimize differences in VI (the primary systematic variance).

-  MINimize error variance.

The researcher cannot control for the secondary systematic variance caused by threats to internal validity.

The research staff do not have the resources to control the formation of the groups. The rule for assigning subjects to groups is not random and, in most cases, it is not known. One of the main tasks of the researcher is to identify possible threats to internal validity in order to take them into account or neutralize them. You will have to separate by other means the effects due to the LV from those due to the VVEE that can influence the RV (the more extraneous factors are ruled out, the more valid the study will have).

Measures the effects of treatment on RV (there may be one or more treatments).

It shares the logic of the experimental paradigm that implies that, in order to establish causal relationships, the following conditions must be met:

1) LV must precede RV.

2) There must be covariation between the variables.

3) It should be possible to rule out alternative explanations. (This is the point that raises the most problems in this context, but it is crucial to be able to establish causal inferences)

The research is carried out in situations where the behavior under study tends to occur naturally, which is why it tends to have greater external validity (although, as we have seen, if internal validity is weaker)

They are very useful in the evaluation of psychological or social intervention programs, to improve their planning and control.

They serve to evaluate the effectiveness and efficacy of programs in various areas: health, education, well-being and other social services.

Classification of Quasi-experimental Designs

Pre-experimental designs:

a) From a single group, only with post-test measure.
b) Only with post-test measurement, with non-equivalent control group. c) From a single group with pretest + posttest measurements ...

Quasi experimental designs:

a) With control group:

- With non-equivalent control group ...

o Pretest + posttest.

o Cohorts.

- Discontinuity in regression.

b) Without control group:

- Repeated treatment.

- Withdrawal of treatment with pretest + posttest. c) From interrupted time series:

- Simple.

- With control group with equivalent.

Pre-experimental designs

They serve to approximate the phenomenon under investigation (to generate hypotheses), without forgetting that, in the interpretation of the data, there may be numerous VVEE that would lead to an erroneous attribution of the effect of LV on RV.

They do not usually allow reasonable causal inferences to be drawn. See exceptional examples of the Virtual Course
They represent the basic modules from which the rest of the quasi-experimental designs are configured..

  1. a) Single group designs, only with post-test measure: X O
    - Lacks control, so causal inferences cannot be drawn.
  2. b) Designs with only post-test measurement, with non-equivalent control group: - Without pre-test, we cannot know if the differences between the groups (post-test) are due to treatment or differential selection (non-equivalent groups). are interpretable in terms of causality.
  3. c) Single-group designs with pre-test + post-test measures: O1 X O2- We cannot assure that the changes (post-test) are due to the effect of the treatment.
    - There are numerous threats to internal validity: history, statistical regression, maturation, test administration, instrumentation ... The more we discard, the more we can consider that the design is interpretable (very difficult).
    - Useful to suggest hypotheses for future research. (More "rigorous")

Quasi-experimental Designs with Control Group

Non-equivalent control group designs

  • The group assignment rule is not known because it works with groups already formed.
  • Although working with groups already formed, the researcher will try to select groups that are as equivalent as possible, trying not to influence VVEE that endanger a unambiguous interpretation of the results..
  • LOGIC: If the treatment has taken effect, the differences between the groups (post-test) will be greater than those that might initially exist between them (pre-test).

Pretest-posttest design with non-equivalent control group

One of the most used in Social Sciences.

Similar to the two-group experimental design although, here, the groups are already formed
(They do not have the initial equivalence that is obtained by randomly assigning subjects to groups). The pretest measurement will indicate the existing differences before introducing the treatment.

The assignment of treatment to one or the other group is controlled by the research staff..

We can find various patterns of results on which we must analyze THREATS TO INTERNAL VALIDITY

Cohort design

COHORT: Group of people who belong to some type of institution (formal or informal) subjected, for a period of time, to the same circumstances. They change from one level to another in these institutions. Very useful in research in the educational field. First class of students undergoing an educational reform

ADVANTAGE:

-  Being able to study how a certain event affects a group (experimental cohort) and compare it with another that did not experience said event (control cohort).

-  Easily affordable groups, even if the equivalence achieved with randomization is not reached (the threat of selection cannot be definitively ruled out). It allows to establish reasonable causal inferences (always evaluating if there are effects of VVEE that interfere with the results).

Discontinuity designs in regression

ADVANTAGE: It allows stable causal hypotheses with more guarantees because the group assignment rule is known (high degree of internal validity). People go to one group or another depending on the pretest (C = Cut-off score from which some people are assigned to the control group and others to the experimental group).

Its name comes from calculating a regression line from the scores obtained (pretest + posttest).

Fig. (A): Treatment without effect.
Fig. (B): Treatment with effect (discontinuity from cut-off point “C”).

As a pre-test measure, it must be possible to apply a continuous variable (depending on which groups will be formed).

In practice, it is rarely used because assignment to conditions cannot always be done according to a single criterion. It is usually used in ...

  • -  Education: To assess educational systems.
  • -  Medicine: To test the effectiveness of a medicine.
    Knowing the allocation rule allows ruling out some THREATS to internal validity, others such as the interaction between treatment and maturation, not.

Quasi-experimental designs without a control group

Sometimes it is not possible to have a control group (for practical and / or ethical reasons). Medical treatment

They are less powerful to justify causal inferences than control group designs (lower internal validity). Some are more suitable as part of more complex designs than as stand-alone designs.

Treatment withdrawal design with pretest and posttest O1 X O2 O3 X O4

With the withdrawal of the treatment, the researcher tries to create conditions that exercise the function of the control group. Experimental sequence: O1 X O2 // Control sequence: O3 X O4 (Police surveillance in a troubled neighborhood)

It can only be applied when the initial effect of the treatment is transitory.

Effective treatment: differences between O1 and O2 in the opposite direction to the differences between O3 and O4.

Possible effects of the treatment between O2 and O3 (even if it is still present):

  • Increase.
  • Maintenance.
  • Decrease.

After withdrawal of treatment there should be a noticeable change.

PROBLEMS it presents:

Large samples and very reliable measurements are needed.

Sometimes withdrawal from treatment can raise ethical issues and / or frustration.

High experimental mortality can occur.

Observations should be made at equal time intervals to monitor for possible spontaneous linear changes that take place within a given period of time..

Repeat treatment design O1 X O2 X O3 X O4

There is only one group in which the research staff introduces, withdraws and reintroduces the

treatment, at different times. Treatment application has to correlate with changes in RV. It can only be applied when the initial effect of the treatment is transitory..
More interpretable results:

-  O1 differs from O2.

-  O3 differs from O4.

-  O3 - O4 differ in the same direction as O1 - O2.

THREATS.

-  Cyclical maturation (differences due to the time of registration and not due to the effect of the treatment): Possible.

-  History: Unlikely. It would be very coincidental if an external event produced the same change as the introduction, withdrawal and reintroduction of treatment.

Interrupted Time Series Designs

They take various measurements of the RV, before and during treatment. Periodic records are made.

For its analysis, it is necessary to know when the treatment is introduced. If effective, subsequent observations will show a change in the series.

Frequent in:

-  Social studies.

-  Educational studies.

-  Health studies.

-  Program evaluation.

SIMPLE design of interrupted time series

O1 O2 O3 O4 O5 X O6 O7 O8 O9 O10 (Methadone)

  • -  Requires only one group.
  • -  It is similar to the pretest-posttest design, but several measurements are taken before and after introducing the treatment.
  • -  The effect of the treatment is appreciated by the variation in the trend of the RV. If the treatment is effective, this trend will change.

Possible THREATS to internal validity:

-  Maturation: Can be controlled by evaluating the pretreatment trend (representing the observations

prior to it).

-  Cyclical changes: The study of the time series makes it possible to detect and control their effect (advantage over a

pretest-posttest design in which it could occur covertly).

-  Statistical regression: It can be controlled by studying the trend before.

-  Instrumentation: Can be controlled using always the same registration procedure.

-  HISTORY: It is the most difficult threat to control. It could be done ...

o Shortening the time intervals between measurements.
o Increasing the number of observations.
o Simultaneously measuring the VVEE that could influence the RV, before and after treatment. o Evaluating the effect of treatment withdrawal (only if it is reversible).

Design of interrupted time series WITH NON-EQUIVALENT CONTROL GROUP

Adding a control group is the best way to control threats to internal validity.

Strong point = Main difficulty: Find a control group as similar as possible to the experimental one.


Yet No Comments