Pragmatic versus Explanatory Trials
When you define your research question you will need to consider whether you are asking a pragmatic or an explanatory question. This will have important implications for your study design.
Pragmatic research asks whether an intervention works under real-life conditions and whether it works in terms that matter to the patient. It is simply concerned with whether the intervention works, not how or why. Pragmatic studies are most useful for deciding what services should be provided but give only limited insight into why interventions do or do not work.
Explanatory research asks whether an intervention works under ideal or selected conditions. It is more concerned with how and why an intervention works. Explanatory studies are valuable for understanding questions of efficacy but are of limited value for telling us whether we should provide a service to a wide variety of patients in a wide variety of circumstances.
The distinction between pragmatic and explanatory research is important because it will determine key methodological issues relating to patient selection, definition of the intervention and controls, use of blinding and placebos, choice of outcome measure and type of analysis.
Patient selection
Patient selection for a pragmatic study should reflect routine practice. All patients who might receive the intervention should be studied. Selection criteria should be broad, with exclusions limited to patient groups for whom either the intervention or control are contraindicated. Thus we will know whether the intervention works for patients in general.
For an explanatory study recruitment may be more selective. By excluding patients with co-morbidity or patients with a doubtful diagnosis we can establish whether the intervention works under ideal conditions. However, we will not know how the intervention works in the rather more complex “real-life” setting.
The intervention
In an explanatory study the intervention will be strictly defined and tightly controlled, while in a pragmatic study it will be left much more to the discretion of the clinician. It may be provided by “experts” or “enthusiasts” in an explanatory study but should be provided by normal staff with routine training in a pragmatic study.
The control
From a pragmatic viewpoint, we are not concerned how an intervention works compared to the control, simply whether it works or not. The control may therefore be routine practice rather than a placebo. An explanatory trial, on the other hand, may use a placebo to determine how the treatment works, i.e. is it a placebo effect.
Other aspects of care for the intervention and control groups
In an explanatory study the two treatment groups should, as far as possible, receive the same management except for the intervention under investigation. Management may need to be protocol driven to achieve this. However, for a pragmatic trial we are interested in the management consequences of the alternative treatments. So, apart from the intervention under investigation, management will usually be left at the discretion of the clinical staff and may ultimately differ between treatment groups.
Blinding
To avoid bias, measurement of outcome should always, if possible, be performed by someone who is blind to the treatment group. It is usually accepted that blinding of both carers and patients is also desirable, and this is certainly true in an explanatory study. If patients or carers are aware of their treatment group they may derive benefit from mechanisms other than the direct effect of the treatment, such as a placebo effect. In a pragmatic study we are simply concerned about whether patients derive benefit, not how or why they derive benefit. Hence blinding of patients and carers may be inappropriate for a pragmatic trial.
Outcome measures
Pragmatic outcome measures are usually patient centred and incorporate broad measures of health-related quality of life. The aim is determine whether the intervention works from the patients point of view. Explanatory outcome measures may focus on specific symptoms or dimensions of quality of life, or include clinical and biological measures. These allow greater understanding of how an intervention works, but may not relate directly to the patients preference.
Type of analysis
Intention to treat analysis is required to avoid bias caused by those failing to comply or changing treatment group being systematically different to those who complete treatment. This is also essentially a pragmatic analysis. Participants are analysed according to the treatment they were allocated to rather than the treatment they received. Hence we are measuring the effect of making treatment available rather than actually receiving treatment. An explanatory approach may analyse participants according to the treatment they actually receive. This may allow insight into why a treatment is, or is not, effective. Because of the risk of bias this analysis cannot demonstrate effectiveness, it can only generate hypotheses about effectiveness.
Does this only apply to studies of therapeutic interventions?
Pragmatic versus explanatory considerations are usually applied to evaluations of therapeutic interventions. However, these considerations may also apply to other studies, such as studies of diagnostic tests. An explanatory study would evaluate the diagnostic test in a closely monitored, selected group of patients with the diagnostic test being performed by highly trained individuals under controlled conditions. A pragmatic study would include a broad group of patients with relatively loose control of the testing procedure.
Explanatory or pragmatic?
These terms represent ends of a continuum rather than distinct entities. A study may include explanatory and pragmatic elements. For example, a pragmatic trial with broad selection criteria, patient-centred outcome measure and intention to treat analysis may also include explanatory elements such as subgroup analysis of selected patient groups; secondary, explanatory outcome measures and analysis with non-compliers excluded. Ultimately, however, the decision as to whether the treatment is provided or not should be based on the pragmatic analysis.
Efficacy and effectiveness
The type of study you undertake will determine whether you are measuring efficacy or effectiveness. These two terms do NOT mean the same thing.
Efficacy refers to whether the intervention works under ideal conditions. It answers the question- Can this work?
Effectiveness refers to whether the intervention works under routine or “real life” conditions. It answers the question- Does this work?
Who’s viewpoint are you interested in?
The choice of a pragmatic or explanatory design may depend upon the viewpoint of the research question. If you are looking at it from a health service viewpoint you probably want to answer the question- shall we provide this intervention? In this case a pragmatic design is more appropriate. If your viewpoint is that of the individual patient you probably want to answer the question- should I receive this intervention? In this case an explanatory design may be more useful.
It all boils down to defining your research question precisely. The methodology will follow.
Further reading-
What are pragmatic trials?
Roland and Torgerson. BMJ 1998;316:285
http://www.bmj.com/cgi/content/full/316/7127/285Can it work? Does it work? Is it worth it?
Haynes. BMJ 1999;319:652-653
http://www.bmj.com/cgi/content/full/319/7211/652
