‘Systematic’ is, by dictionary definition, ‘a system, plan, or organized
method’ that is ‘methodical in procedure or plan’. It is widely assumed that
the ‘systematic review’ is the ‘gold standard’ of the Evidence Based Policy
Movement.
The assumption then is that if you are
going to review the literature on a given topic or subject, that you have come
up with a system of doing so, rather than approaching the topic haphazardly.
A systematic review is a paper that summarizes other papers
and aims to provide an exhaustive summary of literature relevant to a research
question.
The systematic review is an overview of primary studies that
used explicit and reproducible methods.
More and more, however, systematic reviews are investigating
what is called ‘grey literature’ or documents on a particular subject that have
not been peer reviewed or necessarily published in academic journals. It is these kinds of documents that can
sometimes give you information on the most up to date studies and/or pilot
programmes, particularly in disciplines such as healthcare practice, social
studies and community programmes.
Advantages of the systematic review include:
·
explicit methods limit bias;
·
conclusions are more reliable;
·
large amounts of information can be assimilated
quickly;
·
information delay is reduced;
·
results between studies can be compared;
·
new hypotheses can be generated;
·
and precision is increased.
Typically, the first step of a systematic review is a
thorough search of the literature for relevant papers. Appropriate databases and citation
indexes searched, such as Web of Science and PubMed, as well as any individual
journals. Next, the titles and the abstracts of the identified articles are
checked against pre-determined criteria for eligibility and relevance.
If you plan to tackle a systematic review yourself, your
first port of call should be your subject librarian in the University’s
library. She or he will be able to guide you, make suggestions about
appropriate databases to begin searching, and how to compile a list of search
terms that will be not only workable, but produce manageable results.
While systematic reviews are regarded as the strongest form
of medical evidence, a review of 300 studies found that not all systematic
reviews were equally reliable.
Systematic review is often applied in the biomedical or
healthcare context, but it can be applied in any field of research. While many
systematic reviews are based on an explicit quantitative meta-analysis or
combination of many trial results, there are also qualitative reviews which
adhere to the standards for gathering, analyzing and reporting evidence.
Studies suggest that extending searches beyond major
databases, perhaps into grey literature, would increase the effectiveness of
reviews.
The following are but several examples of
recent developments in systematic review that offer promise for more inclusionary
and diverse ways of observing, organizing and classifying evidence,
particularly qualitative studies, in the social sciences.
1. The Delphi process is a way of structuring
communication among a group of people in order to get their opinions, offer
feedback, and offer insights about a course of action.
2. The ‘Nominal Group’ technique (a physical gathering of
participants—unlike the communication process in Delphi technique by
questionnaire), on the other hand, works more closely with a brain-storming technique,
but also includes private ranking of ideas and tabulation.
3. Signal and Noise technique is useful for the fact that
the process does not eliminate research simply because it is not at a certain
level of evidence or if it has certain methodological weaknesses.
4. Grey literature is comprised of the literature that is
not found in peer reviewed journals and is made up of practitioner journal
literature, conference papers, books, literature from a range of public,
private and voluntary sector bodies, and government publications.
5. Scoping Studies are devices by which the ‘scope’ and
aim of a proposed study are investigated. Traditionally, the scoping study uses
a mix of literature review (with a particular purpose of uncovering previous
systematic reviews in the field under study) and stakeholder consultation.
6. Meta-ethnography is a comparative textural analysis of
field studies, using three ways to order them: in terms of one another, set
against one another or tied to one another. The premise is based in the
assumption that there is always a social and theoretical context in which
substantive findings emerge; the recovery of this context is the aim of
meta-ethnography.
Questions to ask of the systematic review:
1.
How can systematic reviews themselves be
evaluated?
2.
Can you find an important question that the
review addresses?
3.
Was a thorough search done and were other
potentially important sources explored?
4.
Was methodological quality assessed?
5.
Have the results been interpreted with common sense?
6.
In the long run, did finding the systematic
reviews on your topic save you time and increase your knowledge?
