Forgotten your password? 

23 November 2017
Linking Research to Policy and Practice: assessing the strength of evidence Print
Defining the Field
Wednesday, 15 April 2015 14:39

The focus of Research Uptake is on ensuring that research is taken up in policy or practice. The quality of research is therefore very important, as is the extent to which the research is grounded in and contributes to the broader body of evidence in a particular area. The cost (human and financial) of making policy or practice recommendations that are not supported by the overall body of evidence can be high.

In the world of science, research plays an important role. Within the many different types of research that is done, only some will be policy relevant, perhaps can be taken up by business, or used by a community to improve its livelihoods and living conditions. Other research, like basic research, contributes to the body of knowledge but may not have immediate applications.

 

In the world of Research Uptake, the focus is on applied, contextualized research. Most universities across Africa, as government funded institutions, align their research strategies with government priorities with the aim of contributing to the body of research evidence that can be used to address these government priorities. How then is the quality of such research assessed, given its potential importance in guiding policy and practise.

 

Quality of research

In the academic world there is an established tradition of peer review as a way of ensuring quality. Many academic journals use peer review as the criterion for publication, and the top academic publishers’ databases will only index journals that are peer reviewed. These are the databases that provide citation analysis and journal impact factors that are commonly used as proxies for excellence.

 

There are many challenges with peer review, particularly concerns about bias. However it remains a key tool for assessing the quality of research. Beyond this, it is also be used to evaluate other aspects of research – the design of the research proposal and the relevance of the research to social and economic priorities amongst other factors. Peer reviewers will assess the quality of research based on their own expertise and experience. They will look at the approach, methodology and theoretical underpinnings, and increasingly its potential utility value, and evaluate on this basis.

 

Where research results are shared with relevant stakeholders in order to improve or change practice or policy, it is not enough for the research to be of a high academic quality. While each research project should contribute to the body of evidence available on a particular topic and it is equally important that that overall body of evidence is sound, the evidence should also be made available and accessible to the stakeholders who are interested and affected.

 

Much of the research that is the focus of Research Uptake is intended to be applied in the policy or practice context – it is often a formal requirement of the funders of the research. This means that the research needs to provide evidence of the value of one approach over another. An example would be whether circumcision reduces the possibility of HIV infection. If research is conducted which shows that it does and governments make policies based on these findings, then it is important that the research results are of a high quality.

 

This is why governments are unlikely to base policy decisions on one piece of research. Literature reviews and systematic reviews are research synthesis techniques that look at the whole “body of evidence” about a particular problem. These reviews look firstly at the quality of individual studies and then the strength of the “body of evidence” which emerges from these studies. According to a DFID “How to” Note Assessing the Strength of Evidence “the overall strength of evidence is determined by the quality (or avoidance of bias) of studies that constitute it, and by the size, context and consistency of the body of evidence”.

 

Single studies

In terms of individual studies the “How to” Note emphasises that although journal rankings can be used as a proxy for quality, not all good quality studies are published and not all studies in peer reviewed journals are of a high quality. The document provides a number of principles for high quality research studies:

  • Conceptual framing – the study must acknowledge the existing research or theory
  • Transparency – high quality studies are transparent about the design and methods that they employ
  • Appropriateness – the appropriate research design (experimental, quasi experimental or observational) must be employed depending on the objectives of the study
  • Cultural sensitivity – studies must consider local cultural factors
  • Validity – the study must have internal, ecological, external and measurement validity
  • Reliability – the study must use consistent measures and have analytical and internal reliability
  • Cogency – there must be a clear logical thread that runs through the paper

Assessing the body of evidence

Once each study has been evaluated for quality the overall body of evidence needs to be evaluated. The main characteristics relate to the technical quality of the studies, the size of the body of evidence, the context in which the evidence is set and the consistency of findings.

The strength of the body of evidence can range from very strong to no evidence and this has implications. If the strength of the evidence is strong, it is possible to make a case(or not) a particular intervention. With limited strength of evidence there have to be caveats to the recommendations that explain the limitations to the probability that the desired impact will occur.

 

Good evidence makes good policy

In order for research to become part of the evidence-base that informs policy and practice its quality must be assessed beyond the traditional peer review. For individual researchers who want their research to be used beyond the academic field this means ensuring that the existing research, including systematic reviews, are taken into account. From the perspective of a policy maker or practitioner it is important to evaluate the quality of the evidence, and from an institutions’ perspective, when evaluating research for its utilization potential, it is important to consider both its quality and its potential application to meet local and regional social and economic development goals.

As Research Uptake (RU) practitioners we need to be clear that for Research Uptake to be successful, it needs not only to be relevant, it also needs to be of sufficient quality to be included in the body of evidence that informs policy and practice.


Alison Bullen, OSD Alison.bullen@drussa.net

 

...Back

Comments