Forgotten your password? 

23 September 2017
Quality assessment and assurance: Sound evidence for sound policy-making Print
Defining the Field
Thursday, 16 April 2015 14:44

A natural concern for university administrators, university stakeholders as well as funding agencies is the assessment and assurance of research-quality. They need to ensure that the evidence that is presented is academically sound.Trust in the research community to a large extent depends on sound quality assurance and assessment processes. Such activities inform funding decisions for projects, teams or even universities, the appointment of researchers, career advancement of researchers and crucially - what is published and disseminated.   And when the research evidence is specifically directed toward influencing policy it has to be properly contextualised, relevant and accessible,  in order to have the research findings taken up.

Quality may be defined as: “the nature or standard of something as measured against other things of a similar kind, and the degree of excellence it possesses” (Oxford English Dictionary)

“’Assessment’ may be defined as the process or means of evaluating academic work. ‘Assurance’, on the other hand, implies a promise or guarantee – a statement that something is of good quality and can be trusted.” (Research Information Network, 2010)

According to the Research Information Network research quality assurance processes should consider how new techniques and technologies could supplement more traditional systems. Technology also may guide new mechanisms for assurance such as social tagging, recommendation systems, plagiarism checking software, and online rating systems. More sophisticated bibliometric measures and new mechanisms such as Altmetrics will also play a role in quality assessment.

Many pressures exist within the research environment to drive research quality assurance activities. There is an increase in the expectation that research can provide solutions to real world issues,  which places pressure on available funding which in turn leads to increased competition between researchers. There are also new kinds of outputs, some of which are not subjected to traditional quality assurance processes such as peer review for journal publications. Advances in technology and new social media platforms need to be considered in modern research assurance procedures and systems (Research Information Network. 2010).

Research quality may have a range of dimensions such as the scope and extent of the study, the applicability to context or the quality of the process that has been followed and methodological issues and shortcomings.

Although there are differences between disciplines and fields of study, six key areas for traditional areas for research quality assessment and assurance are focused on the various stages of the research project namely (Research Information Network. 2010):

Stage 1: The process of conceptualising a programme or project i.e. the proposal stage

Stage 2: The process for monitoring project progress and oversight of the research

Stage 3: The early sharing of findings for input from colleagues and stakeholders

Stage 4: The formal publication stage – where peer review processes usually take place

Stage 5: Data sharing as an essential part of the record of the research and the development of data infrastructure for future research. This includes mechanism for management, curation and availability of datasets

Stage 6: Post-publication: quality assurance and assessment does not end with scholarly publication, it continues as other forms of the research is made available.

According to the Research Information Network research quality assurance processes should consider how new techniques and technologies could supplement more traditional systems. Technology also may guide new mechanisms for assurance such as social tagging, recommendation systems, plagiarism checking software, and online rating systems. More sophisticated bibliometric measures and new mechanisms such as Altmetrics will also play a role in quality assessment.

 

RAND Corporation Standards for High-Quality Research and Analysis

The RAND Corporation has stringent criteria for assessing quality and, as a guideline for individuals conducting, managing and evaluating research, they have codified a set of statements to capture what they term their hall-mark of research excellence:

  1. The problem should be well formulated, and the purpose of the study should be clear.
  2. The study approach should be well designed and executed. The selection of the study approach must be appropriate to the problem and the purpose.
  3. The study should demonstrate understanding of related studies.
  4. The data and information should be the best available.
  5. Assumptions should be explicit and justified.
  6. The findings should advance knowledge and bear on important policy issues.
  7. The implications and recommendations should be logical, warranted by the findings, and explained thoroughly, with appropriate caveats.
  8. The documentation should be accurate, understandable, clearly structured, and temperate in tone.
  9. The study should be compelling, useful, and relevant to stakeholders and decision-makers.
  10. The study should be objective, independent, and balanced.

Source: RAND Corporation (http://www.rand.org/standards.html)

Sources:

Research Information Network (2010). Quality assurance and assessment of scholarly research: A guide for researchers, academic administrators and librarians, available at file:///C:/Users/ssgrobbelaar/Downloads/Quality_Assurance_screen.pdf. Accessed on 15 April 2015

RAND Corporation, website available at http://www.rand.org/standards.html. Accessed on 15 April 2015


Dr Sara Grobbelaar is a senior researcher at the Centre for Research on Evaluation, Science and Technology (CREST) at the University of Stellenbosch, South Africa
...Back

Comments