Thursday, 28 November 2013



SABRE Research UK




THE 10Rs+ RECOMMENDATIONS to improve animal research


The 10Rs+ ~ REGISTER ~ REPORT ~ REPRESENT ~ REPLICATE ~ RETRACT ~ RECORD ~ RESTORE ~ REVIEW ~ REGULATE ~ REAPPRAISE


REGISTER past, present and future pre-clinical animal trials on an open global database.

REPORT in detail the research methods and rationale for each animal trial.

REPRESENT animal research honestly and transparently in all publications.

REPLICATE (reproduce) the trials to confirm or deny validity.

RETRACT (withdraw) any trials that cannot be reproduced.

RECORD (publish) all results of animal trials past, present and future on open access

RESTORE missing, abandoned or unreported animal trials (similar to RIAT in clinical trials)

REVIEW systematically existing animal trials before funding new animal research.

REGULATE with the aim of ensuring that the best available methods of conduct and assessment are in use.

REAPPRAISE (reassess) and monitor the value of animal research to human healthcare by carrying out programmes of systematic reviews of animal trials.

SABRE Research UK is an independent British non-profit registered charity which exists to protect the interests of patients and research volunteers. Our aim is to improve medical research by calling for the best available scientific methods and analysis of pre-clinical animal research. 

The charity has made ten recommendations. They have been assembled from input and information presented by researchers interested in the issues raised by the incidence of bias and methodological problems found in animal research and which affects the quality of the research. The charity has called the recommended measures the 10Rs+ with the intention that if implemented would help to improve pre-clinical animal research in the way it is conducted and assessed. The + sign indicates that the recommendations are encouraged to be added to or amended as necessary. 

Comment from stakeholders is an open invitation.
Key stakeholders are funders, institutions, journals, scientists, government, regulators, industry, medical research charities, and animal protection groups, but most importantly the public and patient's groups that in one way or another are affected by and pay for research. 

The 10Rs+ is licensed under a Creative Commons Attribution 3.0 License allowing anyone to share (copy, distribute and transmit) and to remix (adapt) or to make commercial use of the 10Rs+, under the conditions that they attribute the 10Rs+ to the charity SABRE Research UK (but not in any way that suggests the charity endorses them or their work).
The measures are derived from the best available evidence-based practices used in medical research.  Clinical research in drug development is largely based on data from animal trials and because of bias and methodological problems in the trials there is a cumulative negative impact on the research that ultimately affects patients.  The process of translating animal data to human healthcare is disadvantaged by research that is poorly conducted and analysed. It is of the utmost importance that pre-clinical animal research uses the best scientific practices and methods available to help improve the research and protect patients and research volunteers from unsound research.     

The 10Rs+ recommendations and why they are needed to improve medical research?


REGISTER animal trials. A register of animal trials involving research on studying mechanisms of disease, and for efficacy and toxicity (known as in vivo animal research).  This is not including in vitro animal tissue and animal cell culture etc. unless it involves in vivo work in some way, although a register of in vitro work would be recognised as just as important. The animal trials need to be registered before the work is carried out (prospective registration.)

There is no register in existence for keeping animal trials recorded on a database. This means that the research can be lost or go untracked and that much of it is left unmonitored. It prevents reviewers of animal research having all the information necessary to assess the research in systematic reviews and it is made difficult or impossible to trace unpublished studies. A register of animal trials was first called for in 2002.[1]

The register will allow researchers and the public to learn what research is to be carried out and will help with learning what research has not been carried out. The public has a keen and active interest in animal research and an expectation of openness and transparency from the animal research community.[2]  A register would help to improve the public perception that animal researchers want to keep research secret and not share it.  There is no need for the register to disclose personal details that would identify individuals or institutions and where commercial confidentiality is a concern the relevant level of detail can be withheld and only made available on request from other researchers who would be verified to gain access. The register would assist in avoiding the duplication of research, an ethical requirement that researchers are expected to uphold for the principle[3] of reducing animal research.  A record by way of an open global register of all animal research projects is essential in any efforts to avoid publication bias.  Systematic reviews are affected by publication bias because it diminishes the quality of the assessment and evaluation of research by ommission of relevant research in the review.[4] [5] Some animal researchers and reviewers have commented that contacting authors for missing data is made difficult because contact data are often missing or have changed. The response rate of authors is also very low and this could be improved by registration. 

In an attempt to overcome biased reporting in neuroscience the journal Cortex has launched a system called Registered Reports. Their project explanation provides an example of why the pre-registration of research is so important. 'For most journals, issues such as statistical power and technical rigor are outshone by novelty and originality of findings.' The pre-registration of experimental methods and proposed analyses are reviewed before data are collected.[6]

REPORT animal trials. It is necessary that all the details of the research are set out in the protocol (plan) for the research to demonstrate the necessity and justification of the research and a high level of detail should be included in the final publication of the research. It is also necessary that a systematic review of all relevant animal and clinical research should be shown to have been undertaken at the programme of work level before the new research is funded and approved.

The details to include in:
a) the protocol - the design (endpoints and how the study size* is determined and why), methods (whether the experiments are randomized; whether the investigator is blinded to the experimental animal group; whether excluded animals are reported and the reasons for exclusion), species, genetic considerations, numbers, gender[7], housing and environmental conditions and the rationale for the research, and whether and how the ethical practice of 'refining, reducing and replacing' the experiments are considered by the investigator(s).

b) the report - all the above details (including finer statistical elements) about the trials must be fully reported (documented) in the manuscript before funding and approval and included in the final paper to be submitted for publication.

This information is essential for the justification of the research, its replication, transparency and accountability and to avoid duplication[ii] of research. In clinical research some medical journals make this a requirement before they accept a manuscript for publication. It should also be the case for animal research submissions and some useful reporting guidelines have been published. However, adherence to the guidelines are presently dependent on journals and authors’ voluntary compliance. [8] [9]

REPRESENT animal research truthfully in all publications (scientific and general) by signing the Transparency Declaration that you affirm you have revealed all relevant details of the study openly, honestly and transparently when presenting the results.[10] The responsibility for the research must be placed on the lead researcher. A recent Ipsos MORI consultation on 'the public's view on openness and transparency in animal research' showed evidence that the public want the research to be more open to 'independent' and 'external scrutiny' in order for the public to have more trust in what researchers do and that researchers should be honest about their research.

REPLICATE animal trials. Research funding, investment and resources are wasted when animal trials, testing and experiments are published but then cannot be reproduced elsewhere and independently.[11] To begin to address the issue of irreproducibility in basic research the Science Exchange have launched the Reproducibility Initiative.[12] Multi-centre animal trials may also improve reproducibility, for example, the Multi-PART initiative [13]  

RETRACT published animal trials.  Studies that cannot be reproduced should be published but flagged up as unrepeatable so that other researchers know that the research cannot be reproduced and therefore avoided. Investors can then have better information on which early stage biotech ventures have been withdrawn and so not worth investing in.[14] The irreproducibility is still important information and so retraction should be declared but not concealed. [15] We need a system where a) fabricated research is retracted (withdrawn) and flagged up as fabricated and b) unrepeatable research is flagged up as irreproducible (unrepeatable) and the reasons explained and linked in with the original research. This R ties in with Replicate.  

RECORD all animal trials.  All the results of all animal trials should be recorded (published) in scientific journals. There is an important role for journals and reviewers here, who presently are known to discourage publication of neutral results.

All trials means all trials regardless of the outcome, whether positive, negative[iii] or neutral. So we want to know as much about what didn't work as what did work. If the experiments have been performed correctly, what did not work is just as important as what did work.[16] Systematic reviewers need to have full access to all the raw (unprocessed) animal data, in context with the associated fully described experimental reports, for full information to carry out their own independent research assessments of the research.

RESTORE or RECOVER all animal trials.  Where animal trials are missing, abandoned, or unreported a project similar to the RIAT initiative that recovers clinical trials should be adopted.[17]

REVIEW systematically all animal trials. Systematic reviews are essential if researchers are to make sense of all the research that has been carried out and published. The principle of using a systematic review to review research is to use the best available research methodologies that reduce bias from the review to arrive at a fair and transparent assessment of the research.  Through rigorous systematic reviews of existing animal studies reviewers can make a more robust estimation of the truth about the results. Others such as doctors and clinicians can then make healthcare decisions based on evidence that has a greater chance of being reliable. Those funding animal research should insist on seeing a systematic review of all the relevant animal and human research before funding any new animal research and there should be support and provision for a systematic review if one has not previously been carried out.

High quality systematic reviews are essential for several reasons: to avoid duplication and waste of research funding; to ensure the efficacy and safety of research; and to build the evidence-base. As a prerequisite for successful systematic review of animal studies, there is a need for development of tools to perform systematic reviews and meta-analyses of animal studies, since tools available for clinical reviews cannot be readily applied to animal data. A proposal for such a tool has been presented in a recent paper.[18]

REGULATE animal research. Regulatory bodies (the Home Office (UK), the European Medicines Agency (EMA), the International Conference on Harmonisation (ICH), and the Directive 2010/63/EU regulates animal research in the UK and Europe. The public have the expectation that these bodies regulate for the best available methods of conduct and evaluation of preclinical animal research but the regulations are found to be outdated, lacking in evidence and 'unfit for the 21st Century'.[19] Voluntary commitment to applying measures such as those recommended in the 10Rs+ cannot be relied on and the public will not understand why these measures that would protect patients and research volunteers from flawed research are not already in place or that they are not enforced in some way.

REAPPRAISE animal research.  The findings from systematic reviews of animal studies should be appraised so that the value of animal research and its contribution to human healthcare can be more fully understood. Systematic reviews could expect to answer questions about translation from animals to humans and answer questions about the policies and practices of preclinical animal research. Research that was carried out in 2003 has shown that contrary to earlier research which stated that 62% of all basic and animal research is of value to the clinic the actual estimate is closer to only 2 -20%.[20] Therefore, large-scale programmes of systematic reviews of all existing animal trials need to be carried out in order that the value of animal research can be reassessed and monitored. The publication of the results of such reviews will assist transparency and accountability of animal research and improve safety and efficacy in the translation of animal experiments to human research. Research is also needed to learn why some treatments found positive in animal studies do not translate to patients and whether some lines of research might be rejected because no efficacy is found in the animal trials. The public have the expectation that animal research is scientifically and accurately assessed for its value to human health


Summary of Feedback and Comments (so far):

Both animal researchers and 3Rs[21] advocates were the main respondents to the recommendations. In general both groups responded extremely postively and agreed that all of the 10Rs are very important. There were no responses from medical research charities, patient's groups or animal rights groups. It was generally felt that REGISTER, REPORT, RECORD, REVIEW, REPLICATE and REAPPRAISE were the most important of the Rs, but that the remaining Rs - REPRESENT, RETRACT, RESTORE (or as suggested RECOVER) and REGULATE were also important but perhaps harder to carry out.

An additional R was suggested as RETROSPECTIVE reviewing of ethics approval.[22]   

One 3Rs group saw a register of animal trials as an 'addition' to journal databases, suggestive of a lack of understanding of the scientific implications for having a register.  The same group did not appear to appreciate the scientific importance of restoring 'lost' animal studies, or the distinction between 'reviews' and 'systematic reviews'. 

One animal researcher commented in relation to REPORT  "There tends to be an overemphasis of the issue of sample size (strongly related to the issue of statistical significance and hypothesis testing, both heavily implicated in the terrible predicament animal research is facing)."  References were supplied. 'There should be interest from funding agencies in a system of reward for replication of research results and institutions may play a part in a reward system as well. One type of reward for replication of results is found in the journal Cortex which has launched a system of pre-registering experimental methods and proposed analyses. The author is offerred "in-principle acceptance" of their paper if they pre-register and peer review is favourable, which 'guarantees publication of their future results providing that they adhere precisely to their registered protocol.' '6 
'Like individual patient data (IPD) we need individual animal data (IAD). There should be centralized databases that collect and curate this information for secondary analysis. Such databases could in theory reduce the time to do systematic reviews from years to weeks. The IAD should include all animals that were in the experiment including ones excluded during different phases of the trial.'

Another 3Rs group commenting on REPORT said that with regard to the implementation of the ARRIVE guidelines: "ICLAS (International Council for Laboratory Animal Science) is currently working on a document to harmonize these guidelines and encourage journals to implement the ICLAS principles, leaving the journals free to adopt whichever guidelines document best suits their field. We understand that despite publication of the ARRIVE checklist and the Nature document, that journals are still finding it onerous to impose [them] (probably due to the need to better educate reviewers and insist on attention by the reviewers to the checklists)."

On the issue of RETRACT non-repeatable studies, one animal researcher commented that it was a difficult issue  - "What is non-reproducibility? Beware of simplistic statistical hypothesis testing criteria. Compare this to the issue of an existing meta-analysis of smaller trials homogeneously indicating some benefit. Then a mega trial is published with a null result (ISIS trials in cardiology). Should ISIS be retracted or the trials that made up the meta-analysis?"

Another animal researcher commented on RETRACT, "I was concerned that this information may disappear, as there may be a valid justification why the results cannot be replicated (e.g. I failed to be able to replicate the induction of an enzyme because the animal colony had a pin worm infestation).

Comments from a representative of a regulatory body indicated a lack of understanding of some of the the scientific principles that the recommendations are based on and some of which have been discussed for decades in the medical literature.  


Email: sabre.mail@sabre.org.uk
Deadline: open

References


[ii] Note that meta-analysis presupposes some duplication (replication(s)).
[iii] Recent comment referring to clinical trials suggests that the term 'negative' is dropped because no results of well conducted research can be considered negative (i.e. useless).

[1] Roberts I, Kwan I, Evans P, Haig S. Does animal experimentation inform human healthcare? Observations from a systematic review of international animal experiments on fluid resuscitation. BMJ. 2002 Feb 23;324(7335):474-6. Review 

[2] Ipsos MORI. Public dialogue: Openness in Animal Research: The public's view on openness and transparency in animal research. Report. Nov. 2013. http://www.ipsos-mori.com/researchpublications/publications/1619/The-publics-view-on-openness-and-transparency-in-animal-research.aspx Accessed 11th Nov 2013 

[3] Russell WM, Burch RL. The Principles of Humane Experimental Technique. Wheathampstead (U.K.), Reprinted by Universities Federation for Animal Welfare. 1992 

[4] Kimmelman J, Anderson JA. Should preclinical studies be registered? Nat Biotechnol. 2012 Jun 7;30(6):488-9. doi: 10.1038/nbt.2261

[5] Roberts I, Kwan I, Evans P, Haig S. Does animal experimentation inform human healthcare? Observations from a systematic review of international animal experiments on fluid resuscitation? BMJ. 2002 Feb 23;324(7335):474-6 

[6] New article format aims to prevent 'sloppiness' in science. http://www.elsevier.com/connect/new-article-format-aims-to-prevent-sloppiness-in-science Accessed 9th November, 2013 
[7] Krieger N. Genders, sexes, and health: what are the connections--and why does
it matter? Int J Epidemiol. 2003 Aug;32(4):652-7.  Reference supplied by a commentator
[8] Kilkenny C. et al. Improving bioscience research reporting: the ARRIVE guidelines for reporting animal research. PLoS Biol. 2010 Jun 29;8(6):e1000412
[9] Hooijmans CR, de Vries R, Leenaars M, Curfs J, Ritskes-Hoitinga M. Improving planning, design, reporting and scientific quality of animal experiments by using the Gold Standard Publication Checklist, in addition to the ARRIVE guidelines Br J Pharmacol. 2011 Mar;162(6):1259-60
 
[10] Altman D, Moher D. Declaration of transparency for each research article.   BMJ 2013;347:f4796
[11] Begley CG, Ellis LM. Drug development: Raise standards for preclinical cancer research. Nature. 2012 Mar 28;483(7391):531-3
[12] The Reproducibility Initiative https://www.scienceexchange.com/reproducibility accessed 29th May 2013
[13] CAMARADES Multi-PART Initiative. http://www.camarades.info/MultiPART/MULTIBG.html 
[14] Bruce Booth, Atlas Venture http://lifescivc.com/2011/03/academic-bias-biotech-failures/ accessed 13th May 2013
[16] Sena ES, van der Worp HB, Bath PMW, Howells DW, Macleod MR (2010) Publication Bias in Reports of Animal Stroke Studies Leads to Major Overstatement of Efficacy. PLoS Biol 8(3): e1000344. doi:10.1371/journal.pbio.1000344
 
[17] Doshi P, Dickersin K, Healy D, Vedula SS, Jefferson T. Restoring invisible and abandoned trials: a call for people to publish the findings BMJ. 2013 Jun 13;346:f2865. doi: 10.1136/bmj.f2865 
 
[18] Vesterinen HM, et al. Meta-analysis of data from animal studies: A practical guide. J Neurosci Methods 2013 Oct 4;221C:92-102
 
[19] Rawlins MD, Opinion: Cutting the cost of drug development? Nat Rev Drug Discov, Apr; 3 (4)360-4 (2004)
[20] Grant J, Green L, Mason B. From Bedside to Bench: Comroe and Dripps Revisted. HERG Research Report No.30, 2003, Brunel University
[21] Russell WM, Burch RL. The Principles of Humane Experimental Technique.
Wheathampstead (U.K.), Reprinted by Universities Federation for Animal Welfare. 1992
 
[22] Varga O, Hansen AK, Sandøe P, Olsson IA. Improving transparency and ethical accountability in animal studies: three ways to link ethical approvals to publications. EMBO Rep. 2010 Jul;11(7)

Posted by at No comments:
Subscribe to: Posts (Atom)