Accreditation

Review of the CCB’s pilot audit scheme

Background

Biobanking is recognised as a critical area requiring development if progress is to be made in identifying clinically useful markers of disease and disease progression, discovering new drug targets and understanding the mechanisms of disease in cancer. Researchers continue to report that they are unable to obtain sufficient high quality, well annotated samples of diseased and control tissue, blood and other biological materials [1][2]. At the same time, funders of research, and especially funders of biobanks, are looking to obtain best value from their investments in sample and data collection. There is a need to increase the availability to researchers of large numbers of high quality, well annotated samples of diseased and control tissue, blood and other biological materials and in this way accelerate cancer research.

Researchers report two major problems; their inability to get access to sufficient numbers of samples and the fact that the available samples are not always suitable for their research. The Cancer Genome Atlas (TCGA) group in the US, for example, reported at a meeting of the International Cancer Genomics Consortium that biobanks typically overestimate the quality of the cancer tissue samples they hold [3].

Members of the Confederation of Cancer Biobanks (CCB) reported that there are significant differences in the quality and availability of materials from different member banks, and raised concerns about the risk to member banks’ reputations since there are no stated quality criteria for management of members’ banks or the samples and data held in them. In 2014, the CCB published quality and data standards for biobanks in the UK. These standards were designed to apply to all types of biobanks, irrespective of disease focus and including biobanks with no specific disease focus and cohort collections.

CCB members citing quality differences asked for a way of benchmarking biobanks. Funders of biobanks and researchers using human tissue and data have identified the need for a quality mark of some description for biobanks. This paper describes the findings from a pilot scheme to audit biobanks against the requirements of the CCB’s quality standard.

Benchmarking of biobanks

The aim of any benchmarking scheme is to ensure that biobanks’ processes and procedures, as well as their products, are fit for purpose. In 1999, the OECD suggested that national governments 'should support the development of an accreditation system for biobanks based upon scientifically acceptable objective international criteria for quality, expertise and financial stability'. This is one way in which biobanks can be benchmarked but there are several options for benchmarking of biobanks, namely self-assessment, peer review, registration by an expert body or a formal certification or accreditation procedure.

Self-assessment is the simplest and least expensive route. A series of questions can be drawn up based on required standards and the biobank can rate itself against the questionnaire. Members of the International Society for Biological and Environmental Repositories (ISBER), for example, have access to a web-based self-assessment tool designed to allow biobanks to assess their compliance with the ISBER best practice guidelines. This is the least costly option; it requires administration but little else once it is set up. The main drawbacks to self-assessment are the lack of consistency and transparency in assessments and the lack of a driver for improvement.

Peer review consists of inspection and assessment of compliance with required standards by experts in the field, such as staff from another biobank or associated organisation. This system is more expensive, requiring staff to be released from their normal work to visit the biobank that is seeking assessment. Its advantage over self-assessment is that the assessors can be trained so that assessments are consistent and the relative independence of the assessors gives greater assurance of the validity of the results of the assessment. The assessors, however, are not truly independent since they assess one another’s banks; there is a natural 'professional courtesy' between such assessors that is to the detriment of the perception of independence since the person whose bank you are assessing today may come to assess your bank next time. In addition, there are problems with ensuring confidentiality and protecting intellectual property if the assessors are your 'competitors'. This is handled in most instances by requiring assessors to sign confidentiality agreements but some commercial organisations are not willing to expose their practices and procedures to peers from competitor organisations. Great care is needed when using competitors as assessors.

A registration procedure consists of inspection and assessment of compliance with required standards by experts independent of the biobanks to be assessed. This system is more expensive than self-assessment or peer review because it requires the employment of suitably qualified and experienced staff. It has the advantages of consistency described above and overcomes problems with conflict of interest, confidentiality and protection of intellectual property. It can be tailored to suit the needs of the community but will have limited value outside of that community.

Certification and accreditation are recognised at national and international levels as the 'gold standard' marks of the quality of products, processes and organisations. The terms certification and accreditation tend to be used interchangeably by 'lay' people, however they have different meanings. Certification is granted for consistency in following defined procedures. Accreditation is granted for technical competence. Both certification and accreditation depend upon the implementation of a quality system, leading to quality assurance. Formal certification and accreditation depend upon the availability of written standards and at present there is no internationally recognised technical standard against which biobanks can be accredited. Formal schemes are expensive to implement and maintain because they rely on professional assessors with sufficient technical expertise to judge performance. The United Kingdom Accreditation Service (UKAS) is the only UK-based accreditation body recognised by the UK government and, therefore, the international accreditation community.

The pilot audit scheme considered the options described above and chose to implement a mixture of self-assessment and peer review. This allowed the scheme to be undertaken without specific funding, but relied on the willingness of biobanks to get involved in the scheme, and also to release their staff to act as auditors and auditees.

Position prior to the pilot audit scheme

All biobanks in the UK will have in place already some of the elements of a quality system. This is required to obtain a licence to store human tissue for research from the Human Tissue Authority (HTA) or to be accredited in Scotland. The HTA and Scottish accreditation requirements, however, are primarily concerned with donor consent and traceability of the tissues themselves. The need for a HTA licence does not apply to biobanks storing cell-free material such as serum or plasma, or biobanks processing tissue to obtain RNA, DNA or cell lines thus some biobanks will not hold an HTA licence. In the same way, clinical trials biobanks will have in place quality system elements associated with the clinical trial, and pathology-based biobanks will have Clinical Pathology Accreditation (CPA) or United Kingdom Accreditation Service (UKAS) accreditation of some of their work. It is important that any scheme devised to benchmark biobanks recognises these systems and does not duplicate (or contradict) the requirements of another body.

In the absence of a widely applicable biobank-specific technical standard, some biobanks, for example UK Biobank and the UK DNA Banking Network, have obtained certification of their quality management systems against the requirements of ISO 9001 . UK Biobank has, in addition, obtained certification of its data security systems against the international standard for management of data security, ISO 27001 [4]. The need for a technical standard has been recognised widely and there are several national and international initiatives looking at accreditation schemes to assess biobanks. The International Standardisation Organisation (ISO) has set up a technical committee (TC276) to devise standards for biotechnology, including biobanking. These standards, once published, will provide an international standard against which biobanks can be assessed.

At the same time as standards are being developed, some organisations are developing best practice guidelines . Best practice guidelines cannot be used as a standard for benchmarking because they are guidelines only and do not give requirements. They can be used to support a biobank-specific standard but cannot replace it. Guidelines relevant to biobanks have been published by OECD , NCI, and ISBER. OECD guidelines give general best practice for Biological Resource Centres and are not specific to research biobanks. The NCI guidelines are specific to cancer biobanks but focus on practices, governance and regulation within the USA. ISBER guidelines are very broad, covering banking of environmental samples, species of plants, animals and microorganisms as well as human tissue. None of these guidelines is applicable directly to research biobanks in the UK but they provide a useful starting point for the development of UK-specific guidelines.

The pilot audit scheme

Audits were performed against the requirements of the CCB’s quality standard. The proposed quality mark scheme is described in appendix 1. Volunteer biobanks were asked to complete an application form (appendix 2), self-assessment questionnaire (appendix 3) and submit their quality manual and a list of standard operating procedures (SOPs).

Auditors were recruited by the audit coordinator (Anne Carter) and dates for the audit negotiated. Auditors signed a confidentiality agreement (appendix 4), and received a briefing note describing how to carry out an audit (appendix 5) and copies of documents provided by the auditee. The audit team met prior to the audit to assign tasks and areas of work.

Audit proceedings were recorded on the standard forms shown in appendices 6,7,8,9 (opening meeting, audit observations, non-compliances found, closing meeting). The auditors met periodically during the audit to discuss their findings and, if necessary, change the focus of the audit. Verbal reports were made to the auditee at the end of each day and at the end of the audit, and the auditee received copies of all of the audit documentation.

Records of the audit process were created and maintained by the audit coordinator (appendix 10).

During these pilot audits, the auditees were not required to correct the non-compliances found.

The audits were reviewed at a meeting of the auditors and auditees who took part in the pilot scheme. One of the audited biobanks was unable to participate in this meeting.

Results

Pilot audits were carried out in six biobanks, including cancer and non-cancer biobanks, academic and commercial (pharma) biobanks, biobanks collecting from the living and one collecting only post mortem tissue, biobanks where staff members were part of the CCB’s harmonisation project, and so helped to develop the standard, and biobanks whose staff did not take part and thus did not have prior knowledge of the standard. These biobanks are not named in this report as confidentiality was a condition of their participation. Aggregated results are discussed and raw data from the review meeting is given in appendix 11.

The audit process and record keeping were felt to be appropriate. The first pilot audit took place on one day; this was found to give insufficient time and subsequent audits. Two days allowed a balance between thorough auditing and acknowledging that all audits examine only a sample of processes, procedures and records.

The CCB standard, against which audits were performed, was felt to be comprehensive and achievable. The self-assessment exercise was reported to help auditees to become familiar with the requirements of the standard, however some auditees found it to be over-long and repetitive in places. This may be a reflection of the structure of the standard and should be examined next time the standard is reviewed.

Audits were carried out by the audit coordinator and up to two peer auditors who were senior, experienced biobank staff. The audit coordinator gave consistency to the assessments by taking part in all six pilot audits. The use of peers as auditors was welcomed by the biobanks; auditors and auditees found the exchange of knowledge to be a valuable part of the process. The presence of a lay person as an auditor was not found to be helpful.

Auditors’ training was informal. Whilst auditors found the training to be helpful, the need for more formal training to recognised standards will be needed if the scheme is to continue.

The number and severity of the non-compliances found varied between biobanks. Auditees were not required to provide evidence that they had corrected the non-compliances found however most auditees reported doing so. Audit findings were used by some auditees to raise the non-compliances found with managers, especially non-compliances that were outside the direct control of biobank staff.

Most non-compliances related to the quality assurance and quality management systems, especially document control and record keeping. Some more significant findings, relating to control of access to labs, temperature monitoring and control of sample storage, validation of equipment and procedures, consent, training and change control, were found. One clear finding from all of the pilot audits is that the legal, ethical and practical aspects of biobanking are well controlled and to a reasonably high standard. It was noted that some of the biobanks did not have a quality manual in place, and would find a template quality manual very useful.

Future audit scheme

Participants in the pilot scheme were strongly supportive of continuing a peer-review audit scheme in future. They reported that the pilot highlighted weaknesses, helped identify areas for improvement and gave confidence that most areas are well controlled and in line with peer expectations as shown in the CCB’s quality standard. The auditees would like to have recognition of this through a quality mark scheme or accreditation.

Funding for a future scheme will be required and ways of accessing funding should be investigated. One suggestion was that biobanks seeking to be audited should pay for auditors’ travel and accommodation, while biobanks providing auditors should provide their time free of charge in recognition of the mutual benefits seen during the pilot audits. One of the major concerns expressed at the review meeting was the potential cost of participation in terms of charges to participate, time and staff effort in achieving the required standard and correcting the problems raised by the audit. These costs were seen as a potential barrier to participation, and should be kept to a minimum.

Biobanks have commented that it is difficult for them to meet the HTA requirement for independent audit, and continuation of the CCB’s peer review audit scheme will be useful in achieving this requirement.