Report No. 01-26
September 17, 2001
Office of the Inspector General
APPENDIX III
AUDIT CRITERIA
Quality Assurance Standards
The QAS, recommended by the DNA Advisory Board and formally instituted by the Director of the FBI, are one of the key sources of criteria for an audit of a CODIS?participating laboratory. Two sets of standards have been instituted: the Quality Assurance Standards for Forensic DNA Testing Laboratories effective October 1, 1998 (Forensic QAS); and the Quality Assurance Standards for Convicted Offender DNA Databasing Laboratories effective April 1, 1999 (Offender QAS).
The Forensic QAS contain 155 elements organized under 15 headings, and the Offender QAS contain 136 elements also organized under 15 headings. The information below serves only as a synopsis of these headings, and does not capture many of the individual elements contained under each heading. The use of the QAS in this audit was specific to the laboratory unit being audited. In other words, the 155 Forensic QAS elements were used to audit the portion of a laboratory performing DNA analysis on forensic samples, and the 136 Offender QAS elements were used to audit the portion of the laboratory unit performing DNA analysis on convicted offender samples. To demonstrate the similarities between the two sets of QAS, the elements were separated into those that were either identical or similar and those that were unique to just one set of QAS. There are a total of 119 shared (identical or similar) elements, 36 elements unique to the Forensic QAS, and 17 elements unique to the Offender QAS (delineated by heading below).
- The Quality Assurance Program: one should exist in writing and should contain the required categories of standards. This section contains 15 shared elements.
- Organization and Management: key roles and duties should be accounted for in writing, as should be the interrelation between the personnel involved in DNA analysis. This section contains 3 shared elements and 1 element unique to the Offender QAS.
- Personnel: personnel filling key roles should be properly educated, trained, and should be performing duties appropriate to their position. This section contains 19 shared elements and 5 elements unique to the Offender QAS.
- Facilities: the physical design of the laboratory and additional controls should ensure the integrity of laboratory security and minimize contamination. This section contains 5 shared elements and 1 element unique to the Offender QAS.
- Evidence Control (Forensic QAS only): the laboratory should have a documented control system, and the necessary internal controls to implement it, to ensure the integrity of the evidence and to govern the final disposition of the evidence. This section contains 7 unique elements.
- Sample Control (Offender QAS only): the laboratory should have a documented control system and necessary internal controls to implement it, to ensure the integrity of the offender samples. This section contains 5 unique elements.
- Validation: the laboratory should take the required steps to demonstrate (validate) that it and its analysts are capable of using certain equipment and methods properly. This section contains 8 shared elements and 5 elements unique to the Forensic QAS.
- Analytical Procedures: every procedure used by the laboratory in the DNA analysis process, including those reagents required in the process, should be described in detail in writing and formally approved by laboratory management. This section contains 19 shared elements and 13 elements unique to the Forensic QAS.
- Equipment Calibration and Maintenance: the laboratory should establish a written program for ensuring that equipment used for DNA analysis receives regular calibration and maintenance. Such calibration and maintenance should be clearly documented and be based upon independent national standards. This section contains 8 shared elements.
- Reports: the laboratory should have written guidelines for maintaining documentation that would thoroughly support the conclusions made in a report regarding case evidence. Reports should contain QAS-specified information and written policies should exist to govern the release of such information. This section contains 2 shared elements and 10 elements unique to the Forensic QAS.
- Review: administrative and technical reviews should be conducted of all reports and supporting documentation for all evidence, to ensure the quality of the conclusions and supporting documentation. The testimony of analysts in court should also be reviewed. This section contains 2 shared elements, 1 element unique to the Forensic QAS, and 1 element unique to the Offender QAS.
- Proficiency Testing: those actively engaged in DNA analysis should complete an external proficiency test (a test from an outside agency or commercial test provider that measures an analyst's skill in performing DNA analysis correctly) every 180 days. Such tests should be reviewed and documented as delineated in the QAS. This section contains 16 shared elements.
- Corrective Action: written procedures should exist that govern a laboratory's documentation and resolution of errors made during a proficiency test or DNA analysis. This section contains 2 shared elements.
- Audits: the laboratory should undergo an audit every year, and at least every other year this audit should be conducted by an external entity. This section contains 17 shared elements.
- Safety: the laboratory should have and follow a written environmental health and safety plan. This section has 1 shared element.
- Subcontractor of Analytical Testing for Which Validated Procedures Exist: a laboratory making use of a subcontractor for any part of the DNA analysis process should establish certain specified controls to ensure the integrity of the subcontractor's work and results. This section contains 2 shared elements and 4 elements unique to the offender QAS.
NDIS Requirements
The standards specific to laboratories participating in the national index (generally referred to as NDIS requirements) are contained in the MOU that is enacted between each state index laboratory and the FBI. It is important to note that the MOU covers the participation of the state index laboratory and any local index laboratories that upload profiles to that state index laboratory. Therefore, even though these local index laboratories do not receive national index information or sign the MOU directly, they are to comply with NDIS requirements.
The MOU requires that its participants comply both with general requirements already issued (i.e., federal legislation, the QAS) as well as requirements specific to the national index that accompany the MOU in the form of appendices. These appendices are as follows: Appendix A-NDIS Responsibilities, Appendix B-NDIS Data Acceptance Standards, and Appendix C-NDIS Procedures Manual. 11 From these appendices, 17 elements were included as part of our audit criteria, as described in the remainder of this section.
Our audit criteria included the following 10 elements from Appendix A. Not included in our count are: (1) elements in Appendix A that are also included in the QAS, (2) elements not consistent with our audit scope or objectives, and (3) elements that only require compliance with other established criteria (such as the QAS, federal legislation, or other MOU appendices).
- Comply with FBI requirements for physically and electronically safeguarding CODIS against unauthorized use, including providing an appropriate and secure site for the NDIS system.
- Designate one agency within each state to be responsible for ensuring that conditions and standards for participation in the national index are met.
- Designate one CODIS liaison within the state agency to handle communications with the FBI.
- Ensure that appropriate personnel are provided copies of, understand, and abide by the NDIS Procedures Manual.
- Identify in writing, in prescribed form, personnel approved to access CODIS and ensure that access to CODIS is limited to approved personnel.
- Maintain records on these personnel, including proficiency testing records and any other report required by the FBI, for a period of 10 years.
- Conduct background investigations of personnel designated to input data to or access the national index.
- Maintain a system of controls to ensure that DNA records are kept as long as they are substantiated by the laboratory's internal records and are allowed to be retained by federal or state law, by judicial decree or by consent, and used in local, state, and national indexes in accordance with the Act, applicable state law, and for the national index, in accordance with the Privacy Act of 1974. This is the only NDIS requirement that pertains to the convicted offender profile sample as well as the forensic profile sample.
- Report on a monthly basis confirmed national index matches to the FBI in a form prescribed by the FBI.
- Provide to the FBI a written report of deletions/modifications within 10 business days of discovering a DNA record requires deletion/modification.
Our audit criteria included the following four elements from Appendix B. Not included in our count are: (1) elements in Appendix B that are also included in the QAS, (2) elements not consistent with our audit scope or objectives, and (3) elements that only require compliance with other established criteria (such as the QAS, federal legislation, or other MOU appendices).
- Test results for nine RFLP loci are accepted at the national index. However, an RFLP forensic profile will not be included in the national index unless the laboratory tests for four specific loci. The laboratory must obtain results for three of these four loci for a forensic profile to be searched against.
An RFLP convicted offender profile will not be included in the national index unless the laboratory tests and obtains results for the four required loci.
- Laboratories using STR technology must use one of the FBI-approved kits.
- An STR forensic profile will not be included in the national index unless the laboratory tests for all 13 loci that are accepted at the national index. The laboratory must obtain results for 10 of the 13 loci for a forensic profile to be searched against.
An STR convicted offender profile will not be included in the national index unless the laboratory tests and obtains results for all 13 loci.
- Only forensic profiles derived from crime scene evidence matching the suspected perpetrator(s) or an unknown individual can be uploaded to the national index. Profiles clearly matching the victim or any known person other than the suspected perpetrator(s) cannot be uploaded to the national index. However, if the forensic profile is a mixture that cannot be clearly separated into a portion matching the victim or other known person and the portion matching the suspected perpetrator, such a profile would be accepted.
Only one set of procedures from the NDIS Procedures Manual (considered to be MOU Appendix C) added to our audit criteria. The remainder of the manual consisted of sets of procedures outside the scope of our audit. The one set of relevant procedures contained detailed instructions on confirming and documenting candidate matches, both for case-to-case matches as well as case-to-offender matches. The following three specific elements from this set of procedures were included in our audit criteria.
- Candidate matches must be resolved within 30 business days. Resolution is explained as either refuting or confirming that the candidate match is a valid match.
- In circumstances where a match is confirmed between two cases, at a minimum the law enforcement agencies investigating the cases must be notified.
- A report should be generated and filed for each confirmed candidate match, including at a minimum, the prescribed forms and information delineated in the procedures.
Although not considered additional audit criteria, the NDIS Procedures Manual did contain helpful definitions that clarified the proficiency testing terms contained in the QAS. The 180-day maximum interval between completion of external proficiency tests is defined as the time between the completion of one proficiency test (i.e., submitting the test results to the external provider) and the start of the next test. External tests are further explained as obtained from and submitted to an external provider.
State Legislation
The collection of DNA samples from specified convicted offenders and the establishment of a convicted offender DNA database has been legislated in every state. However, the legislation varies from state to state, particularly in the number and types of crimes that require collection from an offender. The state statutes governing the laboratories we audited varied from the collection of a DNA sample from all felons (Virginia) to the collection of a DNA sample for crimes classified as or related to sexual assault (Illinois). All of the statutes include sexual crimes and, all but the California statute applied to offenders convicted previously and still incarcerated at the time the statute became effective. All applied to juveniles except the North Carolina statute.
The offender profiles reviewed were governed as follows:
- California Penal Code §§295, 3060.5 and California Government Code §76104.5, effective in 1989
- Florida State Statute §943.325, effective January 1, 1990
- Illinois State Statute §730.5, effective July 1, 1990
- North Carolina State Statute §15A-266, effective July 1, 1994
- Pennsylvania Consolidated Statues Title 35, §7651.101, effective in May 1995
- Code of Virginia §16.1-299; 19.2-310.2 et seq, effective July 1, 1989
- The manual, a collection of operational procedures to be followed for various processes pertinent to the functioning of NDIS, was actually issued separate from the MOU although it is still considered an appendix to the MOU.