The FBI DNA Laboratory: A Review of Protocol and Practice Vulnerabilities

May 2004
Office of the Inspector General

Jacqueline Blake's misconduct has required the FBI Laboratory to reassess its oversight of DNA testing in the DNAUI. The objective of this review was to identify ways to make the DNAUI less vulnerable to undetected inadvertent or willful noncompliance with the protocols that govern DNA analysis. Our recommendations focus on two general types of vulnerabilities: protocol vulnerabilities and practice vulnerabilities. In addition, we provide recommendations that we believe will address several of the issues of concern that we identified regarding the management response of the FBI and DOJ to Blake's misconduct. Our recommendations are listed below.

I. MANAGEMENT RESPONSE RECOMMENDATIONS

1. To facilitate prompt communications with evidence contributors and prosecutors in the event of future testing problems, the Laboratory should maintain the information below in an electronic format that can be shared conveniently with other FBI components (e.g., FBI OPR and FBI OGC) and the Department of Justice: all contributor contact and case information currently required for an evidence contributor to request an evidence examination (see FBI Handbook of Forensic Services, (https://www.fbi.gov/file-repository/handbook-of-forensic-services-pdf.pdf/view); the e-mail address of the evidence contributor; and the name, title, agency, address, telephone number, and e-mail address of any associated prosecutor(s);

2. In circumstances where a protocol violation renders testing results scientifically invalid and a report from the Laboratory is not expected to issue within 180 days from the discovery of the violation, the Laboratory should notify the evidence contributor of the following information within 90 days of learning of the violation: the nature of the protocol violation; how the violation occurred; the remedial measures that the Laboratory intends to implement in the case to generate scientifically valid testing results; and the time needed to complete the remedial measures and to issue a final report; and

3. The FBI Laboratory should perform a file review of a sample of cases that Blake is known to have worked on prior to becoming a PCR Biologist to reconfirm that the procedures that were required in fact are documented as appropriate in the case files.

II. REMEDY PROTOCOL VULNERABILITIES

Our analysis of the DNAUI's protocols revealed various weaknesses that leave the Unit vulnerable to employee error and wrongdoing. The recommendations we present below are designed to reduce the Unit's exposure to this vulnerability and fall in four general areas: 1) eliminate vague and ambiguous text from the protocols; 2) incorporate decision aids into the protocols; 3) enhance notetaking requirements; and 4) update the protocols.

1. Eliminate Text Vagueness

   Approximately 20 percent of the protocol sections we examined lacked the detail necessary for a technically qualified DNA scientist to identify the testing methods that should be in use in the DNAUI. Accordingly, DNAUI management should ensure that the document sections identified as vague in Chapter Five of this report are corrected to describe completely and accurately management expectations, Unit procedures and policies, and "best practices" currently in use in the DNAUI. Specifically, DNAUI management should:

   4. Ensure that the Evidence Control Policy of the FBI Laboratory Division Quality Assurance Manual, the Evidence Control and Facilities (Security) sections of the DNA Analysis Unit I Quality Assurance Manual, and the Procedures for the Examination of Evidence section of the FBI Laboratory Division Caseworking Procedures Manual, contain:

      1. Comprehensive guidance regarding the prevention of evidence contamination, loss, and destruction. At a minimum, this information should be included in Evidence Control Policy sections 3.2 and 3.5; Evidence Control sections 7.6.1 and 7.6.2; and Procedures for the Examination of Evidence sections 4.1.3 and 4.5.1.1. The Laboratory-wide sections should identify procedures that are applicable to all Units. Sections associated with a particular Unit should provide a complete listing of the evidence preservation methods in use in that Unit, even when those methods are identified elsewhere in the other Unit protocols.

      2. General guidance regarding DNA evidence handling procedures that is applicable to all Laboratory Units. This information should be provided in Laboratory-wide documents, including the Evidence Control Policy of the FBI Laboratory Division Quality Assurance Manual, and the Procedures for the Examination of Evidence section of the FBI Laboratory Division Caseworking Procedures Manual, and should describe the requirements to process the evidence one case at a time, and to handle evidence one item at a time. The guidance also should identify procedures to ensure that one Unit does not compromise the ability of another Unit to test the same item of evidence.

      3. Unit-specific guidance on DNA evidence handling procedures. This information should be included in the Evidence Control section of the DNA Analysis Unit I Quality Assurance Manual, and should contain all evidence handling, contamination-prevention, and evidence- preservation methods universal to all technical DNAUI staff.

      4. An identification of facility access limitations, including a description of the context and limitations for access to DNAUI areas by non DNAUI personnel. This information should be added to Facilities (Security) section 6.1.

   5. Ensure that the Evidence Control Policy of the FBI Laboratory Division Quality Assurance Manual, the Evidence Control and Facilities (Security) sections of the DNA Analysis Unit I Quality Assurance Manual, and the Procedures for the Examination of Evidence section of the FBI Laboratory Division Caseworking Procedures Manual cross-reference other relevant guidance to ensure that staff members know whether additional information is located in other manuals.

   6. Implement a policy that requires staff members to certify that they have read, understand, and will comply with each protocol that governs DNAUI operations, as well as any approved revisions that have not yet been incorporated into the protocols.

   7. Revise policies for securing evidence under active examination to reflect the current availability of independently securable storage at staff member workstations, and of a securable bulky evidence examination room. The policies should require the use of these new facilities rather than leaving the security of unattended evidence under active examination dependent upon facility access limitations.

   8. Ensure that the Extraction and Amplification sections of the Short Tandem Repeat Analysis Protocol include:

      1. A requirement that the known and unknown (evidence) samples are processed separately during examination, extraction, and amplification, including references to guidance that specifies the amount of time and space that meets the intent of the term "separation."

      2. A requirement for the "separation" of high- and low-quantity DNA samples, as well as references to guidance that specifies the amount of time and space that meets the intent of the term "separation."

   9. Ensure that the Extraction, Amplification, and Laboratory Set-up sections of the Short Tandem Repeat Analysis Protocol include:

      1. Information on the use, cleaning, and decontamination of the "transport trays" used by PCR Biologists to move samples to the amplification area.

      2. General evidence handling information.¹⁰⁰

      3. In the Extraction section, a prohibition on having two sample tubes open at once, clarification in section 4.1.3 regarding required incubation times, and further explanation concerning when the additional organic extractions permitted by section 4.3-9 are appropriate.

      4. In the Amplification section, a requirement that control samples be processed last, and in section 6.7, the specification of the order in which sample tubes should be set up.

   10. Ensure that the Procedures for the Serological Identification of Biological Substances on Evidentiary Materials includes:

       1. Detailed guidance on proper evidence handling methods, similar in content to the guidance contained in the Laboratory Set-up section of the Short Tandem Repeat Analysis Protocol.

       2. In the Procedures for the Preparation of Dried Bloodstains from Coagulated Whole Blood and the Procedures for the Preparation of Dried Bloodstains from Anticoagulated Whole Blood, a requirement to: 1) pretest the cotton sheeting to ensure that there is no DNA contamination; and 2) identify the amount of blood to use when making dried blood stains.

       3. In the Procedures for the Extraction of Suspected Semen Stains: Quality Control Procedures and Procedures for the Extraction of Suspected Semen Stains: Questioned Stain Extraction Procedure, guidance regarding: 1) the usable life of the positive semen control; 2) the size of the swab to use for testing; 3) the need to use disposable paper to reduce the risk of contamination; 4) whether extracts and swabs are both sent to the Biologist for testing; and 5) what happens after the stain extraction procedure has been completed (i.e., what occurs if there is a negative or positive result).

       4. Unwritten internal controls that already are in use by DNAUI staff members and management, including (but not limited to): (a) the requirement for a team's PCR Biologist to record the characteristics of the evidence, supplementing the description generated by the Serologist; and (b) the requirement for Serologists to perform a "general swabbing" of an item to ensure that no possible sources of DNA on that item have been missed.

   11. Ensure that the Case Documentation Policy section found in the FBI Laboratory Division Quality Assurance Manual contains:

       1. A listing of the minimum contents for all Unit case files, along with a reference to that part of each Unit's case documentation and review protocol that addresses case file contents; and

       2. Guidance on notetaking methods and requirements common to all Units, along with a reference to the corresponding Unit-specific protocols.

   12. Ensure that the Case Assignment, Documentation, and Review section found in the DNA Analysis Unit I Quality Assurance Manual contains:

       1. A detailed description of case file review procedures, including a checklist to facilitate the review and to document that the review accounts for each key item in the case file;

       2. Guidance on notetaking methods to ensure that DNAUI staff members understand how and when they should take notes; and

       3. A description of the procedures that must be followed to review and confirm case evidence profiles for entry into CODIS, or at a minimum, a reference to where those procedures are described in another policy document.

   13. Ensure that the Guidelines for Control Samples and Interpretation of Control Samples sections within the Short Tandem Repeat Analysis Protocol contain comprehensive guidance on each of the following: (a) the procedure to complete a case file review; (b) the difference between the review responsibilities of Examiners and of the PCR Biologists; (c) the circumstances in which a control result should cause an entire analysis run to "fail;" and (d) how and when staff members should use section 10.3.3 of the Interpretation of Control Samples. Further, these sections should contain a checklist or summary sheet to assist reviewers to verify the completeness of their work.

   14. Ensure that the Organization and Management and Authority and Accountability sections of the DNA Analysis Unit I Quality Assurance Manual contain:

       1. A complete description of the characteristics, responsibilities, interrelation, and limitations of each job position in the various DNAUI teams;

       2. Problem-resolution guidance for each team member position that describes how to respond to operational and personnel problems (such as suspicion of protocol noncompliance), and that clearly delineates the options for resolution available to staff members; and

       3. A clear statement identifying Laboratory personnel who have the authority to halt DNAUI operations if a significant problem is detected.

   15. Include comprehensive guidance for problem response and resolution, similar to that contained in the DNAUI Quality Assurance Manual, in the FBI Laboratory Division Quality Assurance Manual.

2. Incorporate Decision Aids

   In addition to protocols that fail to specify the procedures that DNAUI staff members should follow when they analyze DNA, our review identified protocols that do not describe adequately the decision criteria staff members should employ when their duties require them to exercise discretion in the testing process. To address this deficiency, we recommend that DNAUI management:

   16. Add work-flow diagrams and decision trees to its protocols to assist staff members to exercise properly their judgment during the DNA testing process. These aids would help to structure decision-making and to ensure that staff members do not overlook relevant information. At a minimum, the following protocol sections should include decision-making aids:

       1. in the Procedures for the Serological Identification of Biological Substances on Evidentiary Materials: 1) the Procedures for the Preparation of Dried Bloodstains from Coagulated Whole Blood and from Anticoagulated Whole Blood; and 2) Procedures for the Extraction of Suspected Semen Stains: Questioned Stain Extraction Procedure;

       2. in the Short Tandem Repeat Analysis Protocol: 1) the GeneScan Analysis section, and 2) the Interpretation of Control Samples section.

   17. Evaluate protocols beyond those listed above, including all of the serology procedures, for process descriptions that would benefit from work flow and decision diagrams.

3. Enhance Notetaking Requirements

   During our review we identified three procedure and protocol sections that depend upon the precision of manual notetaking but that lack comprehensive guidance on notetaking methods. To address this deficiency, Laboratory and DNAUI management should:

   18. Supplement documentation guidance found within the Case Documentation Policy in the FBI Laboratory Division Quality Assurance Manual, the Evidence Control section in the DNA Analysis Unit I Quality Assurance Manual, and the Procedures for the Examination of Evidence in the FBI Laboratory Division Caseworking Procedures Manual, to include comprehensive guidance on notetaking methods.

   19. Require staff members to document contemporaneously the testing performed in each case.

   20. Include in the Unit-specific protocols cleaning and decontamination techniques designed to reduce the risk of contamination or cross-transfer as staff members move back and forth between the evidence items they are examining and their computer keyboards to take notes.

4. Update Protocols

   Our review identified several protocols that are outdated and no longer reflect current procedures in use in the DNAUI. To address this deficiency, DNAUI management should:

   21. Revise the Case Assignment, Documentation and Review section of the DNA Analysis Unit I Quality Assurance Manual, and the Amplification, STR Typing: Setting up a Run, and GeneScan Analysis sections of the Short Tandem Repeat Analysis Protocol to include the current requirement that Examiners review GeneScan® data for all samples that show no DNA peaks on the Genotyper® print-out.

   22. Review the protocol revision process and identify ways to expedite it. Protocols should not be so difficult to update that administrative burden justifies not keeping them current.

5. Implications for DNAUII

   Based upon the extent of the vulnerabilities identified within DNAUI's protocols, we believe that the risk exists that DNAUII's protocols contain vulnerabilities of a similar nature, vulnerabilities that will not be remedied completely by the improvements made as a result of our preceding recommendations. Therefore, we make the following recommendation.

   23. The recommendations in this report should be applied to DNAUII where applicable. In addition, DNAUII management should conduct a comprehensive vulnerability assessment of its own protocols and practices, similar in extent and focus to the assessment the OIG has conducted on the protocols and practices of the DNAUI, and remedy all vulnerabilities identified by that review. We believe that such an assessment will have a greater degree of success if DNAUII management solicits the participation of scientists outside the DNAUII, who can bring an unbiased perspective to the assessment.

III. REMEDY PRACTICE VULNERABILITIES

Our fieldwork focused on the DNAUI staff's application of the protocols we identified as deficient. The information we collected establishes that the DNAUI needs to: 1) promote greater consistency in DNAUI team operations; 2) develop a comprehensive, written training curriculum; 3) improve information dissemination; and 4) implement an information management system to improve evidence tracking capabilities and efficiency of operations.

1. Promote Greater Consistency in Team Operations

   Our interviews of DNAUI staff members indicated that variation exists in the operations of the Unit's teams. Unwarranted flexibility in DNAUI operations can communicate to staff members that they are free to develop idiosyncratic work practices, which can create an environment with greater risk for inadvertent or willful noncompliance with protocols. To promote greater consistency and accountability in DNAUI functions, we recommend that Laboratory and DNAUI management:

   24. Document and standardize the best practices of the Unit's teams and incorporate them in protocols.

   25. To the extent practicable, minimize managerial flexibility permitted in team operations, even in areas considered to be of no "impact" to the analysis process (as we have defined the term for our assessment).

   26. Document in the protocols those flexibilities that remain, with an explanation of the circumstances in which team members have leeway to exercise discretion and to vary testing methods. This documentation should provide staff members with clear guidance on the constraints placed by management on their discretion, so that they can be alert to practices that exceed those constraints. As part of this documentation, the protocols should include notations to staff where the precision of their actions can fall within an acceptable range (i.e., if the protocol calls for a 2-hour incubation time, but 1 hour and 50 minutes is acceptable, the protocol should reflect the range of time that is considered scientifically acceptable).

   27. Ensure that protocols and training materials address the risks posed by protocol drift and protocol noncompliance, and prohibit individual staff or team variation from the protocols. This information should build upon the above-required analysis of team variations, the limitations placed upon existing flexibilities, and the delineation of those procedures where a scientifically acceptable range of activity is permitted. In addition, in circumstances where DNAUI staff members deviate from the protocols, they should be encouraged to note the deviation and its degree in the case file.

2. Formalize Training

   Our review of DNAUI training practices revealed that the Unit lacks a comprehensive curriculum and that training consists largely of individual discussions with a mentor and presentations given by various experienced staff members. To address this deficiency, DNAUI management should:

   28. Convert this "oral tradition" of training into a comprehensive, written curriculum to ensure that trainees receive consistent instruction that comports with the protocols. As part of this process the Laboratory should:

       1. Collect the training materials that currently are in use and organize them into a coherent written course of study;

       2. Cross-reference the training materials to the protocols; and

       3. Ensure that the training materials reflect the standardized best practices and limited flexibilities established in recommendations 18-20.

   29. Ensure that the training materials are kept current as the DNAUI protocols evolve.

3. Improve Information Dissemination

   Laboratory/or DNAUI management (as appropriate to the recommendation) should:

   30. Implement requirements that will ensure that managers within the Unit disseminate protocol-related information promptly and accurately to those under their supervision.

   31. Ensure that serology and PCR program managers inform Unit management (including team supervisors) of changes to procedures employed in the DNAUI.

   32. Solicit and evaluate technical input from staff members on significant changes that affect Unit procedures and protocols.

   33. Develop and implement a communications plan that allows DNAUI staff members to provide feedback on the effectiveness of the Unit's protocols.

4. Implement a Laboratory Information Management System

   We determined that Laboratory management had begun to lay the groundwork for the implementation of a Laboratory Information Management System (LIMS) in 2002, and has since made progress toward the implementation of a LIMS. However, given the benefits that a LIMS will bring to evidence tracking and chain-of-custody documentation, we recommend that Laboratory management:

   34. Ensure that a LIMS is successfully implemented. To accomplish this, Laboratory management must provide to all appropriate personnel:

       1. Ready access to the system;

       2. Adequate training on the proper use of the system; and

       3. The resources needed to convert their current methods and operations to those that will maximize the capabilities of the LIMS.

   35. Retain full utilization of the LIMS as one of the top administrative priorities of the Laboratory. To accomplish this, Laboratory management will need to devote sufficient resources to the LIMS to ensure that it keeps pace with the changes and developments in technology that will occur over time as the Laboratory evolves.

100. Currently, this information is included in the Laboratory Set-up section. We recommend that it be included in all sections of the Short Tandem Repeat Analysis Protocol that call for the handling of evidence, but at a minimum, in the Extraction and Amplification sections.