Combined DNA Index System Operational and Laboratory Vulnerabilities

Audit Report 06-32
May 2006
Office of the Inspector General


Executive Summary


The Federal Bureau of Investigation (FBI) serves as one of the primary components in the Department of Justice’s efforts to further develop the nation’s capacity to prevent and control crime and administer justice fairly and effectively. The FBI assists in these efforts through various means, including providing direct technical support to state, local, and tribal law enforcement. One of the most powerful law enforcement tools that the FBI provides is the Combined DNA Index System (CODIS), a national DNA‑profile matching service comprised of databases containing DNA profiles from crime scenes, convicted offenders, and sources involving missing persons.

DNA, or deoxyribonucleic acid, is a chemical contained in the nucleus of a cell that carries the genetic instructions, or blueprint, for making living organisms. In the context of criminal investigations, scientists examine the DNA that varies widely among people to develop a profile that will be uniquely identifying (except in the instance of identical twins). DNA analysis, a relatively new law enforcement tool, can provide compelling evidence for solving crimes or exonerating suspects. The FBI began the CODIS Program as a pilot project in 1990, allowing participating laboratories to compare DNA profiles obtained from crime scenes and convicted offenders to generate investigative leads.

This Office of the Inspector General (OIG) audit report examines various aspects of CODIS operations and management to discern whether vulnerabilities exist in the FBI’s administration of CODIS.

Background

The FBI implemented CODIS as a database, distributed over three hierarchical levels, that enable federal, state, and local crime laboratories to compare DNA profiles electronically. The National DNA Index System, (NDIS), which became operational in 1998, is the highest level in the CODIS hierarchy. It enables the laboratories participating in the CODIS Program to compare DNA profiles on a national level. Each state maintains a State DNA Index System (SDIS), and participating local laboratories across the country each maintain a Local DNA Index System (LDIS). DNA profiles are entered into CODIS by local and state laboratories, which then flow to the state and national levels where they are compared to determine if a convicted offender can be linked to a crime, if crimes can be linked to each other, or if missing or unidentified persons can be identified.

The CODIS Program is operated by the CODIS Unit, within the FBI Laboratory Division, Scientific Analysis Section, Forensic Analysis Branch. The CODIS Unit is charged with overseeing CODIS and NDIS operations and administration, and ensuring that those operations comply with applicable legislated requirements.

As of November 2005, 175 laboratories were participating in NDIS. These laboratories collectively uploaded nearly 2.9 million profiles to NDIS, including:

  • 2,743,068 convicted offender profiles;

  • 123,835 crime scene (forensic) profiles;

  • 1,481 relatives of missing person profiles;

  • 621 unidentified human remains profiles; and

  • 269 missing person profiles.

The success of CODIS is measured primarily through the number of cases that CODIS assists through a “hit” (a match between DNA profiles produced by CODIS that would not otherwise have been developed), also referred to as “investigations aided.” Through November 2005, CODIS aided 29,666 investigations in 49 states and 2 federal laboratories.

Prior Audits of CODIS

The OIG previously conducted an audit to determine the extent of state and local laboratory participation in CODIS, particularly for those entities receiving laboratory grants, and to evaluate the FBI’s implementation and monitoring of CODIS.1 As part of that audit, we reviewed eight individual laboratories to determine their compliance with applicable statutes and FBI standards.2 That audit report, issued in 2001, concluded that:

  • The FBI needed to improve its oversight of CODIS-participating laboratories to ensure the laboratories were in compliance with applicable legislation, the FBI’s Quality Assurance Standards (QAS), and the FBI requirements for laboratories participating in NDIS.

  • The FBI needed to initiate procedures to ensure that DNA profiles in CODIS are complete, accurate, and allowable.

As a result of these findings, we made the following recommendations to the FBI:

  • Require that the accuracy, completeness, and allowability of the DNA profiles in NDIS be routinely verified through audits or other means.

  • Ensure that analysts performing DNA testing at laboratories uploading DNA profiles to NDIS are aware of the NDIS participation requirements, particularly those requirements delineating the types of allowable profiles.

  • Develop and implement a process to ensure that laboratories adequately resolve all deficiencies noted during the QAS‑required audits.

Since the issuance of the 2001 audit report, the OIG has completed an additional 24 CODIS laboratory audits.3 This audit report follows up on our previous report and assesses the FBI’s administration of CODIS operations.

Audit Approach

This audit was designed to assess the status of CODIS operations and CODIS trends and vulnerabilities. The specific objectives of the audit were to:

  1. assess the adequacy of the FBI’s administration of CODIS, including its oversight of NDIS;

  2. analyze findings from DNA laboratory audits, both OIG‑conducted audits and external quality assurance audits, to determine if they reveal trends and vulnerabilities; and

  3. evaluate the FBI’s implementation of corrective actions in response to findings from the OIG’s September 2001 audit, The Combined DNA Index System.

To accomplish these objectives, we reviewed various data and documentation provided to us by FBI officials, evaluated the results of past OIG CODIS laboratory audits, interviewed members of the CODIS Unit staff, and collected and analyzed documentation from select NDIS-participating laboratories.

Additionally, to obtain the viewpoints of state and local NDIS‑participating laboratories, we surveyed CODIS administrators at those laboratories (not including the FBI).

Summary of OIG Findings

We identified several recommendations for the FBI to: (1) improve its administration of CODIS, (2) track and respond to CODIS trends and vulnerabilities, and (3) improve or complete its corrective action to our 2001 audit, as summarized in the following sections.

FBI Administration of CODIS

The FBI received an overall positive evaluation of its administration of CODIS from the CODIS administrators we surveyed. We determined that the FBI also has given attention to CODIS infrastructure, development, and staffing. However, based on our analysis of the survey responses and FBI documentation, we have identified several areas in need of further improvement. For example:

  1. QAS compliance within the CODIS community can be improved and workloads reduced if the FBI ensures that all CODIS administrators receive QAS auditor training;4

  2. CODIS Unit responsiveness can be improved through sufficient staffing and tracking of information requests;

  3. CODIS community understanding and compliance with profile allowability restrictions can be enhanced through increased emphasis on written sources of guidance available to all CODIS users;

  4. NDIS Audit Review Panel (Review Panel) timeliness can be improved if guidance is disseminated to the appropriate members of the community, who can ensure that submissions to the Review Panel are complete;5 and

  5. The FBI can improve information sharing through better use of the CODIS intranet website to disseminate written guidance to the community that is easy to navigate, consistent, and practical.

In addition, from our review of historical staffing data, we found that in the several years prior to 2004, the FBI failed to staff the CODIS Unit commensurate with growing demands and participation, and thereby put at risk the ability of CODIS staff to properly oversee and administer the CODIS Program. However, in February 2004, FBI management took action to increase CODIS staffing and reaffirm the importance of a sufficient number of program manager positions. Yet, progress in filling the positions assigned to the CODIS Unit has been limited due to a variety of delays and difficulties. Of particular concern is the on-going lack of an NDIS Program Manager, especially in light of the trends and vulnerabilities we identify in our report related to the compliance of NDIS-participating laboratories with standards governing participation. Therefore, we recommend that the FBI make concerted efforts to bring the CODIS Unit up to full staffing levels.

Further, in the written documents provided to us, the FBI appears to capture the mission, goals, objectives, strategies, and performance measurements for the CODIS Unit. These documents are interlinked in a way that allows the performance measurements to be meaningful and measurable. However, we identified three activities which are not reflected in the CODIS Unit’s performance measurements that are an essential part of the Unit accomplishing its mission: (1) auditing of NDIS data; (2) providing training on QAS compliance; and (3) overseeing the activities of the Review Panel. These three activities comprise the CODIS Unit’s primary means of monitoring and assisting NDIS-participants’ compliance with the QAS and verifying the integrity of NDIS data. Consequently, we recommend that these three activities should be formalized and clearly reflected as the CODIS Unit’s responsibilities in its objectives and performance measurements.

The FBI has taken measures to provide for the operations, maintenance, and security of the CODIS system for the near future. However, continued progress is needed to ensure that the development contract process planned for fiscal year (FY) 2006 is completed, and that the development contract awarded allows for continued responsiveness to legislated changes to CODIS operations.

Trends and Vulnerabilities in the CODIS Community

In assessing the results of the OIG CODIS laboratory audits completed in FY 2004 and FY 2005 (a total of 18 audits), we found that common findings occurred with greatest frequency in the two areas of review that are not audited by QAS auditors within the DNA community: compliance with NDIS participation requirements and the proper upload of forensic profiles to NDIS. Further, the FBI does not intend to have CODIS Unit auditors, once hired, routinely audit compliance with NDIS requirements. Instead, the FBI relies upon the annual CODIS user certifications as the primary means of ensuring the compliance of NDIS data.6 From the trends we noted, we concluded that this reliance is insufficient, for the following reasons.

  • We noted 13 incidents where forensic profiles in NDIS violated some aspect of NDIS requirements. This occurred in 11 of the 18 laboratories we audited, and suggests that the annual certification forms have not been successful in ensuring CODIS user compliance with profile allowability restrictions.

  • We found that 6 of 18 laboratories we audited had not completed the annual user certification forms as required. The forms are completed by laboratories on a self-certification basis and are not required to be submitted to the FBI.

In addition to our assessment of the OIG CODIS laboratory audits, we examined 41 state and local external QAS audits conducted by QAS auditors within the DNA community.7 We identified trends in findings that implicate significant aspects of laboratory operations, such as chain-of-custody documentation; labeling of evidence and security of evidence storage; and proper monitoring of critical reagents, equipment, and procedures. Further, 10 percent of the findings noted were overturned after examination by the Review Panel, in some cases without full disclosure of the overturned findings to the audited laboratories.8 In addition, we determined that the FBI is not systematically and completely tracking common and overturned findings. Without a thorough understanding of trends in common findings, the FBI cannot properly provide the CODIS community additional guidance needed to remedy and prevent compliance weaknesses in the trend areas. Without an understanding of trends in overturned findings, the FBI also cannot take the necessary steps to guide all QAS auditors toward a consistent interpretation and application of the standards and to ensure that QAS auditors obtain feedback on their performance.

Overall, we believe the weaknesses we identified leave the FBI potentially vulnerable to undetected inadvertent or willful non-compliance by CODIS participants and consequently could undermine the integrity of the CODIS Program. We conclude that the FBI needs to develop internal controls over compliance of NDIS data beyond its current reliance on the annual certification forms, and should track audit findings to obtain the type of information that will be beneficial to auditors and audited laboratories.

Implementation of Corrective Action

Previous OIG audit findings identified the need to verify the compliance of NDIS data, to ensure NDIS user compliance with NDIS requirements, and to ensure that laboratories remedy QAS audit findings.

The FBI’s corrective action approach to the need to verify NDIS data was two-fold. First, the FBI began requiring FBI QAS auditors to review CODIS profiles as part of their case file reviews (this action was initiated in June 2004). Second, the FBI began taking steps to hire auditors who would systematically audit the profiles contained in NDIS. In assessing this action, we determined that the FBI QAS auditor methodology for reviewing profiles is deficient due to its limited scope. In addition, the FBI does not intend to have the CODIS Unit auditors, once hired, expand the current methodology to include broader profile reviews. Further, the FBI has not implemented a mechanism to document and track how many profiles are confirmed during these reviews, or the frequency with which these reviews are conducted.

To address the need to ensure NDIS user compliance with NDIS requirements, the FBI instituted a requirement for annual CODIS user certifications, completed on a self-certification basis. However, the process for completing these forms does not provide the FBI with the information it needs to confirm that all CODIS users have completed the forms as required. Further, the continued reliance on self-certification perpetuates the weakness we noted in the 2001 audit.

Finally, the FBI implemented various corrective action measures in response to the need for greater oversight of QAS compliance and the adequacy of laboratories’ responses to QAS audit findings. These measures included conducting QAS auditor training courses, implementing a DNA community-wide audit document, and creating the Review Panel to ensure complete and appropriate corrective action to QAS audit findings. However, we identified the need for improved Review Panel timeliness and improved consistency in training through an emphasis on written guidance.

Conclusion and Recommendations

We found that while the FBI has made improvements to several aspects of CODIS operations, the FBI needs to make further improvements to ensure that it properly oversees the CODIS Program and CODIS participants. Further, we identified several opportunities for data tracking and information sharing that would enable the FBI to better assist the CODIS community in its understanding of and compliance with the QAS and NDIS participation requirements.

Accordingly, we made 22 recommendations for corrective actions that are needed for the FBI to improve its administration of CODIS. Among these recommendations are for the FBI to:

  • Develop and implement a plan to ensure that all CODIS administrators attend the FBI QAS auditor training.

  • Improve information sharing through enhancements to the CODIS website.

  • Develop communication policies that will allow the CODIS Unit to provide guidance to members of the DNA community in writing.

  • Develop a staffing plan that identifies current hindrances to filling vacant positions in the CODIS Unit, solutions to those hindrances, and a timeline of action.

  • Incorporate the three activities we identified (auditing of NDIS data, providing training on QAS compliance, and overseeing the activities of the Review Panel) into the CODIS Unit’s objectives and measurements to fully reflect the CODIS Unit’s efforts to address its mission.

  • Ensure that the internal controls over the compliance of NDIS data are strengthened beyond the current reliance on self-certification annual reminder forms.

  • Implement a formal mechanism for tracking findings in audits reviewed by the NDIS Audit Review Panel and for tracking QAS auditor performance.



Footnotes
  1. Department of Justice, Office of the Inspector General. Audit Report No. 01-26, The Combined DNA Index System, September 2001.

  2. Of the eight laboratories, three were in Florida and one each in California, Illinois, North Carolina, Pennsylvania, and Virginia. See Appendix V, “FY 2000” list, for further details.

  3. See Appendix V for a complete listing of the CODIS laboratory audits conducted by the OIG.

  4. The FBI conducts training courses for auditors assessing compliance with the QAS within the DNA community. The primary focus of these courses is to ensure a consistent understanding of the QAS and consistent application of the FBI's audit document.

  5. The NDIS Audit Review Panel is a group of volunteer members of the DNA community who meet specific requirements, as well as FBI DNA staff members. The panel reviews all external QAS audits conducted at NDIS-participating laboratories across the country, with the purpose of ensuring consistent and thorough application of the QAS by the auditors and appropriate and complete corrective action by the laboratories.

  6. At the beginning of each calendar year, each laboratory’s CODIS Administrator is required by NDIS procedures to ensure that each CODIS user is reminded of the categories of DNA data accepted at NDIS. As part of that, the CODIS Administrator has individual users certify that they have received their annual reminder and understand and will abide by what DNA data is accepted at NDIS.

  7. We use the term “QAS auditors” to refer to the scientists within the DNA community who perform QAS audits.

  8. The Review Panel overturns a finding when it determines that the finding was not justified based upon the commonly accepted interpretation of the QAS. Often, for this to occur, the audited laboratory must challenge the finding before the Review Panel.



« Previous Table of Contents Next »