Combined DNA Index System Operational and Laboratory Vulnerabilities
Audit Report 06-32
Office of the Inspector General
We conducted our audit in accordance with the Government Auditing Standards and included such tests as were considered necessary to accomplish the audit objectives. Our audit generally covered the period from October 2003 through November 2005, although in some instances it was necessary to consider documentation from outside that timeframe. The objectives of this audit were to:
To accomplish the objectives of this audit we:
The following sections provide additional detail for work that specific actions listed in the preceding list.
OIG CODIS Administrator Survey
Using information obtained during meetings with the FBI and CODIS administrators, including issues that were raised during open discussion at the SDIS administrator's meeting in May 2005, we developed a survey for completion by CODIS administrators at NDIS‑participating laboratories. The survey provided us with feedback on the FBI’s administration of CODIS and laboratory concerns about quality issues or problems in the CODIS community. We included open-ended and static-option questions. For those questions where we provided static options, we included space for miscellaneous comments. In addition, we assured the CODIS administrators that responses would be confidential and individual responses would not be singled out in a way that could identify the source of the information.
After the initial draft of the survey was created, we tested the survey on members of the CODIS community. We used information received from the FBI to select experienced individuals to test the survey, being careful not to select CODIS administrators to preserve the universe for our final survey. We contacted six members of the DNA community and all six responded. Additional revisions were made to the survey based upon the test respondents’ comments, and were reflected in the final version.
The final version of the survey was e-mailed to 174 CODIS administrators on June 7, 2005. A list of the CODIS participating laboratories to which we sent the survey appears in Appendix II. CODIS administrators were initially given until June 24, 2005, to complete the survey and return it via e-mail, fax, or U.S. mail. On June 21, 2005, we sent out a reminder e-mail and, on June 28, 2005, we sent out a third e-mail extending the deadline to July 7, 2005. We extended the deadline to give non-responding states a chance to reply.
As of July 7, 2005 (the extended deadline), we had received 139 surveys. However, there were 6 states from which we still had not received a response. In a final attempt to give these six states a chance to submit a survey we contacted them via e-mail on July 29, 2005.
After all the extensions (August 15, 2005 was the final cut-off date) we still had not heard from Idaho and Rhode Island. We noted that a member of the Connecticut laboratory had provided a response, but did so during the test phase of the survey, and since the survey changed after that response was received, we could not include it in our results.
The survey contained 46 questions which were broken into seven sections: (1) demographics, (2) FBI CODIS Unit responsiveness, (3) allowability of DNA profiles, (4) laboratory quality, (5) general CODIS operations, (6) NDIS Audit Review Panel, and (7) FBI guidance to the CODIS community.
Additional information about how we tallied the survey responses, and a summary of the actual responses received can be found in Appendix VII.
OIG CODIS Laboratory Audits
During our audit we analyzed a total of 18 OIG CODIS laboratory audits, of which 6 were issued in final for FY 2004 and 12 were issued in final for FY 2005.53 A list of the 18 OIG CODIS laboratory audits is contained in Appendix V.
We identified and analyzed trends from each audit specific to profile allowability as well as the number of findings for the five different QAS sections reviewed during the audits, as follows.
External QAS Evaluations
We tested the FBI’s records for the Review Panel and the external QAS evaluations submitted to it. We judgmentally selected 10 participating states and compared the FBI’s electronic records against the written records used to create the electronic records. We found no material differences in the tracking system. As a result we did not expand our sample and relied on the information contained in the electronic records.
In addition to reviewing the tracking system for data reliability, we determined if delays in the process were significant, if the timeliness was improving, and whether the Review Panel members were meeting the 30‑day deadline set forth in the NDIS procedures for reviewing audits. In order to see if the timeliness had improved we analyzed the information contained in the FBI’s records for both 2003 and 2004.54
Working with the information provided to us from the FBI, we determined that there were 72 closed evaluations in 2004 and 11 closed evaluations in 2005. We tested 50 percent, or 41, of these evaluations.
We used random sample selection over the 2 years of interest since our goal was to conduct a trend analysis and to look at findings at more laboratories. Further, we stratified our sample to ensure we selected a percentage of SDIS and LDIS laboratories representative of the whole universe. We excluded laboratories the OIG had already audited, to avoid requesting information similar to what had been requested previously.
We notified the CODIS Unit Chief that we would be contacting the laboratories to request copies of external QAS evaluations and related correspondence. We provided him with copies of any written correspondence we issued in order to acquire the documentation we needed to complete our review.
Using contact information obtained from the FBI we contacted CODIS administrators for each of the laboratories and explained the following:
The 41 laboratories in our sample represent 19 states and 1 federal agency. We analyzed the evaluations in our sample for trends and statistics. Specifically, we calculated:
In our analysis, we relied upon the findings and conclusions of the QAS evaluators within the DNA community, and did not perform any assessment as to the scope of their work. In addition, we did not confirm whether those evaluators met the requirements for conducting external QAS evaluations, specifically the requirement that they successfully complete the FBI’s QAS auditor training.
We compared the documentation received from the laboratories to the information the FBI had provided in its Review Panel record spreadsheets, to verify accuracy of those records. We also tracked whether the Review Panel had to follow up with the laboratories and whether findings were challenged by the laboratories, to determine if those issues impeded the timeliness of the Review Panel process.
|« Previous||Table of Contents||Next »|