The FBI response to the draft audit report appears in Appendix VIII. In its response, the FBI generally agreed with our recommendations and described the corrective actions it has taken or intends to take with regard to the recommendations. However, the FBI disagreed with a few of the recommendations, and these are identified as “unresolved” in the listing below. The status of the individual recommendations is as follows:
Resolved. This recommendation can be closed when we receive documentation that: (1) the special QAS auditor training classes scheduled for the fall of 2006 have been conducted, and that current CODIS Administrators who have not yet had this training were in attendance; and (2) the NDIS MOU has been revised to reflect a requirement that new administrators receive this training prior to assuming their full CODIS duties.
Resolved. This recommendation can be closed when we receive a description of the changes the FBI has implemented to enhance the information sharing capabilities of the CODIS website.
Resolved. This recommendation can be closed when the FBI provides documentation that it has developed user-friendly resources, on the CODIS website or through other means, for allowing CODIS users to expand and test their understanding of profile allowability.
Relatedly, we believe the FBI needs to reconsider its firm stance against a decision-tree type tool. While we acknowledge that every factual scenario presents different nuances of detail, there are many scenarios that ultimately can be distilled into a series of questions. In other words, to determine profile allowability, a CODIS user has to answer a series of question for each scenario and these could be captured in a tool similar to a decision tree. For example, questions could be given in a series, as follows: “Does this profile, in whole or part, match the victim’s profile? If yes, then is there a suspect or other known profile available to compare to, that enables deduction of the victim’s portion? If yes, then the victim’s portion should not be uploaded to NDIS.” Such a tool would not address every situation, but it would help users reason through the major factors that they should consider to determine allowability. The OIG continues to find unallowable forensic profiles in its CODIS laboratory audits, even in laboratories with experienced CODIS users. We believe that even rudimentary tools that are easy to use and understand would be an assistance to CODIS users as they develop their own understanding of allowability.
Resolved. This recommendation can be closed when we receive a copy of the formal request that has been sent to the SWGDAM Chairman regarding the material operational weaknesses identified during our audit by CODIS Administrators. The FBI did not provide the draft correspondence to SWGDAM noted in its response.
Resolved. This recommendation can be closed when we receive documentation that a listing of appropriate contacts for QAS audit resolution in each CODIS laboratory has been developed, and that guidance has been provided to those contacts on how they can ensure that their submissions to the NDIS Audit Review Panel are complete and appropriate to facilitate resolution.
Resolved. This recommendation can be closed when we receive a written policy or procedure formalizing the process described in the FBI’s response, and documentation of its implementation. The FBI did not provide the draft copy of a request log noted in its response.
Resolved. This recommendation can be closed when we receive the written policy or procedure that formally describes how the CODIS Unit ensures that it provides written guidance to the CODIS community to the fullest extent possible.
Resolved. This recommendation can be closed when we receive a written plan that identifies where delays and hindrances have occurred in filling long-standing vacant CODIS Unit positions, and specific actions being taken to address those delays and hindrances to facilitate full staffing levels. This plan can include such actions as pursuing other avenues of advertising the positions, as described in the FBI response.
Resolved. This recommendation can be closed when we receive documentation that each CODIS Unit position’s duties, responsibilities, and routine activities have been memorialized into a form of training manual for that position.
Resolved. This recommendation can be closed when we receive documentation of the formalization of the three activities we describe into performance measurements for the CODIS Unit.
Resolved. This recommendation can be closed when we receive documentation of the completion of the development contract as well as a description of how that contract provides for continued flexibility to legislative changes to CODIS operations.
Unresolved. The FBI disagrees with the strength of the OIG’s evidence to support this recommendation, as well as what it views as a generalization that the certification forms have not fully accomplished their purpose of ensuring compliance. Yet, the OIG’s evidence shows that one-third of the audits we conducted over a 2-year period (6 of 18) found that the laboratories had not completed the forms as required. Further, roughly two-thirds of the audits we conducted in that 2-year period (11 of 18) revealed forensic profiles that were not acceptable, based upon FBI-established criteria.
To support its argument, the FBI draws a comparison between our results in our 2001 audit and current audit trends. FBI management states that we found 40 instances of inappropriate DNA profiles uploaded to NDIS by 5 out of the 8 laboratories audited. This comparison is false in that it compares the number of profiles identified, from our previous report, to the number of laboratories at which those profiles were found, as we quote in our current report. To be consistent, the comparison should state that our 2001 report identified forensic profiles that were not acceptable at 6 of the 8 laboratories we audited. Consequently, a reduction from a 75 percent incident rate (6 of 8) in our 2001 audit report to a 61 percent incident rate (11 of 18) in FY’s 2004 to 2005 audit reports is not sufficient to support a claim that the annual reminder forms have accomplished their intended purpose.
Further, the FBI argues that other measures are being taken as part of the internal controls over the appropriateness of data uploaded to NDIS. Such an argument actually supports our recommendation, since our recommendation encourages the FBI to take other measures. This is particularly true in light of the fact that one of the key measures the FBI mentions, the addition of special instruction to each CODIS training class, has been implemented since our audit work concluded. Consequently, we conclude that the FBI’s support for disagreement with our recommendation is not sufficient to set aside the legitimate evidence supporting our recommendation.
Resolved. This recommendation can be closed when we receive: (1) documentation that a formalized tracking system has been implemented to identify common and overturned findings from the audits reviewed by the NDIS Audit Review Panel, and (2) a policy for how that information will be used to enhance community consistency and compliance.
Resolved. This recommendation can be closed when we receive: (1) documentation that a formalized tracking system has been implemented to identify auditors who use inconsistent interpretations of the QAS, and (2) a policy for what action should be taken when such auditors are identified.
Resolved. This recommendation can be closed when we receive documentation that a mechanism has been developed to systematically communicate the information gathered in response to recommendation nos. 13 and 14 to training providers in the DNA community, including the FBI’s own QAS audit trainers.
Resolved. This recommendation can be closed when we receive a copy of the formal policy for the conducting of profile allowability reviews on behalf of the CODIS Unit that reflects: (1) the expanded size of the reviews described in the FBI’s response; and (2) the objective and independent methods that will be used to ensure that those profiles are selected from among all of a laboratory’s profiles at NDIS.
Relatedly, we want to address what appears to be a misunderstanding by the FBI regarding the nature of our recommendation. The FBI appears to have read our recommendation as advising the FBI to use QAS auditors to perform profile allowability reviews. In actuality, the OIG’s recommendation, that flows directly from the support in the report, only acknowledges that the FBI has already been using QAS auditors to perform profile allowability reviews. The recommendation communicates that even now, while the FBI is handling the profile allowability reviews in this way, changes need to be made to the methodology. In our report, as well as in our recommendation, we acknowledge the FBI’s stated intention to have the profile allowability reviews conducted by the CODIS Unit auditors. However, at the time of our audit, no such auditors had reported to duty in the Unit. Consequently, our recommendation advises the FBI to implement this change in methodology immediately, rather than at some point in the future when the CODIS Unit auditors are on staff.
Resolved. This recommendation can be closed when we receive a copy of the formal policy or procedure that describes the scope of the CODIS Unit auditor’s reviews, demonstrating that those reviews will include an analysis of compliance with NDIS requirements, as described in the FBI’s response.
Resolved. This recommendation can be closed when we receive documentation that the changes to the NDIS procedures proposed in the FBI’s response have been implemented, to annually confirm that all approved CODIS users have completed their annual user certification forms.
Unresolved. The FBI’s response does not address how it plans to ensure that all guidance given at auditor training courses, including verbal guidance given extemporaneously in discussion sessions as specifically mentioned in our report, is documented in writing for future reference to ensure consistency and to disseminate within the community. Instead, the FBI asserts that the training is already based on a written curriculum. As our report analysis discloses, we agree that a written curriculum exists, but do not believe that it comprehensively documents verbal guidance given supplemental to the audit document in training courses.
Resolved. This recommendation can be closed when we receive documentation of the implementation of web-based tools to aid the CODIS community’s awareness, understanding and consistent interpretation of the QAS.
Unresolved. The FBI disagrees with this recommendation on the basis that the OIG did not provide compelling evidence to support it, in the form of a trend analysis of how many panel members were untimely. Such a trend analysis was not within the scope of the OIG’s work on this audit, but through the course of other work performed, we noted one glaring incident of a panel member being consistently late on audits they reviewed. The FBI argues that since we cite only 1 out of the approximately 88 panel members, our evidence is insufficient. However, the FBI ignores our data analysis of overall panel timeliness that revealed, on average, panel members are taking almost twice as long as permitted to complete their reviews (54 days rather than 30). How many members are implicated by this average was not our concern, but rather the fact that panel member timeliness impacts the overall timeliness of the panel process. Consequently, our audit evidence is sufficient to warrant this recommendation.
Resolved. This recommendation can be closed when we receive documentation that the CODIS Unit has implemented a procedure to begin comparing the audit information reported annually by the SDIS Administrators to the audits received by the NDIS Audit Review Panel, to ensure all appropriate audits have been submitted to that Panel.