| Recommendation #1. |
Develop and implement a plan to ensure that all CODIS Administrators attend the FBI QAS auditor training. |
FBI Response: The FBI agrees that it would be beneficial for all CODIS Administrators to receive the FBI Quality Assurance Standards (QAS) audit training. The CODIS Unit is planning a special auditor training class(es) on the Quality Assurance Standards (QAS) in the fall of 2006 for State and Local CODIS Administrators that have not had auditor training since issuance of the revised FBI Audit Document in July, 2004. All State and Local CODIS Administrators that have not had the auditor training will be expected to attend this training. It will consist of two days of training on the Audit Document and ½ day on the DNA Data Accepted at NDIS scenarios. Following this special auditor course, if there is a new CODIS Administrator, he or she will be required to attend the auditor training on the QAS before assuming his/her full Administrator duties. This requirement will be incorporated into revisions to the Memorandum of Understanding for Participation in the National DNA Index System (NDIS MOU).
| Recommendation #2. |
Improve information sharing through enhancements to the CODIS website, considering the suggestions made by the community and implementing them wherever practicable. |
FBI Response: The FBI agrees that the CODIS website should be used to transmit information of interest and importance to the CODIS community. As a result of inquiries that come into the CODIS unit, we are aware that the CODIS website may not be routinely consulted by the CODIS users so the CODIS Unit will solicit suggestions for improving the utility of this website from the State CODIS Administrators during their June 2006 meeting in Dallas, Texas. These suggestions will be reviewed by the Scientific Working Group on DNA Analysis Methods (SWGDAM) CODIS Committee and to the extent practicable, implemented and shared with the CODIS community during the Annual CODIS Conference in November 2006.
| Recommendation #3. |
Distill profile allowability guidance, including scenarios that are discussed at national meetings, into a decision-tree or other written user-friendly guidance and disseminate that information to all CODIS users. |
FBI Response: The FBI agrees to disseminate additional allowability guidance to CODIS users. The CODIS Unit has included on the CODIS website, the presentations of the scenarios discussed at the Annual CODIS Conference for the past several years. Those scenario presentations are made available on the website following the Conference. With respect to that portion of the recommendation relating to incorporating the rules for profile eligibility into a decision-tree, the CODIS Unit and the SWGDAM CODIS Committee (in preparation for the 2004 Annual CODIS Conference) have each attempted to distill the eligibility determination into a decision tree but these efforts have not been successful. It cannot be overemphasized that each of these factual situations or scenarios is, in fact, unique, and the change of one detail can potentially change the determination of whether the profile is eligible for uploading to NDIS. Accordingly, at this time, we do not believe that the eligibility question can be accurately reduced into a user friendly decision tree.
The CODIS Unit will include all the scenarios discussed at the Annual CODIS Conference in the CODIS Administrators Handbook and on the CODIS web site (with a direct link to the scenarios). Additionally, the CODIS Unit will include on the CODIS web site, to the extent appropriate, scenarios submitted by members of the CODIS community and the response of the NDIS Custodian.
| Recommendation #4. |
Formally request that the Scientific Working Group on DNA Analysis Methods consider, as part of its maintenance of the QAS, the operation material weaknesses identified by the CODIS Administrators, including: (1) the inherent limitations of one-person DNA laboratories; (2) uninvolved off-site technical leaders, and (3) laboratories that upload profiles that have not been fully reviewed. |
FBI Response: The FBI agrees that the issues identified by the CODIS Administrators that impact the quality operations of a forensic DNA laboratory should be shared with SWGDAM - the body charged with the responsibility of recommending revisions to the FBI Director for the Quality Assurance Standards (QAS). The weaknesses identified by the CODIS Administrators in the survey distributed by the OIG will be forwarded (once the OIG CODIS Audit Report has been finalized) to the SWGDAM Chairman for their consideration during SWGDAM’s review of the FBI Director’s QAS. Please see enclosed draft correspondence to the SWGDAM Chairman; Enclosure #1-A.
| Recommendation #5. |
Ensure that guidance on submission of information to the NDIS Audit Review Panel is sent to those members of CODIS labs that are responsible for this activity. |
FBI Response: The FBI agrees that it is important that the relevant personnel in the CODIS laboratories have sufficient information to enable them to submit appropriate and complete audit documentation. The Chief of the CODIS Unit has already requested that CODIS Administrators provide him with the contact information for the person in their laboratory responsible for the QAS audits. The CODIS Unit will be mailing a copy of the NDIS Procedure on “Review of External Audits” as well as a list of the specific information considered audit documentation to the designated contact persons for their information and review. This information will also be included in the Annual CODIS Conference materials. Additionally, the CODIS Unit, in conjunction with the Chair of the NDIS Audit Review Panel, will present this information verbally at the Annual CODIS Conference in November, 2006 (and annually thereafter) and will request permission from the SWGDAM Chairman to present this information at the semiannual SWGDAM meeting in July, 2006 and the public SWGDAM meeting held in conjunction with the Annual Promega Symposium.
| Recommendation #6. |
Develop and utilize a mechanism for tracking information requests that are received by the CODIS Unit to ensure a timely response. |
FBI Response: The FBI agrees that it is important to track requests for information to ensure that they receive an appropriate response. Tracking systems are already in place within the CODIS Unit for the external audit review process as well as the OIG audits of NDIS participating laboratories. The CODIS Unit will post a written request form on the CODIS web site to facilitate inquiries by CODIS users. The written requests submitted to the CODIS Unit that require a response will be logged in and tracked in a Request Log; please see a draft copy of the log - Enclosure #1-B. For those requests requiring a response and that do not contain a due date, a due date for two weeks from the date of receipt will automatically be assigned. This Request Log will be printed out on a weekly basis and provided to the CODIS Unit Chief for review.
| Recommendation #7. |
Develop communication policies that will allow the CODIS Unit to provide written guidance to members of the DNA community to the fullest extent possible. |
FBI Response: As appropriate, the FBI will provide written guidance, through CODIS Technical Bulletins, the CODIS website, or both, on issues of interest and importance to the CODIS community. Additionally, at the time of issuance, all CODIS Technical Bulletins are faxed to each NDIS Participating Laboratory.
| Recommendation #8. |
Develop a staffing plan that identifies current hindrances to filling vacant positions in the CODIS Unit, potential solutions to those hindrances, and a time line of requirements for action to fill those positions. |
FBI Response: The FBI is committed to filling those vacancies that currently exist in the CODIS Unit and will be exploring other avenues for advertising those positions. The NDIS Custodian (Program Manager) and CODIS Auditor positions require that the persons have some familiarity with the National DNA Index System and the FBI Director’s Quality Assurance Standards. Accordingly, the CODIS Unit Chief has mentioned the available positions at meetings of the CODIS State Administrators, CODIS user community and SWGDAM members in an effort to ‘get the word’ on these positions. Additionally, the CODIS Unit Chief has encouraged qualified persons to apply. To date, an insufficient number of qualified persons have applied for these remaining positions so additional advertising forums will be explored with the FBI’s Personnel Unit. For example, advertisements for the available positions could be placed at forensic-related web sites (American Academy of Forensic Sciences, American Society of Crime Laboratory Directors, etc...). Additionally, FBI hiring is handled by the Administrative Services Division and therefore the process and timeline are outside of the control of the CODIS Unit and the Laboratory Division.
| Recommendation #9. |
Develop written descriptions of routine activities and responsibilities for current staff in the CODIS Unit, particularly those with multiple roles, and incorporate this information in a procedure manual for each position. |
FBI Response: The FBI agrees that more detailed information on the routine activities and responsibilities of the current CODIS Unit staff would be helpful in the training process for new staff to the Unit. To ensure that the current staff is not overburdened, the CODIS Unit will consult with the Personnel Unit to determine if this additional task may be added to their performance review objectives for the following year. This should facilitate the collection of this information while ensuring that this additional task is appropriately incorporated into the staff’s responsibilities.
| Recommendation #10. |
Incorporate the three activities we identified that are performed on behalf of the CODIS Unit by other FBI personnel - auditing of NDIS data, providing of training on QAS compliance, and overseeing the activities of the Review Panel - into the CODIS Unit’s objectives and measurements to fully reflect the CODIS Unit’s efforts to address its mission. |
FBI Response: The FBI is supportive of including additional measurements to demonstrate how the CODIS Unit fulfills its mission and statutory responsibilities. Because the CODIS Unit has not previously been tracking the three areas noted above - auditing of NDIS data, providing of training on QAS compliance, and overseeing the activities of the NDIS Audit Review Panel - the CODIS Unit plans to begin to track these additional areas in Federal Fiscal Year 2007.
| Recommendation #11. |
Ensure the development contract process is completed as planned and that the development contract awarded allows for continued responsiveness to legislated changes to CODIS operations. |
FBI Response: In light of the OIG’s statements that the “FBI has taken measures to provide for the operations, maintenance, and security of the CODIS system for the near future...” and that “the independent assessment determined the Justice for All Act could be implemented and operate over the next 3 to 5 years without exceeding capacity of the current CODIS architecture”, it appears that this Recommendation may be unnecessary. The CODIS Unit, with the assistance of the NDIS Procedures Board, has addressed changes in Federal law, first through the Justice For All Act of 2004 and this year with the DNA Fingerprint Act of 2005 and these changes to procedures and the operation of the National DNA Index System have been implemented as soon as practicable (please refer to OIG report at pages 4 and 5). The CODIS Unit will continue to follow its schedule for the development contract. In the event of future legislative changes to the Federal law affecting the operation of the National DNA Index System, such changes will continue to be addressed and implemented as soon as practicable.
| Recommendation #12. |
Ensure that the internal controls over the compliance of NDIS data are strengthened beyond the current reliance on self-certification annual reminder forms. |
FBI Response: The FBI does not agree that the self-certification forms and other mechanisms currently in place are insufficient internal controls for the ensuring the appropriateness of DNA data uploaded to NDIS. The annual reminder forms on DNA Data Accepted at NDIS must be reviewed and signed by each CODIS user. CODIS users in State and Local laboratories submit these forms to their CODIS Administrator who is required to maintain these on file for inspection, if requested by the FBI.
Additionally, the CODIS Unit includes a presentation on the DNA Data Acceptable at NDIS at each Annual CODIS Conference. Beginning in February, 2006, the NDIS Custodian now provides 2 to 3 hours of instruction and discussion on the DNA Data Acceptable at NDIS during each CODIS training class.
The FBI disagrees with the OIG’s generalization that the annual certification forms have not been successful in ensuring compliance with profile allowability restrictions based on its review of OIG audits conducted during 2004 and 2005. We would suggest that the 2004 and 2005 audit data be contrasted with the OIG recommendations from their 2001 audit of the CODIS Program. For example, the 2001 CODIS audit found 40 instances of inappropriate DNA profiles uploaded to NDIS by 5 out of 8 labs. While the OIG reports 13 incidences of inappropriate profiles uploaded to NDIS, a review of the data found in Figure 13 indicates that only 8 of those incidences related to specimen eligibility issues while the remaining 5 findings relate to accuracy and review issues. A comparison of these numbers from 2001 (before the annual reminder forms were implemented) and the 2004/2005 audits does demonstrate fewer instances of findings relating to specimen eligibility at NDIS.
| Recommendation #13. |
Implement a formal mechanism for tracking findings in audits reviewed by an NDIS Audit Review Panel so that common findings and inconsistencies in interpretation can be identified. |
FBI Response: The FBI agrees that information concerning standards frequently cited in audits and differences in interpretation provide valuable information that can be shared with the CODIS community and auditors to ensure consistent interpretation and application of the FBI Director’s Quality Assurance Standards (QAS). The CODIS Unit, and more recently the current and previous Chairs of the NDIS Audit Review Panels, have been informally tracking this information since 2003 when presentations were made at the public SWGDAM meeting held at Promega (September 2003) and the Annual CODIS Conference (November 2003) which included a review of the external audit review process, observations of common pitfalls in submitting the audits and Standards that generated the most findings.
Beginning in 2006, the FBI has been tracking general information relating to those Standards that generate the most findings. The FBI will now track findings that are subsequently overturned. This information will be used in Auditor Training Classes and will be shared with the CODIS community. The FBI will not be tracking information that would identify a specific laboratory in order to maintain the confidentiality of the audit review process.
| Recommendation #14. |
Implement a formal mechanism for tracking auditor performance so that QAS auditors who use incorrect interpretations of the QAS can adjust their performance and also so that the FBI can detect whether individual QAS auditors require additional guidance. |
FBI Response: The FBI has informally been tracking issues relating to inconsistent interpretation of the QAS for the past several years and has informally communicated with the auditors’ employing organization concerning such interpretations. Since the FBI is not the employing organization for the auditors, it is left up to these organizations to take whatever corrective measures deemed appropriate by the organizations. As part of the tracking mechanism that will be implemented for QAS standards, the FBI will also track issues of inconsistent interpretation by an auditor. The FBI will continue to advise the auditor’s employing organization, as necessary. The FBI will also establish relationships with the regional auditing groups so as to keep them informed of any inconsistency in interpretations of the QAS.
| Recommendation #15. |
Use these mechanisms to provide specific training to the DNA community on common findings and inconsistencies observed, to aid the DNA community’s compliance, and to further improve consistency between organizations and QAS auditors. |
FBI Response: The FBI will continue to share information with the CODIS community concerning the proper interpretation of the FBI Director’s QAS. Additionally, the CODIS Unit will include presentations on such topics during the Annual CODIS Conference and will consult with the DNA Analysis Unit I concerning a more formal integration of this information into the FBI sponsored QAS auditor training.
| Recommendation #16. |
Broaden the current methodology used by FBI QAS auditors for NDIS profile verification to permit the selection of profiles from each laboratory’s total profiles in NDIS. This revised methodology should continue once CODIS Unit auditors are on staff. |
FBI Response: The external QAS audit currently conducted by qualified auditors from the FBI’s DNA Analysis Unit I is governed by the QAS Audit document. This Audit document is used by the CODIS community to satisfy requirements for participation in the National DNA Index System. The purpose of the external QAS audit is to ensure compliance with the FBI Director’s QAS - a requirement in Federal law for participation in NDIS.
The issue of a profile’s eligibility for the National Index is not a quality issue but, rather, an issue of the integrity of the DNA records uploaded to and maintained at NDIS. The eligibility of DNA profiles, while also governed by Federal law, is an issue addressed by NDIS Procedures. As such, the eligibility of DNA profiles is ultimately determined by the NDIS Custodian. The FBI believes it appropriate to have the review the issue of profile eligibility separate from the external quality audit of an NDIS participating laboratory. Thus, the FBI proposes that the review of profile eligibility remain with the CODIS Unit auditors. The CODIS Unit auditors will conduct external QAS audits of NDIS Participating Laboratories that will also include a review of 50-150 DNA profiles per laboratory to ascertain whether DNA profiles uploaded to NDIS were eligible for NDIS. For forensic caseworking laboratories, a total of 50 DNA profiles may be reviewed and for offender databasing laboratories, a total of 100 DNA profiles may be reviewed.
| Recommendation #17. |
Expand the scope of CODIS Unit auditor duties to include verification of compliance with NDIS requirements. |
FBI Response: Please refer to the FBI’s response to Recommendation #16 above. Additionally, the CODIS Unit auditors, during the external QAS audit process, will perform a review of the following NDIS Procedure requirements:
- Documentation to ensure that every CODIS user has complied with the Annual Reminder of DNA Data Acceptable at NDIS;
- DNA profile eligibility (including review of DNA profiles at NDIS for required loci);
- Confirmation of Interstate Candidate Matches; and
- Outsourced DNA data subject to technical review.
| Recommendation #18. |
Alter the annual user certification documentation required from laboratories to include information sufficient to confirm that all CODIS users are completing the forms as required. |
FBI Response: The FBI believes that the use of the annual certification forms has increased the CODIS user’s awareness of the DNA profiles eligible for NDIS. To ensure that all CODIS users are completing the forms as required, the FBI now requires that the annual certification form is submitted by all new CODIS users with the other documentation required for Adding a CODIS User. Additionally, the CODIS Unit will be proposing changes to the NDIS Procedures to require that each CODIS State Administrator provide the NDIS Custodian, on an annual basis, with a listing of those CODIS Users in their State who have completed and signed their Annual Reminder forms on DNA Data Accepted at NDIS. The CODIS Unit will then check the CODIS users identified on this annual listing to ensure that all approved CODIS users have completed their annual reminder forms. Please refer to response to Recommendation #12.
| Recommendation #19. |
Ensure that QAS auditor training is based upon a comprehensive written curriculum, including guidance that reaches beyond the contents of the audit document. |
FBI Response: The FBI’s DNA Analysis Unit I has been providing auditor training for five years since September 2000 when the QAS Audit document was first introduced. To date, over 1,000 individuals have received the FBI sponsored auditor training. The training is given by the Chief of the DNA Analysis Unit I and follows a written curriculum. Each student is provided with a notebook containing the presentation (to assist in documenting the course, interpreting Standards and note-taking) as well as the FBI Audit Document. At the conclusion of the auditor training, an examination is administered to the participants and a grade of pass/fail is given.
To ensure the consistent interpretation of the Standards, appropriate guidance has been included in the comment and discussion sections of the QAS Audit Document and that constitutes the written guidance, in addition to the training materials, provided to the participants. Auditors are encouraged to contact the DNA Analysis Unit I or the CODIS Unit if they have a question concerning the interpretation of a Standard.
| Recommendation #20. |
Develop web-based training tools for QAS compliance and auditing information, to aid the CODIS community’s awareness, understanding, and consistent interpretation of the QAS. |
FBI Response: The FBI is supportive of any mechanism that will facilitate the CODIS community’s awareness, understanding and consistent interpretation of the QAS. The FBI believes that the auditor training is one such mechanism and efforts to expand that training to the internet could further encourage consistent interpretation of the QAS. The FBI will explore what additional resources would be needed for the development of computer-based training tools for QAS compliance and auditing information. Meanwhile, the integration of the CODIS Unit auditors into the external QAS audit process and audit reviews are expected to further consistency in interpreting the QAS.
| Recommendation #21. |
Monitor NDIS Audit Review Panel member performance to ensure that members are timely, and implement procedures for taking action in cases where members are consistently untimely. |
FBI Response: The FBI acknowledges the participation of State and local forensic DNA scientists in the NDIS audit review process, and without whose participation, this review process could not have been implemented. The FBI does not agree that there is any need to formally monitor the performance of NDIS Audit Review Panel Members to ensure that members are timely. The overwhelming majority of NDIS Audit Review Panel Members perform their reviews in a timely and satisfactory manner. While the OIG audit has found one Panel Member who has been consistently late in his/her responses, there are currently over 88 NDIS Audit Review Panel Members. Accordingly, in light of the efforts of the NDIS Audit Review Panel members who volunteer their time to assist in this endeavor and the lack of any trend indicating that Panel members are consistently late in their responses, the FBI does not see any need, at this time, to monitor Panel Members’ performance for timeliness.
| Recommendation #22. |
Track information currently collected from NDIS participants to ensure all external QAS audits reported to the CODIS Unit are also submitted to the NDIS Audit Review Panel. |
FBI Response: The FBI is supportive of efforts to further improve the audit review process. The CODIS Unit does currently track the audits from receipt to completion and closure of the audit. The CODIS Unit will also compare the audit information reported by the State CODIS Administrators in accordance with NDIS Procedures with the audit information tracked by the Unit in an effort to ensure that all external audits conducted are subject to the NDIS Audit Review process.
|