Federal Bureau of Investigation's Foreign Language Translation Program Follow-Up

Audit Report 05-33
July 2005
Office of the Inspector General

Our follow-up review found that the FBI had strengthened its policies and guidelines regarding quality control of language translations and in April 2005 implemented a nation-wide tracking system to ensure field offices were performing the required reviews and monitoring results of the reviews. During our follow–up work we asked the FBI to provide documentation about the current status of the quality control program. The FBI provided us with the revised Translation Quality Control Policy and Guidelines that became effective on December 30, 2004.39 On July 12, 2005, the FBI provided the OIG with the spreadsheets it used to track the first quarterly report from the field offices and the revised spreadsheet it plans to use to track the second quarterly reports. Quality Control Policy Our July 2004 audit report stated that the FBI was not meeting its quality control review requirements for newly hired linguists or linguists with more than one year of experience. That policy required all translations from English into the foreign language to be reviewed; all translations from a foreign language into English that would be disseminated to the public as public source material outside the FBI to be reviewed; and all summaries, transcripts, and translations for use in court to be reviewed. We recommended that the procedures be strengthened to ensure that these quality control reviews were performed. In response to the OIG recommendation, the LSS modified the Translation Quality Control Policy and Guidelines. These revised guidelines became effective on December 30, 2004. The new policy requires that reviews must be conducted only by certified reviewers.40 If no certified reviewer is available, the field offices may use designated GS-12 or GS-13 linguists. Formerly, linguists with more than one year of experience with the FBI were required to undergo an annual review which included one or more of the following depending on the regularly assigned tasks of each linguist: verbatim and summaries of documents. verbatim and summaries of audio which could have included Title III (criminal) or FISA audio. an assessment of the linguist’s interpreting ability, if applicable. Under the FBI’s new policy, these requirements are still in effect. However, the new policy also requires two quality control reviews that include randomly selected materials marked as “Not Pertinent” by the linguist being reviewed.41 Under the previous policy, linguists with less than one year of experience with the FBI were to have 100 percent of their work reviewed for the first 3 months, followed by random quarterly reviews during the next 9 months and annual reviews thereafter. Under the new policy, after completion of administrative, operational, and equipment/technology training, new linguists will have all of their translations reviewed for only the first 40 hours of work. Random reviews will be conducted during the next 80 hours and must include at least two reviews of materials marked as “Not Pertinent” by the linguist. Thereafter, annual reviews are to be conducted. Other changes in the new policy include: requiring that reviews be conducted anonymously whenever possible; reviewing each language a linguist translates or interprets for at least 20 percent of their time; or if the linguist occasionally translates material that may impact an investigation or affect a substantive matter, then the linguist will be reviewed in all languages they translate; requiring supervisors or Field Office Representatives to give prompt feedback to the linguists, resolving any deficiencies, and tracking and maintaining a record of all reviews; and identifying specific actions for supervisors to take when there is a disagreement with results of the quality control reviews. We believe the policy changes address the concerns raised in our July 2004 audit. However, we could not perform testing to determine if the new guidelines for quality control reviews were being followed because the first quarterly report under the modified quality control policy was not due until April 15, 2005, after we completed our fieldwork for this follow-up review. Tracking Results of Quality Control Reviews In our July 2004 audit, we found that the FBI had no nation-wide system to track the results of quality control reviews. We recommended that the FBI implement a system to monitor compliance with the quality control procedures, both at the field office and national levels. During our follow-up work performed in March 2005, we inquired about the FBI’s progress toward developing a nation-wide system to track the results of its quality control reviews. Since the completion of our previous audit work, annual reviews for full-time linguists in the field offices should have been forwarded to the LSS by June 30, 2004; reviews for contract linguists by September 30, 2004, before their annual contracts were renewed on October 1; and reviews for full-time linguists at the Language Services Translation Center by November 30, 2004. We checked on the status of these reviews and found that, as of March 2005, the LSS had not monitored whether the field offices had performed the required quality control reviews. Even when the field offices had provided appropriate forms documenting completed reviews to the LSS, these forms had not been entered into a tracking system. However, on July 12, 2005, the FBI provided the OIG with documentation showing that it had initiated a nation-wide tracking system and had used the new system to track the first quarterly report received in April 2005. In its response to our July 2004 audit, the FBI also stated that two applicants had been selected who would serve as Foreign Language Program Specialists and would track approximately 650 linguists each. The FBI predicted that by the end of calendar year 2004, the Translation and Deployment Unit would be able to monitor and track the field’s compliance regarding quality control reviews. However, only one person was assisting the Program Manager of the FBI Linguist Program at the time of our follow up work in March 2005 and she had only been working in that position since January 24, 2005. She stated that her duties included making sure the field offices are in compliance with filing quarterly reports; keeping track of quality control reviews for new-hires and annual reviews for linguists with over one year of experience; and coordinating the certified reviewer training sessions. But she had not been informed that it was her responsibility to maintain a tracking system for results of the quality control reviews. On June 7, 2005, the OIG received notification that the FBI had added an Acting Program Manager to the Quality Control Program and had plans to add three more staff members within the month. However, as of July 13, 2005, no additional staff had been added. The Unit Chief of the new Operations Management Unit responsible for tracking compliance with the quality control policy told us on April 22, 2005, that the LSS had received the first submission of the QC Quarterly Compliance Report Forms required by the new quality control policy and guidelines and was following up with field offices that did not comply with the new requirements. The Unit Chief said that the FBI expected difficulties for the first reporting period, but he planned to follow up with the field offices. The Acting Program Manager of the Quality Control Program told us that the field offices had been sent more simplified forms and clearer instructions to facilitate the second quarterly quality control report due on July 15, 2005. The OIG received a copy of the new format on July 12, 2005, and agrees that it is much simpler to complete and should alleviate the problems encountered with the first quarterly report. After receiving the second quarterly report, the FBI expects to make further refinements to the tracking system. During the exit conference for this follow-up review on May 25, 2005, the Unit Chief provided the OIG with a summary of the results of the first quarterly quality control review. However, the supporting documentation provided to the OIG in July 2005 did not verify the May 2005 statistics and therefore are not included in this report. In responding to the recommendations in our July 2004 audit, the FBI also stated that in order to adequately monitor its quality control program, it would need to hire additional language analysts to address the compliance requirements for new linguists and the annual reviews of full-time and contract linguists. The Section Chief of the LSS told us that the FBI expects 10 of the 274 language analysts its plans to hire in FY 2006 to be dedicated to quality control reviews. The Section Chief also stated that the additional 10 language analysts should provide enough resources for full implementation of the enhanced quality control program. According to the Unit Chief of the new Operations Management Unit, regional program managers will guide and monitor field supervisors in adhering to quality control standards and procedures. As noted previously, the Operations Management Unit will develop, implement, and ensure compliance with administrative and operational policies, procedures, and guidelines by Foreign Language Program personnel throughout the FBI. Footnotes The statistics in each column are not exclusive to the timeframe indicated, since some actions may have been in process prior to that timeframe. On May 25, 2005, the FBI presented the OIG with a summary of the results of the first quarterly quality control review completed according to the modified guidelines. Because this date was after the completion of our onsite work, we were unable to verify the statistics the FBI provided. Certified reviewers have received specialized training and have passed an exam to be certified. In addition to using the Quality Control Work Review Error Notation Key to annotate anomalies as before, they also are required to adhere to the FBI’s standards for translation outlined in the Language Services Section’s Manual of Standards of Translation. As of the middle of April 2005, the Language Services Section had completed four training sessions and trained approximately 100 certified reviewers. Material marked as “Not Pertinent” does not require the production of a tech cut. A tech cut is a summary of the pertinent details of a telephone call rather than a verbatim translation.