Following the September 11, 2001, terrorist attacks, the Attorney General and the Director of the Federal Bureau of Investigation (FBI) made clear that prevention of terrorism is the top priority of the Department of Justice (DOJ) and the FBI. Effective use of information technology (IT) is crucial to the FBI's ability to meet this priority as well as its other critical responsibilities. For FY 2003, the FBI allocated nearly $606 million to information technology projects.

As computer technology has advanced, federal agencies have become increasingly dependent on information systems to carry out operations and process, maintain, and report essential information. The FBI's computerized information systems affect many mission-critical activities, such as financial management, security of sensitive and classified data, and investigative work.

Recognizing the importance and vulnerability of data processed, maintained and reported by the FBI, the Office of the Inspector General (OIG), the General Accounting Office (GAO), and other entities have conducted audits, investigations, and reviews of the FBI's management of IT. For years, reviews have found major weaknesses associated with the FBI's IT. The FBI has made upgrading its information technology one of its top ten priorities.

To assess the FBI's progress in improving its IT, the OIG conducted this audit of the FBI's implementation of prior OIG and GAO recommendations. To perform our audit, we conducted 27 interviews with officials from the FBI, OIG, and GAO. The FBI officials interviewed were from the Inspection Division, Information Resources Division, and National Infrastructure Protection Center. Additionally, we reviewed over 100 documents including prior GAO and OIG reports, Congressional testimony, and documentation on the FBI's process for tracking the resolution of recommendations.

1. Policies and Procedures for Following-Up on Report Recommendations

The Office of Management and Budget (OMB) and DOJ have issued policies and procedures for following-up on recommendations of audit reports. According to OMB Circular A-50, audit follow-up is an integral part of good management, and is a shared responsibility of agency management and auditors. OMB Circular A-50 requires agencies to establish systems to assure the prompt and proper resolution and implementation of audit recommendations. These systems are to provide for a complete record of action taken on both monetary and non-monetary findings and recommendations.

Department of Justice Order 2900.6A, Audit Follow-Up and Resolution, established Departmental policies and criteria for the follow-up and resolution of audit findings and recommendations to ensure that all OIG audit reports are adequately and timely resolved, and that all resolution actions are consistent with the governing laws and regulations. The order states that the head of the DOJ component is responsible for overall audit resolution and follow-up activities within his or her organizational unit and is accountable to the Deputy Attorney General. Further, the DOJ component should establish an audit follow-up and resolution system that ensures written comments on audit findings and recommendations are made within four months after the issuance of the report.

The order also states that DOJ components should assign a high priority to the immediate implementation of the order so that the DOJ will be in full compliance with the legislative and regulatory requirements pertaining to the timely resolution of audits. Although subjective, the timeliness of corrective actions is assessed on a recommendation-by-recommendation basis due to the inherent difficulties associated with implementing certain recommendations.

When issuing other OIG reports that contain recommendations, such as special investigations or reviews, the OIG elicits responses from components regarding planned corrective actions. When received by the OIG, the responses are reviewed to determine whether the planned corrective actions meet the intent of the recommendations. Periodically, the OIG makes subsequent inquiries with components to monitor the implementation of these actions. As with audit reports, component managers are ultimately responsible for ensuring that recommendations are implemented in a timely manner.

2. The FBI's Implementation of IT Recommendations

Since 1990, OIG reports have identified numerous deficiencies with the FBI's IT program, including outdated infrastructures, fragmented management, ineffective systems, and inadequate training. While the FBI has implemented many of the recommendations contained in these reports (93 out of 148), significant further actions are necessary to ensure that the FBI's IT program effectively supports its mission. For example, recent audits and reviews conducted by the OIG have found repeated deficiencies with the FBI's IT control environment and compliance with information security requirements.

These repeated deficiencies indicate that, in the past, FBI management had not paid sufficient attention to improving its IT program. Until recently, the FBI lacked an effective system of management controls to ensure that recommendations issued by the OIG are implemented in a timely and consistent manner. However, current FBI leadership has stated that they are committed to enhancing controls to ensure recommendations are implemented in a consistent and timely manner. The FBI has recently established a system to facilitate the tracking and implementation of recommendations. Additionally, the FBI expects significant improvements from its current IT modernization efforts, which the FBI believes will correct many of the deficiencies identified by the OIG.

A. OIG Reports on the FBI's IT

To assess the FBI's progress in implementing recommendations directed toward improving its information technology, the audit examined the following OIG reports that related to the FBI's use and management of IT:

• the 1990 audit report on the FBI's automated data processing (ADP) controls;

• the 2002 audit report on the FBI's IT investment management (ITIM);

• five detailed reports issued in support of annual financial statement audits for FYs 1996 through 2001¹ on the FBI's control environment over its IT systems;

• three audit reports pursuant to the Government Information Security Reform Act (GISRA) issued for FYs 2001 and 2002; and

• two special investigative reports that contained FBI IT-related recommendations issued in 1999 and 2002.

For the 1990 ADP audit report and the 2002 ITIM report, we examined similarities between the reports' findings to assess the FBI's progress in improving its IT. For the OIG's detailed IT reports, FY 2001 GISRA audit report, and special reports, we obtained the status of FBI IT-related recommendations. The table below summarizes the status of FBI IT-related recommendations contained in these reports.

Summary of the Status of IT Recommendations Issued to the FBI

The following sections provide background information on these reports and an assessment of the FBI's progress toward implementing IT-related recommendations contained in these reports.

(1) Reports on the FBI's ADP Controls and IT Investment Management

In 1990, the OIG issued an audit report entitled, "The FBI's Automatic Data Processing General Controls." This report found 11 major internal control weaknesses, many of which were still applicable 12 years later. Specifically, the report found the following.

1. The FBI's phased implementation of its 10-year Long Range Automation Strategy, scheduled for completion in 1990, was severely behind schedule.

2. The FBI's Information Resources Management program was fragmented and ineffective, and the FBI's Information Resources Management official did not have effective organization-wide authority.

3. The FBI had not developed and implemented a data architecture.²

4. The FBI had not adequately involved top management in FBI Headquarters (FBIHQ) or the field offices in systems development through an Executive Review Committee.

5. The FBI's major mainframe investigative systems were labor intensive, complex, untimely, and non-user friendly and few special agents used these systems.

Many of the weaknesses identified in the 1990 report on ADP controls were mentioned again in the 2002 audit report on the FBI's ITIM. In December 2002, the OIG issued a report entitled, "The FBI's Management of IT Investments." The OIG concluded that the FBI had not effectively managed its IT investments because it had not fully implemented the management processes associated with successful IT investments.

The ITIM report contained 30 recommendations directed toward improving the FBI's management of its IT investments. Because our evaluation of the FBI's progress toward implementing recommendations was close to the final issuance of the ITIM report, we did not assess the FBI's progress in implementing the recommendations. However, our 2002 ITIM report found that many of the weaknesses described in the 1990 report on the FBI's ADP controls still existed.

• The FBI's IT infrastructure was severely outdated.

• The FBI's Information Resources Management program was decentralized. The FBI had completed several restructurings, including one in February 2002 that was intended to give the Information Resources Management program more authority over the divisions that manage IT.

• The FBI still had not completed an enterprise architecture framework, which included the technical and data architecture.

• The FBI did not have formally established IT investment review boards or committees until March 2002.

• The FBI's major investigative systems remained labor intensive, complex, non-user friendly, and many special agents still did not use these systems.

The OIG concluded that the FBI's ability to completely and timely implement the 30 recommendations listed in the ITIM report will, in part, depend on management's commitment to do so. This management commitment must be incorporated into a comprehensive process to ensure that the recommendations are tracked and implemented.

(2) Reports on the FBI's Control Environment over its Financial IT Systems

The OIG conducts annual financial statement audits of the FBI, with the most recent report covering FY 2001. Financial statement audits are intended to play a central role in (1) providing more reliable and useful financial information to decision-makers, and (2) improving the adequacy of internal controls and underlying financial management systems. In support of the FBI's annual financial statement audits, the OIG has issued detailed reports since FY 1996 on the effectiveness of the FBI's general and application controls over IT systems used to process financial transactions.

To conduct these reviews, the OIG used the GAO's Federal Information System Controls Audit Manual (FISCAM). The FISCAM describes the computer-related controls by category that auditors should consider when assessing the integrity, confidentiality, and availability of computerized data.

We found that the FBI made progress in correcting deficiencies identified in the detailed reports supporting the annual financial statement audits from FY 1996 to 2001. Of the 105 recommendations contained in these reports, 83 have been implemented and closed, and 22 are still open. The following table summarizes the status of the FBI's IT control environment recommendations by FISCAM category.

By implementing 83 of the recommendations, the FBI improved its IT internal control environment. The FY 2001 report did not contain any system software control deficiencies.⁴ The FBI also made progress toward correcting deficiencies in entity-wide security program planning, access controls, application software development and change controls, segregation of duties, service continuity, and application controls.

Despite the progress, however, as of April 2003 material weaknesses⁵ remained in the following general control areas:

• entity-wide security program planning and management - increasing the risk that the integrity of sensitive information can be compromised;

• access controls - increasing the risk of erroneous or fraudulent financial transactions; and

• application software development and change controls -increasing the risk of inaccurate and unauthorized software changes.

In addition to these material weaknesses, other vulnerabilities existed in the following internal control areas:

• segregation of duty controls - increasing the risk that erroneous or fraudulent transactions could be processed, improper program changes could be implemented, and computer resources could be damaged or destroyed;

• service continuity controls - increasing the risk that during an extended outage or disaster, information system processing functions and vital business operations will be damaged and unable to function since critical information and computer resources would be unavailable or inaccessible; and

• application controls - increasing the risk of inaccurate valuation or allocation of data, and unauthorized transactions.

We also noted that 30 of the both open and closed recommendations were repeated in subsequent reports on the FBI's financial IT systems' control environment. For example, the OIG's review for FY 1998 reported that an automated tool was used to perform an assessment of the technical controls over the FBI's Finance Division Local Area Networks (LAN). The assessment found weaknesses in three areas of security: account restrictions, system monitoring, and data confidentiality. In FY 1999, another automated tool was used to perform the assessment of the technical controls over the FBI's Finance Division LANs. Although corrective action had been initiated on the prior weaknesses found, the OIG reported that these weaknesses still existed during FY 1999. The FY 2000 review stated that auditing remained disabled on the Finance Division's Windows NT and Novell NetWare environments. In addition, according to the OIG FY 2001 review, although FBI management had stated that corrective actions have been taken with respect to the recommended settings for account restrictions, system monitoring, and data confidentiality, the conditions continued to be identified during the annual financial statement audit process. Because of the uncorrected deficiencies identified in these audits, the FBI is at increased risk to failures in its financial management and computer security functions.

(3) Computer Security Reports in Response to GISRA

Beginning in FY 2001, the OIG was required by GISRA to perform an independent evaluation of the DOJ's information security program and practices using standards developed by the GAO and the National Institute of Standards and Technology (NIST).⁶ In May 2002, pursuant to GISRA, the OIG issued an audit report on the FBI's investigative and administrative mainframe systems.

The NIST, in conjunction with GISRA, issued guidance detailing the specific controls that should be documented by federal agencies in their system security plan. The purpose of the security plan is to provide an overview of the security requirements of the system and describe the controls in place or planned for meeting those requirements. The system security plan also delineates responsibilities and expected behavior of all individuals who access the system.

The NIST separated the security plan controls into three major control areas: (1) management controls, (2) operational controls, and (3) technical controls. Within each of the three control areas, there are a number of subordinate categories of controls. For example, technical controls include password management, logon management, account integrity management, and system auditing management.

We found that the FBI made some progress in correcting deficiencies reported in the OIG FY 2001 GISRA audit. Of the 23 recommendations contained in the report, 6 have been implemented and closed, and 17 are still open. The following table summarizes the status of the FBI's FY 2001 GISRA audit report recommendations by category.

By implementing six of the recommendations, the FBI improved the security of its investigative and administrative mainframe systems at its Headquarters and Clarksburg Data Centers. These improvements included (a) defining and documenting all criticality levels used to classify applications, (b) establishing optimal operating system capacities and implementing procedures to alleviate the near capacity usage, (c) fully implementing and using the System Access Request function to document user logon and verify that user access is commensurate with assigned responsibilities, and (d) ensuring that the communication carrier signals are not connected to unencrypted network devices.

Despite the progress made, as of April 2003 vulnerabilities remained in the following areas:

• security policies, procedures, standards, and guidelines;

• system and network backup and restoration controls;

• password management;

• logon management;

• account integrity management;

• system auditing management; and

• system patches.

The OIG assessed these vulnerabilities as a high-to-moderate risk for the protection of the FBI's administrative and investigative mainframe systems from unauthorized use, loss, or modification. These vulnerabilities occurred because DOJ and FBI security management had not enforced compliance with existing security policies, developed a complete set of policies to effectively secure the administrative and investigative mainframes, or held FBI personnel responsible for timely correction of recurring findings. Further, the report stated that the lack of timely and effective oversight from DOJ and FBI management caused inconsistencies in the implementation of security guidelines and resulted in a weakened security infrastructure.

The FY 2002 GISRA report on the Automated Case Support (ACS) and DRUGX systems, like the FY 2001 GISRA report, noted repeated deficiencies in general control areas. Specifically, vulnerabilities were noted within password management, logon management, account integrity management, system auditing management, and system patches. The report further stated that, if not corrected, these security vulnerabilities threaten the ACS system and its data with the potential for unauthorized use, loss, or modification.

According to the FY 2002 GISRA reports, the FBI did not maintain a system of recording, tracking progress, ensuring attention to, or determining the completion of action in response to any information security vulnerability uncovered during a non-OIG review. As a result, the FY 2002 GISRA reports recommended that the FBI determine the responsible organization for tracking and maintaining all vulnerabilities identified during audits and reviews. In addition, the reports recommended that the FBI develop a mechanism for tracking the vulnerabilities and the status of the associated corrective actions resulting from all IT audits and reviews. Since September 2002, the FBI has been developing new procedures and databases to assist with the audit resolution and follow-up process. FBI officials informed us that the Inspection Division now manages the audit follow-up and resolution process for both OIG and GAO audits. Additionally, for system audits, the FBI has reported that its Information Assurance Section has taken steps to centrally manage the status of vulnerabilities and corrective actions.

(4) Reports on Special Investigations of the FBI

Since 1998, the OIG has issued two special investigation reports containing significant FBI IT-related recommendations:

• the 1999 report on the DOJ's Campaign Finance Task Force investigation (Campaign Finance); and

• the 2002 report on the FBI's investigation into the Oklahoma City Bombing case (McVeigh).

These reports, among other issues, considered the policies and procedures related to the management of information within the FBI, the dissemination of the information to organizations outside the FBI, and the effectiveness of the information technology utilized by the FBI. The reports cited deficiencies in the FBI's management of IT, and provided 20 recommendations directed toward correcting these deficiencies.⁷ The table below summarizes the status of the special investigation recommendations made to the FBI by report.

We found the FBI's current and planned corrective actions, including the implementation of Trilogy, has the potential to address 16 of the 20 recommendations that we examined from the Campaign Finance and McVeigh reports. However, the ultimate success of Trilogy will not be determined until at least June 2004 when the final phases of the project are scheduled for completion.

The following section provides background information on Trilogy, since its successful completion is critical to not only addressing OIG recommendations, but also to the future of the FBI's IT program.

(a) Background on Trilogy

Trilogy is an IT modernization project designed to upgrade the FBI's: (1) hardware and software or Information Presentation Component (IPC), (2) communication networks or Transportation Network Component (TNC), and (3) User Application Component (UAC). The IPC and TNC upgrades will provide the physical infrastructure needed to run the applications from the UAC. The UAC is intended to replace five of the FBI's primary investigative applications in order to reduce agents' reliance on paperwork and improve efficiency. Through the creation of the Virtual Case File (VCF), a web-based "point-and-click" case management system, agents are expected to have multi-media capability that will allow them to scan documents, photos, and other electronic media into the case file.

In November 2000, Congress appropriated $100.7 million for the first year of the $379.8 million Trilogy project, which was to be funded over a 3-year period. In January 2002, Congress supplemented the FY 2002 Trilogy budget with $78 million to expedite the deployment of all three components. This supplemental appropriation increased the total funding of Trilogy to approximately $458 million. Even with these additional funds, the FBI missed its July 2002 milestone date for completing the "Fast Track" portion of the IPC and TNC phases.

In April 2003, the FBI Director reported to the Senate Appropriations Committee that over 21,000 new desktop computers and nearly 5,000 printers and scanners have been deployed (IPC phase). Additionally, the FBI reported that it completed the Trilogy Wide Area Network (TNC phase) on March 28, 2003. The new network, which has been deployed to 622 sites, provides increased bandwidth and three layers of security. According to the FBI, the network is highly expandable, so additional capacity or even additional sites could be added as needed. This network replaces the FBI's dated local area and wide area networks, enabling FBI personnel to transmit data at much greater speeds. Further, the FBI expects to use the network to transport the Investigative Data Warehouse, which will link 31 FBI databases for single-portal searches and data mining. Also, the network lays the foundation for improved information sharing with partner agencies, and other new applications, such as the VCF.

The VCF will serve as the backbone of the FBI's information systems, replacing the FBI's paper files with electronic case files that include multi-media capabilities. The FBI expects to deploy the VCF in three releases. The initial VCF release will consolidate data from the current ACS and IntelPlus systems and has a targeted completion date of December 2003. This release is intended to allow different types of users, such as agents, analysts, and supervisors, to access information from their desktop computers that is specific to their individual needs. This VCF release is also intended to enhance the FBI's capability to set and track case leads, index case information, and move document drafts more quickly through the approval process with digital signatures.

The second and third releases are intended to upgrade three other investigative applications into the VCF: the Integrated Intelligence Information Application (IIIA), Telephone Application, and Criminal Law Enforcement Application. These releases have a targeted completion date of June 2004 and are intended to provide agents with Audio/Video Streaming capability and content management capability. According to FBI documentation, content management should help agents access information from the FBI's data warehouse, regardless of where in the system the information was entered, providing a single query for all of the FBI's systems that are connected to the Investigative Data Warehouse.

The OIG ITIM report, issued in December 2002, stated that the VCF, which is recognized by FBI officials as the most important aspect of the Trilogy project in terms of improving agent performance, was at high risk of not being completed within the funding levels appropriated by Congress. FBI officials confirmed the OIG's assessment in January 2003 when they told us that an additional $138 million⁸ was needed to complete Trilogy, bringing the total project cost to $596 million. Despite the cost overruns, FBI officials stated that they still expect to deliver the first release of VCF in December 2003, and that funding for the second and third releases of the VCF has been secured.

The following sections provide further details on the IT and document management related deficiencies noted in Campaign Finance and McVeigh reports, as well as an assessment of the how the VCF will address these deficiencies.

(b) Campaign Finance Report

In July 1999, the OIG issued a report entitled, "Handling of FBI Intelligence Information Related to the Justice Department's Campaign Finance Investigation" (Campaign Finance). In response to a request by the Attorney General, the OIG reviewed the FBI's practices for disseminating intelligence information associated with the Campaign Finance Task Force (Task Force) investigation.

With respect to the use and maintenance of the FBI's computer database systems, deficiencies were noted in: the Task Force's familiarity with the FBI's databases, the FBI's practices and policies that limited the usefulness of the databases, the training of FBI personnel on the ACS system, and the entry of foreign names into the FBI's databases. These findings highlighted the need for FBI and Task Force personnel to be familiar with information search techniques within the FBI's databases, how information should be entered into the databases in order to take advantage of search capabilities, and potential errors in data entry to ensure that all possible searches within the databases are conducted. Of the Campaign Finance report's 18 recommendations, 5 pertained to the IT-related deficiencies. These five recommendations related to revising the FBI's ACS and IIIA systems to require the uploading of documents and mandatory indexing of names, and training the users of these systems.

Regarding the uploading of documents, the FBI issued Electronic Communications (EC) in July 2000 and June 2002 that required all e-mails and ECs to be uploaded into the ACS system, unless otherwise prohibited by their sensitive nature. Additionally, FBI officials stated that with the VCF, documents will have to be uploaded since the VCF will contain all official records and case files.

Regarding the mandatory indexing of names, FBI officials stated that the VCF will facilitate indexing on various web-based documents by providing data fields in searchable databases. The index of data fields, except for narrative fields, will be automatically created once the document is approved and entered into the VCF. Agents and analysts can then search the index of data fields by using search screens or viewing the serialized document.

Regarding training, FBI officials said that they increased the ACS system training for veteran agents and have plans in place to train FBI employees and task force members on the VCF. Additionally, the FBI continues to offer training on the "Romanization" of foreign names, including those in Arabic and Chinese.

Despite the FBI's progress in taking corrective actions, a more comprehensive enterprise-wide solution to the underlying deficiencies will not occur until the VCF is implemented. As a result, some of these deficiencies have gone uncorrected for over three years.

(c) McVeigh Report

In March 2002, the OIG issued a report entitled, "An Investigation of the Belated Production of Documents in the Oklahoma City Bombing Case" (McVeigh). This report analyzed the causes for the belated production of many documents in the Oklahoma City bombing case.

The McVeigh report concluded that the belated production of case-related documents resulted in part from the following long-standing problems at the FBI: (1) antiquated and inefficient computer systems, (2) inattention to information management, and (3) inadequate quality control systems. The report further stated that the FBI's troubled information management systems were likely to have a continuing negative effect on the FBI's ability to properly investigate crimes.

The report stated that the FBI had not given sufficient attention to correcting deficiencies in information management and the ACS system. The findings of the report relating to information technology showed that the ACS system is extraordinarily difficult to use, has significant deficiencies, and is not suitable for the FBI in the 21st century. The report noted that inefficiencies and complexities with the ACS system, combined with the lack of a true information management system, were significant factors in the FBI's failure to provide hundreds of investigative documents to the defendants in the Oklahoma City bombing case. To overcome these problems, the report made recommendations on how future information systems should be developed.

The McVeigh report provided 21 recommendations to the FBI, 15 of which directly related to IT. Eleven of the fifteen recommendations pertain to correcting deficiencies associated with the FBI's investigative systems, including the tracking of leads and other records management policies. FBI officials have stated that the VCF, when implemented, will address these 11 recommendations.

In May 2003, FBI officials stated that agents will be required to use the VCF, since all official case records and files (up to the Secret level) will be within the application. According to the FBI, unlike the currently used ACS system, agents will not be able to circumvent the use of the VCF. However, the FBI still has not finalized its policies for how agents will utilize VCF from remote locations. Additionally, the VCF has only been approved for use up to the Secret classification level, so Top Secret and Sensitive Compartmented Information (SCI) records will still be maintained in a SCI facility.

FBI officials stated that the VCF will streamline the workflow process by including electronic signatures and reducing the number of required forms. Under the FBI's current investigative process, a case file is started by using one of the FBI's many different standardized forms. The use of these forms will be replaced by the "intake" function of the VCF, which will simplify the initiation of a case file by eliminating the use of these forms. As a result, the VCF will essentially replace all paper copies of investigative events.

According to FBI officials, the VCF will operate in a "point-and-click" web environment that will simplify the FBI's workflow process for document storage and retrieval. Further, FBI officials told us that the VCF's automated document creation, receipt, and management system will partially eliminate the need for traditional tracking systems. The VCF will include capabilities to scan into the case file any documents received from sources external to the FBI, as well as to capture summary descriptions of any documents and items, such as physical evidence, that cannot be stored electronically. The FBI's intent is to eventually track external items through a bar code identification system that would be placed upon a physical label on the external document and then linked to an electronic record. Additionally, FBI officials said that the Records Management Division (RMD) is establishing systems and processes to effectively track documents and records contained in FBI systems.

While the VCF will consolidate five of the FBI's investigative applications, FBI officials recognize that the VCF is only a starting point since numerous other investigative and application systems exist that could be integrated into the VCF. Additionally, because of unresolved connectivity issues, crisis management software may still need to be used by agents after the initial deployment of the VCF. The FBI is identifying and defining other databases and crisis management software that should be included in future VCF releases to maximize the efficiency of the workflow process. FBI officials told us that additional funding will be needed to solve the connectivity issues at remote locations, as well as consolidate and eliminate other databases and crisis management software.

We believe the FBI has demonstrated progress toward implementing these recommendations in the McVeigh report, based on its corrective actions taken to date and its plans for the VCF. However, 11 of these recommendations remain open since the adequacy of the FBI's corrective actions cannot be determined until the VCF has been completed. Because the FBI's ability to implement many IT recommendations and improve its IT program depends on the successful implementation of the VCF, the following sections discuss the factors affecting the success of the VCF.

(d) Factors Affecting the Success of the VCF

In our judgment, if the VCF can do what the FBI expects, it will represent a significant technological advancement from the ACS system. The VCF has the potential to reduce redundancy in searching multiple databases, improve the FBI's case file management, and maximize the use of information in the FBI's possession.

While the VCF has the potential to significantly improve the FBI's information technology, as well as its record management and investigative efficiency, the ultimate success of VCF depends on a number of different factors, including whether the VCF will meet its technical and performance expectations and be accepted and used by FBI employees.

     1. Technical and Performance Expectations of the VCF

To ensure its success, the VCF must meet technical and performance expectations. As mentioned above, the Trilogy project has encountered significant cost overruns and schedule delays due to the FBI not following critical management processes. The OIG's ITIM report stated that these management problems contributed to difficulties with establishing the technical requirements for the VCF. Because the VCF is focused on making significant changes to five of the FBI's investigative systems, documentation for the exact configuration of these legacy systems was critical to designing the requirements for the VCF. Lack of documentation for the configuration of these five investigative systems caused the FBI to engage in a process of reverse engineering, which is trying to determine the structure and components of the systems after deployment. Because the FBI had to perform reverse engineering on five systems, there are limitations as to how rapidly the VCF can be developed and deployed.

As of April 2003, the FBI was still defining the technical requirements for the second and third releases of the VCF. Because the technical requirements had not yet been finalized and funding has not been approved, baselines for the VCF had not been established. We believe that the lack of technical, cost, and schedule baselines not only creates uncertainties for how much the VCF will cost and when it will be completed, but also how it will perform upon implementation.

     2. Acceptance and Use of the VCF

If the VCF is to be a vehicle for moving the FBI's information management into the 21st century, it must be accepted and used throughout the FBI. Historically, the FBI has been a paper-driven organization. A goal of the VCF is to move toward a near paperless environment so the FBI can maximize the use of technology to digitally capture information for data management and control. According to FBI officials, the VCF is the first real change in the FBI's workflow and processes that originated in the 1950's. Director Mueller recently stated that "Trilogy [VCF] will change the FBI culture from paper to electronic."

As noted in the Campaign Finance and McVeigh reports, special agents did not always use the ACS system to manage their case files. For various reasons, they found alternative ways to manage case files. The VCF must be used by all special agents for the FBI to fully realize its benefits.

FBI officials told us that since the VCF will contain the official case files, agents will have to use it since there will be no other acceptable means to manage case files. However, FBI officials also acknowledged that because of unresolved connectivity issues at remote locations, agents may still need to use crisis management software such as Rapid Start.

B. Other Reports Relating to the FBI's IT Program

Three GAO reports that we examined also noted deficiencies with certain aspects of the FBI's IT program. The first report, entitled "Gun Control: Implementation of the National Instant Criminal Background Check System," stated in 1999 that the FBI did not properly accredit and certify the IT system. We later found that the system subsequently was certified and accredited on March 31, 2000. The second GAO report, entitled "Campaign Finance Task Force: Problems and Disagreements Initially Hampered Justice's Investigation," stated in 1999 that the FBI lacked an adequate information system that could manage and interrelate the evidence that had been gathered in relation the Campaign Task Force's investigations. These deficiencies were similar to those reported by the OIG's Campaign Finance report. The third report, entitled "Enterprise Architecture Use Across the Federal Government Can Be Improved," stated in 2002 that the FBI lacked a foundation for managing an enterprise architecture. The recently released OIG ITIM report reiterated the importance of having an established enterprise architecture when developing an IT investment management process. Although these GAO reports did not include any FBI IT-related recommendations, the reports provide further support that previously identified deficiencies continue to affect the FBI.

Other entities have also issued reports in recent years that include analyses of the FBI's IT management. One report relating to IT security was issued by the Webster Commission in March 2002, entitled "A Review of FBI Security Programs." The Commission, chaired by former FBI Director William H. Webster, was established to review the FBI's security practices in light of the espionage by FBI Supervisory Special Agent Robert Hanssen.

The report identified a wide range of problems affecting the FBI's computer systems and information security policies, including:

• Classified information had been moved into systems not properly accredited for protection of classified information.

• Until recently, the FBI had not begun to certify and accredit most of its computer systems, including many classified systems.

• Inadequate physical protections placed electronically stored information at risk of compromise.

• The FBI's approach to system design had been deficient because it had failed to ascertain the security requirements of the "owners" of information on its systems and identify the threats and vulnerabilities that must be countered.

• Classified information stored on some of the FBI's most widely utilized systems was not adequately protected because computer users lacked sufficient guidance about critical security features.

• Some FBI inspectors had insufficient resources to perform required audits, and when audits were performed, audit logs were reviewed sporadically, if at all.

The Webster Commission's report concluded that these findings resulted from the FBI's lack of attention to IT security in developing and managing computer systems.

C. FBI's Process for Following-Up on Recommendations

Until recently, the FBI had not implemented an effective system of management controls to ensure that recommendations are resolved and implemented in a timely and consistent manner. FBI personnel told us that while a formal process to track and resolve recommendations did not exist prior to September 2002, an informal process was used. Upon the final issuance of an OIG or GAO report containing recommendations, the recommendations were forwarded to the various FBI Divisions. Someone within the Division was then assigned to respond to the recommendations until closure occurred. The FBI recognized that this informal process was not sufficient to ensure corrective actions were timely and responsive. Specifically, the FBI officials acknowledged that the informal process:

• was not documented in formal policies and procedures,

• was not adequately monitored by executive management and not kept up-to-date,

• used multiple applications,

• did not keep measures of timeliness and responsiveness, and

• did not provide for sufficient follow-up once the original response or corrective action plan was submitted.

According to the Deputy Assistant Director of the Inspection Division, high turnover within FBI management also contributed to problems with maintaining current responses to OIG and GAO reports. Under the informal process, when individuals left the FBI or were reassigned within the FBI, their replacements were not always made aware of recommendations or requests that were left pending. As a result, responses to recommendations and any related corrective action were often delayed, and the auditing or investigating agency had to again request a response to its recommendations.

The FBI recognized that improvements in its system of managing follow-up were needed to resolve and timely implement recommendations resulting from OIG and GAO reports. In September 2002, the FBI's Inspection Division began to establish a new management process to improve the FBI's timeliness and responsiveness of corrective actions resulting from OIG and GAO recommendations and to bring the FBI in compliance with applicable regulations (OMB Circular A-50 and DOJ Order 2900.6A) for the follow-up and resolution of audit recommendations. To facilitate the implementation of this new management process, the Inspection Division developed a database, referred to as the "Automated Response and Compliance System" (ARCS). According to FBI documentation, ARCS is an automated tool that is intended to:

• document and track audits and data requests from OIG, GAO, and others;

• track OIG and GAO audits, investigations, and reviews until closure; and

• provide status information to FBI's executive management on, or close to, a real time basis.

The FBI's new database tracks the receipt and resolution of audits, investigations, and data requests from OIG, GAO, and others. It also tracks the tasks associated with FBI's current engineering efforts. Among its functions, the database is intended to provide information to FBI managers on a regular basis to keep them informed of a report's progress and to ensure timely implementation of recommendations. However, this database does not include vulnerabilities generated by system audits required by GISRA. The FBI's Information Assurance Section has taken steps to develop a separate database to manage the status of system audit vulnerabilities.

In conjunction with the development of the ARCS database, the FBI has also developed policies and procedures for the Inspection Division's responsibilities for resolving OIG and GAO reports. These policies and procedures require the Inspection Division to assign a liaison for each report with outstanding recommendations or for scheduled audits and reviews. The liaison has the primary responsibility for entering information into the database, including deadlines for when tasks should be completed. The liaison also has the responsibility to ensure that the report is assigned to a "project manager" - who ensures that all tasks are assigned to appropriate FBI personnel. This control ensures that appropriate FBI personnel can be held accountable for taking timely corrective actions. The liaison monitors the completion of tasks and is instructed to send periodic e-mail notices when tasks are near their due date or past due. Additionally, Inspection Division management reviews the activities of the liaisons to ensure that they are adequately monitoring their assigned projects.

FBI officials said that the database, which is maintained on the FBI's intranet, generates reports for senior FBI management on upcoming suspense dates. For example, FBI Deputy Directors are required to perform quarterly reviews on their Division's progress in completing outstanding tasks. According to FBI officials, the Inspection Division Assistant Director uses reports generated by the ARCS database to discuss outstanding tasks at weekly executive meetings, which are attended by FBI Assistant Directors, Executive Assistant Directors, and the Director. These and other reports have been periodically forwarded to the Director, upon his request. FBI officials told us that the Director has taken a particular interest in the timeliness and responsiveness of the FBI's corrective actions, re-engineering efforts, and responses to Congressional requests. The Director asks to be notified, especially with regard to high profile reviews, when the FBI has not been timely and responsive in its planned actions.

While the FBI's database can be a useful tool for the FBI's establishment of a management process directed toward improving the timeliness and responsiveness of its corrective actions, the ultimate effectiveness of this system depends on formal and consistent oversight from senior FBI management. Thus far, however, the FBI has not promulgated written directives FBI-wide that instruct program managers and senior officials (outside of the Inspection Division) regarding their obligation to take corrective actions that will close recommendations. In our judgment, the FBI must develop and institute a formal written process that requires senior management oversight over the timeliness and responsiveness of recommendations. These written procedures should also incorporate the policies for tracking the status of vulnerabilities generated by system audits.

3. OIG Recommendations

In this report, we make three recommendations for the FBI to improve its implementation of IT recommendations. These recommendations are:

• Develop, document, and implement Bureau-wide procedures to follow-up and close audit and investigative recommendations (including those generated from system audits), in accordance with OMB Circular A-50 and DOJ Order 2900.6A.

• Ensure that the ARCS database is complete and includes recommendations from all sources of OIG audits and special reviews.

• Demonstrate through the timely closure of OIG audit and other recommendations that managers are being held accountable for taking corrective actions.

1. The OIG issued one report for FYs 1996 and 1997.
2. Data architecture is the identification and definition of major types of data within an organization.
3. These recommendations were not identified by FISCAM categories.
4. The FISCAM distinguishes system software controls from application software development and change controls. Beginning on page 12, we provide more detailed information on these general control areas.
5. As defined by the American Institute of Certified Public Accountants, a material weakness is a reportable condition in which the design or operation of one or more of the internal control components does not reduce to a relatively low level the risk that misstatements caused by error or fraud in amounts that would be material in relation to the financial statements being audited may occur and not be detected by employees in the normal course of performing their assigned functions.
6. The NIST is a non-regulatory entity of the U.S. Department of Commerce.
7. We included recommendations related to document management because FBI documents are generally produced electronically or managed in automated databases and systems.
8. Of this amount, $57 million was needed for the VCF.