The Federal Bureau of Investigation's
Management of the Trilogy Information Technology Modernization Project

Audit Report No. 05-07
February 2005
Office of the Inspector General


Appendix 2

Prior Reports on the FBI's Information Technology


Below is a listing of relevant reports concerning the FBI’s information technology systems. These include reports issued by the Department of Justice, Office of the Inspector General (OIG), the Government Accountability Office (GAO), and from other external entities, as well as FBI internal reports.

OIG Reports on the FBI's IT

OIG reports issued over the past 14 years have highlighted issues concerning the FBI’s utilization of IT, including its investigative systems. In 1990, the OIG issued a report entitled, The FBI’s Automatic Data Processing General Controls. This report described 11 internal control weaknesses and found that:

  • the FBI’s phased implementation of its 10-year Long Range Automation Strategy, scheduled for completion in 1990, was severely behind schedule and may not be accomplished;

  • the FBI’s Information Resources Management program was fragmented and ineffective, and the FBI’s Information Resources Management official did not have effective organization-wide authority;

  • the FBI had not developed and implemented a data architecture; and

  • the FBI’s major mainframe investigative systems were labor intensive, complex, untimely, and non-user friendly and few agents used these systems.

The OIG’s July 1999 special report, The Handling of FBI Intelligence Information Related to the Justice Department’s Campaign Finance Investigation, stated that FBI personnel were not well versed in the ACS system and other databases. Additionally, a November 1999 OIG report entitled A Review of the Justice Department’s Handling of the Death of Kenneth Michael Trentadue at the Bureau of Prison’s Federal Transfer Center in Oklahoma City, noted deficiencies in uploading key evidence into the ACS.

A March 2002 OIG report entitled, An Investigation of the Belated Production of Documents in the Oklahoma City Bombing Case, analyzed the causes for the FBI’s belated delivery of many documents in the Oklahoma City bombing case. This report concluded that the ACS system was extraordinarily difficult to use, had significant deficiencies, and was not the vehicle for moving the FBI into the 21st century. The report noted that inefficiencies and complexities in the ACS, combined with the lack of a true information management system, were contributing factors in the FBI’s failure to provide hundreds of investigative documents to the defendants in the Oklahoma City bombing case.

In May 2002, the OIG issued a report on the FBI’s administrative and investigative mainframe systems entitled the Independent Evaluation Pursuant to the Government Information Security Reform Act, Fiscal Year 2002. The report identified continued vulnerabilities with management, operational, and technical controls. The report stated that these vulnerabilities occurred because the Department and FBI security management had not enforced compliance with existing security policies, developed a complete set of policies to effectively secure the administrative and investigative mainframes, or held FBI personnel responsible for timely correction of recurring findings. Further, the report stated that FBI management has been slow to correct identified weaknesses and implement corrective action and, as a result, many of these deficiencies repeat year after year in subsequent audits.

In December 2002, the OIG issued a report on The FBI’s Management of Information Technology Investments, which included a case study of the Trilogy project. The report made 30 recommendations, 8 of which addressed the Trilogy project. The report’s focus was on the need to adopt sound investment management practices as recommended by the GAO. The report also stated that the FBI did not fully implement the management processes associated with successful IT investments. Specifically, the FBI had failed to implement the following critical processes:

  • defining and developing IT investment boards,

  • following a disciplined process of tracking and overseeing each project’s cost and schedule milestones over time,

  • identifying existing IT systems and projects,

  • identifying the business needs for each IT project, and

  • using defined processes to select new IT project proposals.

The audit found that the lack of critical IT investment management processes for Trilogy contributed to missed milestones and led to uncertainties about cost, schedule, and technical goals.

In September 2003, the OIG issued a report entitled The Federal Bureau of Investigation’s Implementation of Information Technology Recommendations that outlined the FBI’s continued need to address the recommendations made by oversight organizations concerning its IT strategies. The report stated that although OIG audits found repeated deficiencies in the FBI’s IT control environment and lack of compliance with information security requirements, the current FBI leadership has committed to enhancing controls to ensure that recommendations are implemented in a consistent and timely manner. Additionally, the report noted that the FBI established a system to facilitate the tracking and implementation of OIG recommendations.

External Reports on the FBI’s IT and Trilogy

In March 2002, the Commission for the Review of FBI Security Programs issued a report entitled A Review of FBI Security Programs. This commission, chaired by former FBI Director William H. Webster, was established to investigate the espionage of former FBI Supervisory Special Agent, Robert Hanssen. The report identified a wide range of IT security issues, including Hanssen’s utilization of the ACS system to obtain information for the Soviet Union and to track an FBI counterintelligence investigation. According to Hanssen, "any clerk in the Bureau could come up with the stuff on that system," and he described the lack of security on the ACS system as criminal negligence. The report asserted that many of its findings resulted from the FBI’s lack of attention to IT security in developing and managing computer systems.

The National Research Council of the National Academies issued a report in May 2004 entitled A Review of the FBI’s Trilogy Information Technology Modernization Program. The report was updated in June 2004 to reflect the FBI’s response to the report, because significant changes had occurred in many of the areas critically reviewed by the Council. The original report identified significant issues in four major areas: Enterprise Architecture, system design, project and contract management, and human resources. For each of these areas, recommendations were made to address the likelihood of success in and drive an accelerated pace for the FBI’s IT modernization efforts. The report concluded that the FBI had made significant progress in some areas of its IT modernization efforts, such as the modernization of the computing hardware and baseline software and the deployment of its networking infrastructure. However, because the FBI’s IT infrastructure was so inadequate in the past, there was still an enormous gap between the FBI’s IT capabilities and the capabilities that are urgently needed.

The update to the report also stated that the Council saw clear evidence of progress being made by the FBI to move ahead in its IT modernization program. This included the appointment of a permanent CIO and the formation of a staffed program office for improved IT contract management. The progress being made by the FBI appeared to the Council to have been more rapid than expected, although many challenges remained. Sustained success in IT, the update noted, "require strong and forceful leadership over an extended period of time." The Council also emphasized that the FBI’s missions constitute increasingly information-intensive challenges, and the ability to integrate and exploit rapid advances in IT capabilities will only become more critical with time. The update concluded that even with perfect program management and execution, substantial IT expenses on an ongoing basis are inevitable and must be anticipated in the budget process if the FBI is to maximize the operational leverage that IT offers. The update also concluded that no one should expect a decrease in expenses for IT when the Trilogy program is completed.

The GAO has issued several reports and related testimony that highlight deficiencies with the FBI’s IT. In a review of the Department’s Campaign Finance Task Force, the GAO reported in May 2002 that the FBI lacked an adequate information system that could manage and interrelate the evidence that had been gathered in relation to the Task Force’s investigations. Also, as part of a government-wide assessment of federal agencies, the GAO reported in February 2002 that the FBI needed to fully establish the management foundation that is necessary to successfully develop, implement, and maintain an Enterprise Architecture.

In September 2003, the GAO issued a report entitled, Information Technology: FBI Needs an Enterprise Architecture to Guide Its Modernization Activities. This report reiterated the GAO’s assertion, made in the May 2002 report on the Department’s Campaign Finance Task Force, that the FBI did not have an Enterprise Architecture, although it had begun efforts to develop one. Additionally, the GAO found that the FBI still did not have the processes in place to effectively develop, maintain, and implement an Enterprise Architecture.

In September 2004, the GAO issued a report entitled, Information Technology: Foundational Steps Being Taken to Make Needed FBI Systems Modernization Management Improvements. This report stated that although improvements are under way and more are planned, the FBI did not have an integrated plan for modernizing its IT systems. Each of the FBI’s divisions and other organizational units that manage IT projects performs integrated planning for its respective IT projects. However, the plans did not provide a common, authoritative, and integrated view of how IT investments will help optimize mission performance, and they do not consistently contain the elements expected to be found in effective systems modernization plans. The GAO recommended that the FBI limit its near-term investments in IT systems until the FBI develops an integrated systems and modernization plan and effective policies and procedures for systems acquisition and investment management. Additionally, the GAO recommended that the FBI’s CIO be provided with the responsibility and authority to effectively manage IT FBI-wide.

FBI Internal Assessments on Trilogy

In 2001 and 2002 the FBI performed internal assessments concerning the management of the Trilogy project. The FBI’s Inspection Division, Criminal Justice Information Services Division (CJIS), and a contractor performing independent verification and validation (IV&V) work for the FBI completed these assessments. The assessments found that a lack of baselines and oversight posed potential risks for the Trilogy project to meet its budget, schedule, technical, and performance goals. The assessments recommended that the FBI designate a program manager specifically for Trilogy, and that the program manager immediately take steps to establish baselines and requirements for the project.

The assessments addressed areas of potential risk within Trilogy, such as security and configuration management.21 Based on the recommendations of these reports, the OIG recommended in its December 2002 report on the FBI’s IT investment management (ITIM) that Trilogy project managers prepare an action plan to address the risks identified by the three internal reports on Trilogy. Overviews of the three independent assessments (FBI Inspection Division Trilogy Risk Assessment, November 2001; Trilogy Independent Validation and Verification, December 2001; and CJIS Division Trilogy Assessment, January 2002) follow.

Inspection Division Trilogy Risk Assessment

Because of the size and importance of Trilogy to the FBI, the FBI Inspection Division’s Major Project Management Oversight Unit (MPMOU) issued a risk assessment report to the Director in November 2001. This assessment identified areas of high risk within the acquisition, financial, requirements, and overall project management of Trilogy. The areas found to be high risk included a lack of project requirements and baselines, the lack of a defined program organizational structure and program manager, and improper scheduling and cost estimates.

The report recommended that the FBI institute a short-term strategy to provide interim capabilities and a long-term strategy to restructure Trilogy. The report also recommended that the short-term strategy should include a detailed plan identifying what can realistically be accomplished within a pre-determined period. It further stated that the short-term plan should have a clearly defined scope so that progress can be measured and quantified.

The MPMOU issued follow-up letters to the Director in December 2001 and February 2002 assessing the FBI’s progress in taking action on the recommendations and mitigating Trilogy’s risks. In December 2001, the Inspection Division stated that while Trilogy project managers acknowledged certain project risks, the managers were willing to accept aspects of those risks and move forward. However, to address those risks, FBI senior management hired a program manager for Trilogy in March 2002.

In February 2002, the Inspection Division’s letter to the Director stated that Trilogy project managers disagreed on the level of project risk for Trilogy. The Inspection Division pointed to the CJIS review and an outside independent validation and verification report on Trilogy, both discussed below, establishing that significant risks to the project exist in the areas originally identified by the Inspection Division. The Inspection Division reiterated its previous recommendation that called for the development of a short-term strategy and a long-term strategy for Trilogy. Inspection Division personnel told us that Trilogy management did not sufficiently develop these recommended strategies.

Trilogy Independent Validation and Verification

The FBI hired an outside contractor to determine the labor requirements, level of effort, and verification and validation tasks necessary to ensure that the Trilogy acquisition would meet the requirements of FBI users into the future within the established schedule and budget. The IV&V report, issued in December 2001, disclosed risks in the program management of Trilogy, the IPC/TNC portion, and the UAC portion, including: 1) a lack of program management structure and focus; 2) a lack of formal requirements, schedules, and baselines; 3) modifications to the UAC/IPC/TNC portions without formal changes to the contracts.

CJIS Division Trilogy Assessment

Upon reviewing the Inspection Division’s risk assessment, the Director requested the CJIS Division perform an independent review of Trilogy to get another perspective on the project. The CJIS Division performed its assessment from January 3-16, 2002. This assessment covered management, quality assurance, configuration management, IT security, administrative and technical requirements, and technical management. The assessment found weaknesses similar to those identified by the Inspection Division, including: 1) a lack of clear lines of authority; 2) no clearly designated Program Manager; 3) a lack of authority and support in the areas of quality assurance, security, configuration management, and technical requirements; and 4) insufficient technical reviews of Trilogy documentation.


Footnotes

  1. Configuration management is the discipline of identifying the configuration of hardware or software systems at each life cycle phase for the purpose of controlling changes to the configuration and maintaining the integrity and traceability of the configuration through the entire life cycle.