| Return to the USDOJ/OIG Home Page |
Federal Bureau of Investigation's Management of Information Technology Investments
Report No. 03-09
December 2002
Office of the Inspector General
Following the September 11, 2001, terrorist attacks, the Attorney General and the Director of the Federal Bureau of Investigation (FBI) made clear that prevention of terrorism is the top priority of the Department of Justice (DOJ) and the FBI. Effective use of information technology (IT) is crucial to the FBI’s ability to meet this priority as well as its other critical responsibilities.
However, reviews conducted by the Office of the Inspector General (OIG) and the General Accounting Office (GAO) have found major weaknesses associated with the FBI’s IT. The FBI has listed upgrading its information technology as one of its top ten highest priorities. In June 2002 Congressional testimony, the FBI acknowledged that its IT infrastructure is severely outdated.
Because of the importance of the FBI’s management of its IT systems, we performed this audit to: (1) determine whether the FBI was effectively managing its IT investments; and (2) assess the FBI’s IT-related strategic planning and performance measurement activities.1 We also examined the FBI’s efforts to develop enterprise architecture2 and project management capabilities.
In this audit, we conducted approximately 85 interviews with 70 officials from the FBI, DOJ, GAO, and the Office of Management and Budget (OMB). The FBI officials interviewed were from the Director’s office, Information Resources Division, Criminal Justice Information Services Division, Laboratory Division, Inspection Division, and Finance Division. Additionally, OIG auditors and analysts traveled to FBI laboratory facilities in Quantico, VA, and five FBI field offices to conduct interviews and assess the FBI’s implementation of IT initiatives. We also reviewed more than 200 documents, including the FBI’s IT management policies and procedures, project management guidance, strategic and program plans, IT project proposals and management plans, budget documentation, organizational structures, Congressional testimony, and prior OIG and GAO reports.
1. Summary of Audit Findings
We concluded that the FBI has not effectively managed its IT investments because it has not fully implemented the management processes associated with successful IT investments. The foundation for sound IT investment management (ITIM) includes the following fundamental elements:
The FBI failed to implement these critical processes. We found that the FBI does not have fully functioning IT investment boards that are engaged in all phases of IT investment management. The FBI was not following a disciplined process of tracking and overseeing each project’s cost and schedule milestones. The FBI failed to document a complete inventory of existing IT systems and projects, and did not consistently identify the business needs for each IT project. The FBI did not have a fully established process for selecting new IT project proposals that considered both existing IT projects and new projects.
Because the FBI has not fully implemented the critical processes associated with effective IT investment management, the FBI continues to spend hundreds of millions of dollars on IT projects without adequate assurance that these projects will meet their intended goals.
We concluded that these shortcomings primarily resulted from the FBI not devoting sufficient management attention in the past to IT investment management.
However, FBI management has recognized that its past methods to manage IT projects have been deficient, and the FBI recently has committed to changing those practices. In January 2002, the FBI developed a conceptual model for selecting, controlling, and evaluating IT investments. The model seeks to define a process that will promote a Bureau-wide perspective on IT investment management, so that only IT projects with the best probability of improving mission performance are selected. Further, the process is intended to provide the methods, structures, disciplines, and management framework that governs the way IT projects are controlled and evaluated.
In addition to developing a conceptual model for a new ITIM process, in early 2002 the FBI began a pilot test of the new process for the selection ofIT proposals. We found that the FBI made improvements during the pilot testing of the new selection process. Pursuant to the new process, the FBI created three IT investment review boards that reviewed IT proposals for technical compliance and “mission fit.” These boards, comprised of the FBI Director, FBI executives and IT managers, selected new IT proposals that will be considered for inclusion in the Fiscal Year (FY) 2004 budget request.
While the FBI has made efforts to improve its IT investment management practices, the FBI must take further actions to ensure that it can implement the fundamental processes necessary to build an IT investment foundation, as well as the more mature processes associated with highly effective IT investment management. These actions include:
Our audit also reviewed the FBI’s management of Trilogy, the FBI’s largest and most critical IT project. We found that the lack of critical IT investment management processes contributed to missed milestones and led to uncertainties about cost, schedule, and technical goals. Specifically, despite $78 million in additional funding, the FBI missed its July 2002 milestone date for completing the physical IT infrastructure upgrades to field offices, including new computer hardware and networks.3 FBI officials stated that they are not expecting the physical infrastructure components of Trilogy to be completed until March 2003. In addition, the user application component of Trilogy, recognized by FBI officials as the most important aspect of the project in terms of improving agent performance, is at high risk of not being completed within the funding levels appropriated by Congress. In our judgment, the management problems associated with Trilogy demonstrate the FBI’s urgent need for enhanced IT investment management.
We also concluded that the FBI’s IT strategic planning and IT performance measurement are inadequate. We found that the FBI's strategic plan does not include goals for IT investment management, and the FBI’s strategic plan and performance plan are not consistent with the DOJ’s annual performance plan.
The remainder of this executive summary provides more background and details on our audit findings and recommendations to help improve the FBI’s management of its IT investments.
2. Background
The Clinger-Cohen Act of 1996 requires each federal agency to implement a process for maximizing the value of its IT investments. This process is intended to ensure that IT projects are being implemented at acceptable costs and within reasonable time frames, and that the projects are contributing to enhanced mission performance. Specifically, the Clinger-Cohen Act requires federal agencies to: (1) develop an enterprise architecture framework, and (2) follow a “select/control/evaluate” approach to managing IT investments.
In May 2000, the GAO developed the IT Investment Management Framework (Framework) to provide a common methodology for assessing IT capital planning and investment management practices at federal agencies. The Framework specifically describes the organizational processes required to carry out sound IT investment management.
The Framework, based on best practices of leading organizations, is a hierarchical model comprised of five maturity stages. These maturity stages represent steps toward achieving stable and mature investment management processes. As agencies advance through these stages, their capability to effectively manage IT increases. With the exception of the first stage, each maturity stage is comprised of critical processes that must be implemented and institutionalized for the agency to satisfy the requirements of that stage. These critical processes are further broken down into key practices an agency should perform to successfully implement each critical process.
An agency using these critical processes is in a better position to successfully invest in IT and use its IT investments to achieve its priorities. Conversely, an agency that does not have these critical processes in place is at high risk that its IT projects will fail to support the achievement of priorities.
To determine whether the FBI was effectively managing its IT investments, we utilized the Framework because it is: (1) a standardized tool for internal and external evaluations of an agency’s IT investment management process; (2) a consistent and understandable mechanism for reporting the results of these assessments; and (3) a road map agencies can use for improving their IT investment management process.
In addition, the Government Performance and Results Act of 1993 (Results Act) requires strategic planning and performance measurement throughout the federal government. The Results Act seeks to improve the effectiveness, efficiency, and accountability of federal programs by requiring federal agencies to establish goals for program performance and measurement. The Results Act requires agencies to prepare a strategic plan, annual performance plan, and annual performance report.
While IT strategic planning is a function somewhat independent of IT investment management, these two functions are interrelated and complementary. The DOJ has recognized the importance of integrating strategic planning with IT management. In July 2002, the DOJ released its IT Strategic Plan that included a strategic initiative to establish and improve investment management processes.
3. The FBI’s Management of IT Investments
Our audit found that the FBI has not established an IT investment foundation and therefore is in Stage One maturity according to the ITIM Framework. Stage One maturity is characterized by inconsistent, unstructured, and unpredictable investment processes. Our observations of the FBI’s IT investment processes found that the FBI’s actual processes are consistent with these Stage One deficiencies.
The critical processes necessary to establish an IT investment foundation include: (1) defining investment review board operations, (2) developing project-level investment control processes, (3) identifying IT projects and systems, (4) identifying the business needs for each IT project, and (5) developing a basic process for selecting new IT proposals.
We found that the FBI failed to implement these critical processes. The FBI did not have a fully established investment review board operation because the FBI did not provide adequate resources for operating the IT investment boards. Additionally, we found insufficient evidence to demonstrate that: (1) organization executives and line managers supported and carried out IT investment board decisions and (2) board members understood the investment board’s policies and procedures and exhibited core competencies in using the IT investment approach via training, education, or experience. Specifically, the FBI did not provide ample time to adequately prepare and train IT board members prior to initiating the pilot test of its recently developed ITIM process. This resulted in inadequate training of board members and minimal preparation time to develop IT proposals. For example, Technical Review Board members had only three business days to review over 50 IT proposals prior to their first board meeting.
Additionally, we found that the FBI is not effectively overseeing its IT projects. For example, while the FBI has issued project management guidance, the guidance is not being followed on a consistent basis. Depending on whom we talked to, we obtained different answers as to which document represented the FBI’s official project management guidance.
Without effective oversight of IT projects, FBI officials do not have adequate assurance that IT projects are being developed on schedule and within established budgets. According to a former Chief Information Officer at the FBI, the lack of effective oversight of IT projects has prevented IT project managers from being held accountable for cost and schedule overruns and the ultimate performance of projects. Senior FBI officials also told us that the Bureau’s budget formulation process focuses only on the acquisition costs for IT projects and not the full life-cycle costs, especially operations and maintenance costs.
We also found that the FBI’s investment review boards are not aware of all the IT projects and resources for which the boards are responsible. FBI Divisions maintained some version of an IT inventory for the projects and systems under their jurisdiction, and there was no centralized office responsible for maintaining a uniform listing Bureau-wide. FBI managers told us they were in the process of developing an IT asset inventory, but at the time of our audit they were unable to provide an estimated date for completing the inventory.
FBI personnel told us that staff shortages are the primary cause for the incomplete IT asset inventory. In our judgment, staff shortages may be a contributing factor, but the lack of centralized management over IT investments was the significant reason for this problem. Until June 2002, the FBI did not have a centralized project management office to assist the investment boards in overseeing IT projects. The FBI maintained three separate division-level project management offices to manage IT projects.
We also determined that the FBI did not have a fully established process for selecting IT proposals. FBI officials told us that, prior to March 2002, individual divisions determined IT needs in a “stovepipe,” without knowledge of the business needs and priorities of the Bureau as a whole. The FBI did not have a clearly designated official to manage the proposal selection process. According to Information Resources Management Section personnel, the Finance Division managed the IT selection process. However, according to Finance Division personnel, the Information Resources Management office was responsible for managing the proposal selection process.
Without a comprehensive proposal selection process that includes adequate resources and training, the FBI cannot ensure that it is selecting the best IT projects that meet mission-critical needs.
Because the FBI did not fully implement any of the critical processes associated with Stage Two, the FBI continues to spend hundreds of millions of dollars on IT projects without having adequate selection and project management controls in place to ensure that IT projects will deliver their intended benefits.
The FBI began pilot testing the select phase of its new ITIM process in March 2002, and since then has made measurable progress towards implementing the key practices that comprise the critical processes – particularly in the area of selecting new proposals for IT projects. Specifically, at the beginning of our audit in January 2002, the FBI only was executing 4 of the 38 required key practices; however, as of June 2002, the FBI was executing 14 of the key practices.
With the pilot testing of its new ITIM process, the FBI created an IT investment process guide containing policies and procedures to direct board operations, and created and defined three investment review boards integrating both IT and business knowledge. Additionally, the FBI has designated an official responsible for managing the IT project and system identification process and ensuring that the inventory meets the needs of the investment management process. Further, during the test pilot of the ITIM process, the board reviews of IT project proposals provided assurance that business needs were clearly identified and defined. Also during the test pilot, we determined that FBI IT investment board members analyzed and prioritized new IT proposals according to established selection criteria for the FY 2004 budget cycle.
Despite the progress made, full implementation of the ITIM process will require the FBI to (1) fully develop and document its new ITIM process; (2) require more input and participation from IT managers and users; and (3) further develop its project management and enterprise architecture functions. Completion of the initial steps taken by the FBI will ensure that IT projects are developed within cost and schedule requirements, and meet performance expectations. The Trilogy project provides an example of how the non-implementation of fundamental IT investment management practices can put a project at risk of not delivering what was promised, within cost and schedule requirements.
4. Trilogy
We also performed a case study of the FBI’s implementation of its Trilogy project. We selected Trilogy because it is the FBI’s largest ongoing IT project and is considered vital to the FBI’s ability to perform its mission. Trilogy is intended to upgrade the FBI’s: (1) hardware and software – referred to as the Information Presentation Component (IPC), (2) communication networks – referred to as the Transportation Network Component (TNC), and (3) five most important investigative applications – referred to as the User Applications Component (UAC). The IPC and TNC upgrades will provide the physical infrastructure needed to run the applications from the UAC portion. The UAC portion is intended to upgrade and consolidate five of the FBI’s 42 investigative applications. Because of the 37 other investigative applications and approximately 160 non-investigative applications that Trilogy will not cover, Trilogy is only a starting point towards upgrading the FBI’s entire IT infrastructure. According to the FBI, Trilogy is not designed to provide the FBI with state-of-the-art IT; it is intended to provide the foundation so that the FBI can eventually attain state-of-the-art IT.
In November 2000, Congress appropriated $100.7 million for the first year of the $379.8 million Trilogy project, which was to be funded over a three-year period (from the date contractors were hired). The $100.7 million was a combination of new program funding and a re-direction of base resources. When the FBI requested contractor support for Trilogy, it combined the IPC and TNC portions for continuity as both encompass physical IT infrastructure enhancements. The contractor for the IPC/TNC portions was hired in May 2001, and the originally scheduled completion date for these components was May 2004. A different contractor was hired in June 2001 to complete the UAC portion of Trilogy by June 2004.
After the terrorist attacks on September 11, 2001, the urgency of completing Trilogy increased, and the FBI explored options to accelerate the deployment of all three components of Trilogy. The FBI informed Congress in February 2002 that, with an additional $70 million, the FBI could accelerate the deployment of Trilogy. This acceleration would include completion of the IPC/TNC phase by July 2002 and rapid deployment of the most critical analytical tools included as part of the UAC phase.
In January 2002, Congress supplemented Trilogy’s FY 2002 budget with $78 million4 to expedite the deployment of all three components. This supplemental appropriation increased the total funding of Trilogy from approximately $380 million to $458 million.
Even with these additional funds, the FBI missed its July 2002 milestone date for completing the IPC and TNC phases. FBI officials stated that they are not expecting these components of Trilogy to be completed until March 2003. In addition, the user application component of Trilogy, recognized by FBI officials as the most important aspect of the project in terms of improving agent performance, is at high risk of not being completed within the funding levels appropriated by Congress. Further, despite receiving an additional $78 million from Congress in January 2002, FBI managers have acknowledged to us that the last phase of UAC will not be completed any sooner than originally planned (in June 2004).
In terms of a cost baseline, FBI officials told us that the rapid procurement and deployment of Trilogy has prevented the project managers from performing earned value management,5 as promised to Congress. While FBI officials were confident they know how much money has been spent on Trilogy to date, and how much funding has been committed, they have less assurance as to whether Trilogy is on budget, over budget, or under budget.
A schedule baseline for Trilogy has never been well-established. First, FBI officials said they would complete IPC/TNC deployment in May 2004. Then, they said it could be finished in June 2003. Next, they said it would be finished by December 2002. After receiving $78 million of supplemental funding, they said it would be done by July 2002. Then, they said they could not make the July 2002 deadline and moved it to October 2002. As of June 2002, FBI officials have said deployment will probably not be complete until March 2003. Also as of June 2002, the FBI was still in the process of building a comprehensive schedule of Trilogy milestones.
Regarding the technical requirements for Trilogy, we were told that some aspects of Trilogy as submitted to Congress did not turn out to be technically feasible. For example, FBI officials told us that the thin-client strategy was not pursued because it was found that this type of network could not be achieved given the technical requirements of the FBI.6 Another example is web-enablement of the Automated Case Support (ACS) system, which was also discontinued when it was realized that it would require more resources than anticipated.7 Had a more rigorous proposal selection process been in place to require sufficient documentation of the technical requirements and risks of the project, the expending of time and resources on thin-client technology and web-enablement of ACS may have been minimized.
Another technical issue involves the development of the UAC portion of Trilogy. Because the UAC portion is focused on making significant changes to, or possibly complete replacements of, five of the FBI’s investigative systems, documentation for the exact configuration of these systems is critical to designing the requirements for UAC. According to a senior FBI official, the FBI must know what it has before it can define the right solution to fix the problem. Lack of documentation for the configuration of these five investigative systems has caused the FBI to engage in a process of reverse engineering, which is trying to determine the structure and components of the systems after deployment. Because the FBI has to perform reverse engineering on the FBI’s five investigative systems, there are limitations as to how rapidly UAC can be developed and deployed.
Our observations at five FBI field offices indicated that deployment of the IT physical infrastructure was still ongoing as of June 2002. For two field offices, additional installation work remained to be completed, and for four field offices hundreds of desktop computers still remained to be delivered. A lack of clear communication between FBI Headquarters and the field offices contributed to the confusion over the number of desktop computers to be delivered and shortages of fiber optic cable. Additionally contractor maintenance support for the Trilogy architecture was inefficient, resulting in agents being without computers for weeks at a time. Improvements in agent and support personnel training, procurement of trouble-shooting equipment for the Trilogy architecture, and timely completion of FBI unique macros for Microsoft Word will enhance user utilization of the Trilogy architecture.
The new Trilogy project executive, hired in March 2002, has taken a different approach to managing Trilogy. She has emphasized the importance of having more structured oversight of the project. She has been developing a comprehensive schedule for all three components. Additionally, she has indicated that there are limitations to how fast Trilogy can be deployed, without risking the security of the system. In our judgment, while these actions taken since March 2002 represent positive changes to Trilogy’s project management function, the project’s completion time, final cost, and ultimate performance remain uncertain. Also, we concluded that for the Trilogy project management function to be effective, it must include oversight from IT investment review boards to provide much needed monitoring.
5. FBI’s IT Strategic Planning and Performance Measurement
We also assessed the FBI’s IT strategic planning and performance measurement. We found that the FBI’s strategic plan does not include IT investment management goals and the FBI’s strategic plan and performance plan are not consistent with the DOJ’s annual performance plan. Also, as of the end of June 2002, the FBI did not have a current strategic plan dedicated to IT. Instead, individual FBI divisions had program plans that included the use of IT within particular programs.
This occurred because the FBI has not updated its strategic plan since 1998, and its performance plan does not include the same strategic objectives, goals, and strategies relating to IT as does the DOJ's annual performance plan. We believe that the FBI will have difficulty improving its IT investment management process without incorporating it into the strategic plan. Additionally, without adequate strategic planning and performance measurements, there is a heightened risk that the FBI may not be appropriately allocating resources to meet the DOJ’s strategic priorities.
In our judgment, the FBI must change the division-specific IT focus and implement a Bureau-wide IT strategic plan. The purpose of the FBI’s ITIM process is to move away from the decentralized IT focus to a centralized one. As a result, we recommend that the FBI update its IT strategic plan and performance plans to (1) fully integrate these plans with the FBI’s ITIM process; and (2) include those performance goals and indicators defined in the DOJ’s IT Strategic Plan.
6. OIG Recommendations
In this report, we make 30 recommendations that focus on specific and immediate steps the FBI should take to help improve its IT investment management. These recommendations include:
7. Conclusion
The underlying practices we assessed are fundamental to any project management endeavor. However, the FBI has not executed the majority of these tasks to select and manage its IT resources. For example, organizational policies were not clearly established to ensure that critical IT investment policies endure. Additionally, there were no clearly defined, uniform procedures for project management, tracking project performance, and taking corrective actions as necessary. Prior to the development of its ITIM process in early 2002, the FBI did not give sufficient attention to IT investment management. Since the FBI developed its ITIM process in early 2002, it has focused more management attention in this area and has made progress towards attaining a basic IT investment management foundation. Despite the progress, the FBI did not fully implement any of the critical processes necessary to build an IT investment foundation. As a result, the FBI continues to spend hundreds of millions of dollars on IT projects without having adequate selection and project management controls in place to ensure that IT projects will deliver their intended benefits.