The Status of Enterprise Architecture and Information Technology
Investment Management in the Department of Justice
Audit Report 06-02
Office of the Inspector General
Efforts to develop a Department Enterprise Architecture have been underway since 1999. However, the Department's efforts to develop an Enterprise Architecture have suffered from a lack of institutional commitment and a changing perception of the composition of, and priority for, a Department-level Enterprise Architecture. Adding to this confusion are the additional Enterprise Architectures developed by components.
In 2001, the Department began developing an Enterprise Architecture based on the Federal Enterprise Architecture Framework (FEAF). The Department secured funding and hired System, Data, Infrastructure, and Business Architects and an Investment Management Coordinator. This group assembled "as-is" business, data, and application architectures by December 2001. However, a Department official told us that other priorities prevented this early Enterprise Architecture effort from continuing. Further, the "as-is" architectures were not updated and were not useful for later efforts to develop a Department-wide Enterprise Architecture.
In 2002, the Department began using the Federal Enterprise Architecture Management System (FEAMS), a web-based automated tool that provides agencies with access to initiatives aligned to the FEAF and associated reference models to assist in developing an Enterprise Architecture.7 The FEAMS was designed in close cooperation with the OMB, and the OMB required the Department to use the FEAMS to develop its Enterprise Architecture. According to a Department official, the Department considered the FEAMS to be a cumbersome system that made inputting and extracting data difficult. Further, while the system served as a storage place for models, it could not perform analyses. Consequently, despite the OMB's direction, the Department discontinued the FEAMS.
In 2003, the Department piloted the Popkin System Architect software for use as its automated tool. Although the DEA used the Popkin software in developing its Enterprise Architecture, a Department official stated that Popkin would require significant modifications to serve the Department's purposes. Based on the results of the pilot, the Department decided not to use Popkin. A Department official stated that commercial off-the-shelf tools are now being explored as aids to the development of the Enterprise Architecture. However, the Department has no timetable for acquiring an automated tool to document the development of the Department's Enterprise Architecture. In addition, Department officials were unable to provide expenditure data for Enterprise Architecture efforts prior to FY 2004.
After rejecting the FEAF along with the FEAMS automated tool, the Department began devising its own framework intended to lead to a Department-wide Enterprise Architecture. The Department expects the framework, called the Capability Delivery Model, to be completed in late FY 2005 and the resulting Enterprise Architecture by late FY 2009. According to Department officials, the Capability Delivery Model, while including the basic elements of the FEAF, will not be as high-level as the FEAF, but rather is intended to be more useful and relevant to day-to-day operations. The Department expects the Enterprise Architecture developed through the Capability Delivery Model to cover the Department's major, cross-cutting IT systems and enable the Department to more effectively and efficiently manage its current and future IT infrastructure and applications.
The Department anticipates the Capability Delivery Model will be a more detailed, refined, Department-specific version of the FEAF. The foundation of the model is the Department's mission areas. For each mission area, component-specific goals and objectives will be developed, and capabilities will be identified to achieve them. Mechanisms - including systems, hardware, and software - will be obtained to support multiple capabilities. This process is illustrated in the following diagram.
The Capability Delivery Model is being developed by creating several pilot architectures for categories of systems, such as an architecture for Terrorism Information Sharing, which stems from the Department's mission to prevent terrorism. A goal derived from this mission is the sharing of information among Department components involved in counterterrorism efforts. Examples of objectives within this goal are the interoperability, accessibility, and security of shared information. Once the goals and objectives are clarified, relevant component business processes are evaluated to develop a capability to meet them. In this example, the capability is the Intelligence Terrorism Information Sharing Environment. Specific IT mechanisms will then be put in place to enable the Terrorism Information Sharing Environment to be implemented.
The Department has developed Capability Architectures for Security, Public Key Infrastructure (PKI), and Telecommunications, and is currently developing the following Capability Architectures:
The Department expects to combine these Capability Architectures to form an overall Department-level Enterprise Architecture and then use it to manage the development of IT systems that cross-cut multiple Department components. This approach of using Capability Architectures is what makes the Department's Enterprise Architecture different from one created through the FEAF. The FEAF methodology relies on the development of various reference models that describe an organization's business, data necessary to conduct the business, applications to manage the data, and technology to support the applications. Instead of relying on various reference models, the Department's Enterprise Architecture will focus on the specific missions of the organization. Department managers told us that this approach provides a more specific and useful architecture tailored to the Department. While these two methodologies for developing Enterprise Architectures differ, Department officials stated that the elements required in the FEAF will be present in the Department's Enterprise Architecture.
We used the criteria in the GAO's Enterprise Architecture framework to evaluate the Department's progress in developing a Department-wide Enterprise Architecture. To implement each of the five maturity stages of the GAO framework discussed below, the Department must complete four critical success attributes: (1) demonstrate commitment, (2) provide the capability to meet the commitment, (3) demonstrate satisfaction of commitment, and (4) verify satisfaction of commitment. Each attribute contains core elements that contribute to the effective implementation and institutionalization of the critical success attribute. Collectively, these attributes form the basis by which an organization can institutionalize management of any given function or program.
We found that the Department has nearly completed what equates to Stage 2 of the five-stage GAO framework and has made some progress toward the third stage of maturity.
Stage 1 — Completed
In meeting the criteria for this stage, the Department created an awareness of the value of developing and using an Enterprise Architecture by providing the management foundation necessary for successful Enterprise Architecture development, as defined in Stage 2.
Stage 2 — Nearing Completion
The Department has completed five of the nine core elements required by the GAO framework and has achieved one of the four critical attributes. To meet the criteria for this stage, the Department needs to: (1) ensure the existence of adequate resources; (2) establish Department-wide committees responsible for directing, overseeing, and approving the Enterprise Architecture; (3) develop the Enterprise Architecture using an automated tool; and (4) develop metrics for measuring Enterprise Architecture progress, quality, compliance, and return on investment.
Critical Attribute 1: Demonstrates Commitment
To complete the first critical attribute for Stage 2 of the GAO framework, the Department must demonstrate its commitment to building an Enterprise Architecture management foundation by establishing two core elements:
We determined the Department has not fully implemented the two core elements under the first critical attribute for Stage 2.
Adequate Resources. Obtaining adequate resources includes: (1) identifying and securing the funding necessary to support Enterprise Architecture activities; (2) hiring and retaining employees with the proper knowledge, skills, and abilities to plan and execute the Enterprise Architecture program; and (3) selecting and acquiring the tools and technology to support Enterprise Architecture activities.
According to a Department official, the Department spent approximately $1 million on developing its Enterprise Architecture in FY 2004 and plans to spend approximately $1.1 million in FY 2005, amounts that appear adequate for continuing development at this point. The Enterprise Architecture Program Management Office includes the Chief Architect, Enterprise Architecture Program Manager, Business Architect, Systems Architect, Data Architect, Infrastructure Architect, Security Architect, Configuration Manager, Senior Systems Architecture Consultant, and Technical Writer. In our opinion, these employees have sufficient knowledge and experience to establish an Enterprise Architecture.
However, the Department does not yet have a tool to assist in the development of its Enterprise Architecture that clearly and completely documents the Department's Enterprise Architecture. As discussed previously, the Department tested the Popkin System Architect tool and found it unacceptable. The Department is in the process of identifying tools and technology to support its Enterprise Architecture activities. Because the Department does not have all the adequate resources for an Enterprise Architecture, the first core element is not fully implemented.
Enterprise Architecture Governing Committees. Responsibility for directing, overseeing, and approving architectures should be given to a committee or group with cross-representation from throughout the enterprise. Establishing agency-wide responsibility and accountability is important to demonstrate the agency's commitment to building a management foundation for the Enterprise Architecture and obtaining buy-in from across the agency. Accordingly, the committee or group should include executive-level representatives from each line of the business, and these executive representatives should have the authority to commit resources and enforce decisions within their respective organizational units.
The Department had established an Enterprise Architecture Committee (EAC) in 2001, which reported to the Department of Justice CIO Council. However, the EAC is no longer active.8 The EAC was established to support the formulation and adoption of a Departmental Enterprise Architecture by ensuring that the Department-level Enterprise Architecture met all federal requirements. Further, the EAC was a deliberative body for the Department's chief IT architects to:
In our judgment, the membership of the EAC demonstrated an agency-wide leadership commitment to the Enterprise Architecture process. The EAC was comprised of the Chief Architects from the Federal Bureau of Prisons; DEA; FBI; Bureau of Alcohol, Tobacco, Firearms and Explosives (ATF); Office of Justice Programs; Executive Office for U.S. Attorneys; JMD; U.S. Marshals Service; and other key architects within the Department. Also, a component CIO was designated to serve as EAC Chair, and the Department's Chief Architect was designated vice-chair.
The Committee met monthly from 2001 to 2002, then intermittently until disbanding in early 2004. Although a Department official stated the committee planned in early 2004 to regroup and begin meeting again, the Committee has been inactive since early 2004. The official explained that the Committee stopped meeting to rethink, regroup, and decide where the Department-wide Enterprise Architecture efforts were going. Therefore, the Department no longer meets one of the core elements required under the GAO framework to demonstrate its commitment.
Critical Attribute 2: Provides Capability to Meet Commitment
The completion of the second critical attribute for achieving Stage 2 requires the Department to establish three core elements:
The Department has made progress toward implementing these three core elements. The Department has implemented core elements 1 and 2, but core element 3 is not fully implemented.
Enterprise Architecture Program Office. Enterprise Architecture development and maintenance should be managed as a formal program. Accordingly, responsibility for Enterprise Architecture management should be assigned to an organizational unit and not an individual. The CIO Practical Guide, discussed in the Background section of this report, states that the primary responsibility of the Enterprise Architecture Program Office is to ensure the success of the Enterprise Architecture program.
Within the Department, JMD's Policy and Planning staff is responsible for maintaining, refining, updating, and applying the Department Enterprise Architecture. To implement this core element, the Policy and Planning staff gathers and maintains information about the Department's current state of IT resources and a "to be" target state. The target state aims to improve the current state in ways such as minimizing redundancy of IT services, improving the ability to share information Department-wide and with external stakeholders, and retiring IT assets that are no longer providing optimum service. Enterprise Architecture information, tightly coupled with cost information on IT business investments, helps the CIO make strategic decisions about the direction and evolution of the Department's IT services.
Chief Architect. The CIO Practical Guide and the GAO framework state that an agency should appoint a Chief Architect who is responsible and accountable for the Enterprise Architecture and whose background and qualifications include both the business and technology areas of the organization. Additionally, the Chief Architect is responsible for ensuring the integrity of the Enterprise Architecture development process and for the content of the Enterprise Architecture products.
The Department has a Chief Architect who is the principal advisor to the Department's Chief Technology Officer and CIO on all Department-wide Enterprise Architecture matters. The Department's Chief Architect is responsible for:
Framework, Methodology, and Automated Tool. The Department is developing its own Enterprise Architecture framework and methodology through its Capability Delivery Model. The Department's framework is to be an architecture based on capability areas within the entire Department and its components, with individual capability architectures acting as building blocks that are intended to form a Department-wide Enterprise Architecture. This Department-wide Enterprise Architecture will include both cross-cutting and component-specific capabilities.
An Enterprise Architecture automated tool serves as the storehouse of the architecture products. Architecture products include the current and target architectures and the transition plan. The choice of tool is based on the agency's needs and the size and complexity of the architecture. As stated previously, the Department tested the Popkin automated tool to store its architecture products but is now exploring alternatives to Popkin.
Critical Attribute 3: Demonstrates Satisfaction of Commitment
The completion of the third critical attribute for achieving Stage 2 requires the Department to establish an Enterprise Architecture Program Plan that includes the following core elements:
The Department's Enterprise Architecture Completion and Use Plan completes the three core elements under Critical Attribute 3.
Current and Target Architectures, and Transition Plan. The interagency CIO Council requires that agencies have a written Enterprise Architecture Program Plan. The plan should describe the steps to be taken and the tasks to be performed in managing the Enterprise Architecture program. The plan should also make provision for the development of architectural descriptions of how the organization currently operates (the current architecture), how it intends to operate in the future (the target architecture), and how it will transition from the current to the target environment (the transition plan).
The Department submitted a Department Enterprise Architecture Completion and Use Plan to the OMB in February 2005, and is working on a Department Enterprise Architecture Program Management Plan. The Department's Management Plan will:
Security. The Department has a comprehensive Security Architecture in place that will be aligned with security standards for the Department's overall Enterprise Architecture efforts.
Critical Attribute 4: Verifies Satisfaction of Commitment
The completion of the fourth critical attribute to achieve Stage 2 requires the Department to ensure that the Program Plan calls for developing metrics for measuring Enterprise Architecture progress, quality, compliance, and return on investment. The Department has not implemented this core element.
The measurement of Enterprise Architecture progress, quality, and compliance is necessary to ensure that the Enterprise Architecture meets the targeted milestones and is compliant with necessary regulations. Measuring return on investment would tell the Department what benefits are realized by the development of the Enterprise Architecture in relation to its cost.
Developing Metrics for Measuring Enterprise Architecture Progress. The Department has not yet established metrics for measuring Enterprise Architecture progress, quality, compliance, and return on investment. The Department's Enterprise Architecture Completion and Use Plan states that Enterprise Architecture links performance measures to some portions of the architecture segments. This does not meet the criteria for the fourth attribute.
Stage 3 — Limited Progress
The Department is moving from building the Enterprise Architecture management foundation to developing Enterprise Architecture products for Stage 3. To complete Stage 3, the Department must still: (1) establish an organization policy for the Enterprise Architecture development; (2) ensure that Enterprise Architecture products are under configuration management; (3) ensure that Enterprise Architecture products describe both the current and target environments of the agency; (4) ensure that the business, data, application, and technology descriptions address security; and (5) ensure that progress against Enterprise Architecture plans is measured and reported.
The Department has made limited progress toward attaining Stage 3 maturity of the GAO Enterprise Architecture Management Framework. The Department has made progress on developing a process for developing current, target, and transition architectures. However, the Department lacks a written and approved policy for Enterprise Architecture development, implementation, and maintenance. In addition, the Department must ensure that when completed, all Enterprise Architecture products undergo configuration management and that the Enterprise Architecture addresses security, as stated in the Enterprise Architecture Completion and Use Plan.
Critical Attribute 1: Demonstrate Commitment
To complete the first critical attribute for Stage 3 of the Enterprise Architecture Management Framework, the Department must establish the following core element: develop a written and approved organization policy for the Enterprise Architecture development. The Department has not completed this core element.
According to the Enterprise Architecture Management Framework, an organization policy is an important means for ensuring agency-wide commitment to developing the Enterprise Architecture and for clearly assigning responsibility for doing so. The architecture policy should define the scope of the architecture, including a description of the current and target architecture, as well as a transition plan that supports the move from the current to the target architecture. Additionally, the policy should provide processes for Enterprise Architecture oversight and control, review, and validation. The policy should also address the purpose and value of an Enterprise Architecture, its relationship to the organization's strategic vision and plans, and its relationship to the capital planning process.
The Department has not established a written and approved organization policy for Enterprise Architecture development. As described in Stage 2, the Department established the Enterprise Architecture Program Office with responsibility for developing the Enterprise Architecture. In addition, the Enterprise Architecture Program Management Plan - discussed in Stage 2 - outlines a high-level scope of the architecture, including a description of the planned current and target architecture, as well as the transition plan. The Enterprise Architecture Program Management Plan also addresses Enterprise Architecture oversight, control, review, and validation responsibilities, but in little detail.
Critical Attribute 2: Provides Capability to Meet Commitment
The completion of the second critical attribute for achieving Stage 3 maturity requires the Department to establish the following core element: ensure that Enterprise Architecture products are under configuration management.9 The Department has not yet met this standard.
According to the draft of the Department Enterprise Architecture Program Management Plan, the Enterprise Architecture Program Office will perform configuration management of Enterprise Architecture Products. The Office will also prepare and publish policy to include establishment of necessary configuration committees.
Critical Attribute 3: Demonstrates Satisfaction of Commitment
The completion of the third critical attribute for achieving Stage 3 maturity requires the Department to establish three core elements:
The Department has not implemented core elements 1 and 2. The Department addresses security in its Enterprise Architecture plans; therefore, core element 3 is complete.
Current and Target Architectures, and Transition Plan. According to the Enterprise Architecture Program Management Plan, Enterprise Architecture products will describe the current and target agency environments, as well as the transition plan. As stated earlier, the Department has not completed all components of the Enterprise Architecture. The current, target, and transition processes for the Department are to be identified, approved, and documented by the end of FY 2006. The Enterprise Architecture Program Plan also states that Enterprise Architecture products - current and target architectures and the transition plan - will be described in terms of business, data, application, and technology.
Security. The Department Enterprise Architecture Completion and Use Plan states that Enterprise Architecture will align security standards to the Technical Reference Model.
Critical Attribute 4: Verifies Satisfaction of Commitment
The completion of the fourth critical attribute to achieve Stage 3 maturity requires the Department to establish the following core element: ensure that progress against Enterprise Architecture plans is measured and reported. The Department has not implemented this core element.
As stated in Stage 2, the Department has not established metrics for measuring Enterprise Architecture progress. The measurement of such progress against Enterprise Architecture development plans is necessary to ensure that the development meets targeted milestones.
Stage 4 — to be Completed
Additional work must be completed before the Enterprise Architecture is used as intended in Stage 4 - to drive sound IT investments that are consistent with the Department's goals and missions.
To complete Stage 4, an agency must: (1) establish policy for maintaining the Enterprise Architecture; and (2) complete the Enterprise Architecture, including the current and target architectures along with the transition plan to get from the current to the targeted environments. The completed Enterprise Architecture must be described in terms of business, data, application, and technology; the descriptions must address security and be approved by the agency CIO and the committee or group representing the agency or the investment review board. The Department has not established a policy for maintaining the Enterprise Architecture.
Stage 5 — to be Completed
To reach Stage 5 maturity, an agency must use its Enterprise Architecture to drive IT investments and ensure systems' interoperability. The Department cannot meet Stage 5 requirements of the Enterprise Architecture Management Framework until it completes its Enterprise Architecture.
According to the GAO framework, an organization at Stage 5 maturity has completed the Enterprise Architecture and secured senior leadership approval of it. Further, decision-makers are using the architecture to identify and address ongoing and proposed IT investments that are conflicting, overlapping, not strategically linked, or redundant. Stage 5 agencies are therefore able to avoid unwarranted overlap across investments and ensure maximum system interoperability, which in turn ensures the selection and funding of IT investments with manageable risks and acceptable returns on investment.
The foundation of the Department's Enterprise Architecture lies in its IT infrastructure. The Department's IT Strategic Plan describes the state of the Department's IT infrastructure as follows.
Currently, the Department's infrastructure is largely decentralized, fragmented and outdated. It is essentially an amalgamation of infrastructures designed, developed and maintained by individual components to meet their specific needs. This approach has introduced an unnecessary level of complexity, cost and risk, and inadvertently created technical barriers to sharing information.
The IT Strategic Plan establishes a Strategic Initiative to "develop the Infrastructure architecture layer of the Department's Enterprise Architecture." Specifically, "The Department will work with the components to develop a Department-wide infrastructure architecture - a layer of the Department's overall Enterprise Architecture. The infrastructure architecture will provide a common conceptual framework to support technical interoperability, define a common DOJ vocabulary, and provide a high-level description of the information technology deployed throughout the Department."
A consolidated infrastructure will aid the Capability Architecture effort. The Department is developing the elements of a consolidated infrastructure through a number of pilot programs. One example is the Public Key Infrastructure (PKI), created to resolve the Department's computer security concerns. PKI is intended to implement an IT security program as well as complete the design, development, and implementation of a secure and trusted IT environment.
The Infrastructure Architect is the person responsible for consolidating the Department's IT infrastructure. The Infrastructure Architect has described the following four critical elements of a consolidated infrastructure.
The Infrastructure Architect foresees cost savings, economies of scale in IT acquisitions, and enhanced enforcement of security and management of IT performance as benefits resulting from this consolidation. At the time of our field work, a draft Consolidated Infrastructure plan was nearing completion.
Completion of a clear and comprehensive Department Enterprise Architecture will require a collaborative effort between the Department and the major Department components. The two-tiered architecture envisioned by the Department will require components to contribute Enterprise Architectures that encompass those component-specific IT systems that are not included in the Department's cross-cutting Capability Architectures. Some components, such as the FBI and the DEA, have made progress in developing their component-level Enterprise Architectures. Others, such as JMD, have not. In JMD's case, efforts begun in 2003 to develop a component-level Enterprise Architecture were held in abeyance as work began on the higher-priority Department-level Enterprise Architecture.
The Department's FY 2004 Report on Information Technology identifies funds budgeted for Enterprise Architecture and related planning. The table provides a 1-year snapshot of money budgeted for Enterprise Architecture efforts.
FY 2004 Component Enterprise Architecture (EA) Budgets
In 2001, the Department requested that components submit their ITIM processes for review. However, the Department did not make a similar request for Enterprise Architectures. The Department also issued guidance, based on a Technical Reference Model, to develop a high-level Enterprise Architecture for the Department. A Department official stated that the only guidance provided to the components on Enterprise Architecture was through the Technical Reference Model and the Enterprise Architecture Committee (discussed in the Department Enterprise Architecture section of this report). Also, the Department did not track the development of components' Enterprise Architectures, validate Enterprise Architectures developed, or ensure that Enterprise Architectures were kept current.
According to the CIO, the Department conducts little oversight of component Enterprise Architectures. The CIO described a "broad brush" programmatic approach to the oversight of component Enterprise Architectures, which includes establishing standards and Enterprise Architecture tools, developing work plans for a Department-wide Enterprise Architecture, and establishing management of component-level Enterprise Architectures. However, as of June 2005, none of these efforts had been completed. At the same time, according to the CIO, the Department takes a "deep dive" approach in overseeing the components with selected Enterprise Architecture capability areas. According to the Chief Technology Officer (CTO), as discussed earlier the capability areas cross-cut multiple components and include Financial Management, Law Enforcement Information Sharing, Case Management, and the Justice Consolidated Office Network (JCON). The CTO also said there are several E-government-related architecture efforts in progress at the federal level for which the Department is either the managing partner or is an active participant. Therefore, these select few architectures merit more intensive Department oversight.
The CTO stated that components should develop project-specific architectures as necessary for projects that are a priority to the component, because these projects may not be included in the Department Enterprise Architecture. For architecture projects currently identified as common solutions or E-government projects, the Department Enterprise Architecture will provide guidance to the Enterprise Architecture program teams as necessary. The CTO stated that even though the Department is already informally involved to varying degrees with some components' architecture efforts, it is in the process of establishing a formal Department Enterprise Architecture governance structure. The Department Enterprise Architecture program document will provide guidance to the components and program managers of other cross-component architecture efforts at the same time. According to the CIO, some of the common solution projects are underway and need a lesser degree of involvement from the Department's Enterprise Architecture team. For architecture efforts that are identified as common, multi-component solutions in the future, the Department Enterprise Architecture will take the lead in developing the Enterprise Architecture teams and play a greater role in developing the architecture. All architecture efforts within the Department are to map their Enterprise Architecture to meet OMB, FEAF, and GAO guidance.
An organization without a completed Enterprise Architecture assumes the risk that it will invest in IT that is duplicative, not well integrated, costly, or not supportive of the agency's mission. Until a Department-wide Enterprise Architecture is completed, the Department faces such risks. Once the Enterprise Architecture is completed, the risks will be reduced and the Department will have a more realistic vision of its future IT requirements.
The current effort to develop a Department-wide, capability-based Enterprise Architecture for systems that cross-cut two or more components is in an early stage. Instead of being based on the generally accepted FEAF, this Enterprise Architecture will be based on the Department's self-created framework: the Capability Delivery Model. We believe it is too soon in the development of the model to determine if it will contain all the necessary elements of an Enterprise Architecture. It is also too soon to determine if, when fully developed, the model will result in an Enterprise Architecture that conforms to GAO, FEAF, and OMB guidance.
The Department believes that the most efficient approach to its Enterprise Architecture is to focus its efforts on major, cross-cutting IT systems with individual architectures for groups of systems such as case management systems. Component-level projects are expected to be covered by component-level Enterprise Architectures, which together with the Capability Architectures are to form the Department Enterprise Architecture. The Capability Delivery Model approach focuses on the high-visibility and high-cost cross-cutting IT projects. We believe that focusing management attention on high-risk projects is a prudent approach. However, a successful Enterprise Architecture should present a clear and comprehensive view of an organization, and the Department must take care to avoid a disjointed, fragmented, or incomplete Enterprise Architecture. Our audit found a lack of consistent coordination between the Department and component Enterprise Architecture efforts, which increases the risk that the Department's two-tiered approach could result in gaps within the Department's Enterprise Architecture.
In the course of conducting this audit, and in reviewing previous audits of DEA and FBI IT management, we found that the Department's oversight of component Enterprise Architecture efforts in general continues to be inconsistent. Components have been developing Enterprise Architectures for several years at considerable cost without ongoing and substantive Department-level guidance or monitoring. The Department is not currently providing direction to ensure that components' Enterprise Architecture efforts are consistent with, and will meet the needs of, the overall Department Enterprise Architecture under development. The Department's two-tiered approach to Enterprise Architecture will require all major components responsible for IT systems to develop Enterprise Architectures in order for the overall Department Architecture to present a clear and comprehensive view of the Department's IT environment. However, the components have generally been working on their Enterprise Architectures independently, without specific guidance or monitoring to ensure full compatibility with the Department-level Enterprise Architecture when it is developed.
The Department has begun to improve its oversight and guidance of components' Enterprise Architecture efforts. For example, an Enterprise Architecture Program Management Plan, completed in June 2005, discusses the Department's Enterprise Architecture organization, interaction between the components and the Department, the need for a Department-wide Enterprise Architecture tool, and components' use of the FEAF. However, more progress is needed, and we provide the following recommendations for the Department.
We recommend that JMD:
A key objective of the Clinger-Cohen Act is to ensure that agencies implement processes for maximizing the value of IT investments and for assessing and managing the risks of IT acquisitions. To accomplish this objective, agencies must establish processes to ensure that IT projects are being implemented at acceptable costs and within reasonable timeframes, and that the projects are contributing to tangible, observable improvements in mission performance. Additionally, OMB Circular A-130 requires each federal agency to establish and maintain a capital planning and investment control process for IT. The Department is in the early stages of developing Department-wide ITIM processes.
The Department and its components made various attempts to develop ITIM policies and procedures under Clinger-Cohen beginning in 2001, but progress has been slow. In October 2004, the Department issued a framework for developing ITIM processes, called IT Strategic Management (ITSM). The purpose of the ITSM framework is to:
Once the processes created through the ITSM framework are fully developed, the Department expects that the components' ITIM processes and functions will be integrated within the Department's overall ITIM structure. The Department-level ITIM will then support all components regardless of size, funding, or resources. In order to achieve this objective and ensure coverage of all IT projects within the Department, components that have or are developing ITIM processes will be required to incorporate the Department's ITSM framework into their own frameworks.
The following tables summarize Clinger-Cohen and OMB A-130 requirements and how the ITSM framework is to meet them.
To assist organizations with developing ITIM processes, the GAO developed ITIM: A Framework for Assessing and Improving Process Maturity, which provides a method for evaluating how well an agency is selecting and managing its IT resources. This framework is built around the select/control/evaluate approach described in Clinger-Cohen. The most current version, issued in 2004, is a maturity model composed of five progressive stages.11 Appendix 4 outlines the GAO framework. We intended to use the GAO ITIM framework to evaluate the status of the Department's ITIM but did not do so because of the Department's limited progress in establishing its ITIM processes. Instead, we examined the Department's ITSM framework to determine whether it would allow for the development of effective ITIM processes.
Since 2002, the Department has worked on various policies and procedures related to developing a Department-wide ITIM. In addition to the Department's ITSM framework, the Department created a web-based "Dashboard" tool to show IT investment information and status, an IT Strategic Plan to set IT strategic goals, and a Department Executive Review Board (DERB) Charter to provide oversight for components' IT investments. A discussion of the ITSM framework and the other initiatives follows.
Department IT Strategic Management Framework
The Department ITSM Framework is designed to establish a Department-wide ITIM process in three phases: IT Planning, IT Funding and Architecture, and IT Investment Oversight.
IT Planning Phase
The Strategic Planning Process, part of the IT Planning Phase, identifies the long range goals, objectives, strategies, and measures of project success. IT strategic planning occurs early in the ITIM process planning years and results in an IT strategic plan. The Department's IT Strategic Plan was first created in 2002 and revised in 2005. The plan contains priorities that drive management and investment decisions for the remainder of the strategic management cycle. Strategic planning also considers business priorities, information use and management requirements, technology integration, and other strategies to migrate the current IT structure to the target structure. Furthermore, the Strategic Planning Process assesses the current state of IT programs, identifies mission requirements for IT, and defines the goals to be pursued and the measures to be used to monitor progress. This strategy guides the development of the Department IT portfolio and component investment plans.
Prior to FY 2004, investments were not selected or funded in consideration of developing a cohesive Department IT portfolio. Instead, the Department reviewed component concept proposals and budget requests to ensure alignment with the 2002 IT Strategic Plan and IT Guiding Principles.13 The principles were to be applied Department-wide.
In FY 2004, the Department began implementing the ITSM framework's IT Planning Phase for the FY 2005 budget. Each component met to discuss the Department's IT needs that were to be integrated into component IT systems and concept proposals. Since 2004, review meetings have been held with components proposing major IT investments. While no new Department-wide IT Strategic Planning policies or procedures were issued in FY 2004, the Department took the following actions for Strategic Planning:
The Department completed its IT Investment Plan in May 2005 and is currently updating the plan for submission to the OMB for the next budget cycle. In FY 2006 and beyond, the Department plans to work on processes and products related to the Strategic, Transition, and Investment plans and develop a Human Capital Plan.
IT Strategic Planning Process
The Department has recognized the need to focus more attention on IT management and information sharing. As a result, the Department has decided to take a more proactive role in matching technology to identified business needs. Instead of a decentralized approach whereby only the Department components develop ITIM processes, the Department wants to develop a more centralized approach to IT management by developing Department-level ITIM processes. This approach requires components without an ITIM system to use the Department's ITSM framework, while components with established ITIM processes will need to integrate the Department's ITIM processes with their own. The plan's three main goals are to provide: (1) information sharing among all components, (2) a reliable and cost-effective infrastructure to conduct Department-wide electronic business, and (3) management processes and policies to support and improve the Department's IT performance and continuity.
The Department's IT Strategic Plan is based on the 2003 IT Strategic Planning Guide. The strategic goals listed in the IT Strategic Plan include the following.
Investment Planning Process
The Investment Planning Process identifies specific investments needed to achieve the strategic priorities of the Department consistent with the IT Strategic Plan, and seeks to create an investment plan that balances business priorities and funding resources. Using the IT Strategic Plan and the investment plans from portfolio managers and the components, IT planners and business leaders prioritize needed investments. The Department IT Investment Plan is the result of this investment planning. The Investment Plan identifies the recommended IT investments to support the IT Strategic Plan and the investment performance measures that define the expected business results. The Investment Planning Process provides a method for converting the strategic goals and objectives defined by the IT Strategic Plan into a set of prioritized investments for the future.
For the Investment Planning Process, the Department developed a draft investment process guide, an investment plan with performance metrics, portfolio strategies, and a Transition Planning Process Guide. Additionally, IT questionnaires and surveys from component CIOs were collected to determine human capital needs. The development of the Human Capital Plan is ongoing and involves performing the analysis, planning, and organizational transitions needed to staff and manage IT investment portfolios and approved projects. Additionally, the skills and staffing needed to implement the IT initiatives to be funded are assessed, and the actions required to budget for, reassign, acquire, develop, and retain human resources are performed. To date, the Department appears to be making progress toward completing the Investment Planning Process.
IT Funding and Architecture Phase
The IT Funding and Architecture Phase of the Department's ITSM framework consists of ongoing processes that establish the budget and architectures to be used by the Department and its components in developing, operating, or terminating IT projects. Funding for IT projects follows the same process the Department uses to obtain funding for all other functions: the Budget Submission Process. This process converts the IT Investment Plan into a fully documented and properly formatted IT budget request ready to be combined with the Department's full budget request. This involves the development of investment business cases and other documentation from Department staffs, components, and other sources, and the integration of these individual investments into a unified portfolio for review by the Department's leadership and submission to OMB.
The subsequent pass-back from OMB and Spend Plan Process, as part of the IT Funding and Architecture Phase, leads proposed investments through the budget process to become incorporated into the enterprise portfolio. This occurs through three steps: (1) the OMB pass-back, which provides the Department with initial OMB budget decisions; (2) the submission of the final Department budget, which is then incorporated into the fiscal year budget by the OMB; and (3) the revision of the IT candidate portfolio and preparation of spending plans after the fiscal year budget is enacted. Once funded, the candidate investments are moved to the enterprise portfolio for investment management. The enterprise portfolio contains all of the funded IT investments for the Department.
In the Capability Architectures process, project managers work toward converting the strategies defined in the strategic planning process into an overall Enterprise Architecture. Capability architectures are used by the project managers to drive the development of solution architectures for investment projects. These capability architectures, which focus on providing Enterprise Architecture capabilities for Department-wide support of business needs, are also used to review solution architectures to ensure compliance with initial conceptual architectures.
According to Department officials, the objective for Enterprise Architecture efforts in FY 2003 and prior years was to build a foundation to develop a mature Enterprise Architecture. Business, System, and Data Architectures were developed along with an Enterprise Architecture Management Systems Tool. For the FY 2004 budget, requests for investment and project funding were submitted to the Department by project managers. The Attorney General's IT Budget Guidance, which is a memorandum initiating the annual Department budget process, was also developed for funding the Department's IT projects. For FY 2005, an integrated budget submission process was developed and, according to Department officials, this process allowed the Department to work closely with the JMD Budget Staff to integrate Department IT needs into the budget. For FY 2006 and future years, the Department intends to institutionalize an integrated budget submission process. However, the actual processes and policies have not yet been determined. Additionally, a Budget Submission Guide and a performance measurement document are still needed to complete the IT Funding and Architecture Phase.
IT Oversight Phase
As mandated by Clinger-Cohen, each agency head must establish ITIM processes and provide oversight by determining: (1) which employees should perform certain IT management functions; (2) if certain IT functions should be contracted to outside sources; (3) which IT missions, processes, and administrative practices must be revised to support each other in making significant investments; and (4) if the information security policies, procedures, and practices are adequate.
In complying with the oversight responsibilities outlined in Clinger-Cohen, Department Order 2880.1A stated that the Department's CIO is responsible for:
In response to Order 2880.1A, issued in March 2001, 29 of the 34 major components required to submit ITIM processes to the Department complied by September 2002.14 Five components did not submit an ITIM process for approval, while another five components that were not required to submit a process submitted one. None of the ITIM processes were fully developed. According to a Department IT official, the five components that did not submit ITIM processes were small and did not have significant IT investments. Initially, the Department tracked whether the components submitted ITIM processes and whether the processes were approved by the Department's CIO. The Department responded to the components, stating that their ITIM processes would be evaluated and either accepted or rejected. The Department then provided suggestions for improving their processes. In addition, the Department surveyed the components in May 2003, nearly 1 year after the ITIM processes were submitted, to determine how the components had progressed. According to the Department CIO, the components were having a difficult time developing their ITIM processes and progress was slow. In JMD's case, its efforts begun in 2002 to develop component-level ITIM processes were abandoned in 2004 as it focused on developing the ITSM framework for the Department's overall ITIM effort. The Department is planning to issue a revised version of Order 2880.1A which is expected to better outline component responsibility as well as the Department's oversight role. The Department does not have an estimated date for issuance of the revised version.
For the components considered by the Department to be so small that it would not be beneficial to spend time developing a component-based ITIM process, yet they have IT systems necessitating an ITIM process, the Department developed what it refers to as an "ITIM-lite" process to facilitate decision making throughout the life cycle of an IT project. The purpose of ITIM-lite was to allow management to:
The Department abandoned ITIM-lite and in 2004 began developing its Department-wide ITIM processes, which are expected to encompass the smaller components.
The Department has not recently been overseeing the development of components' ITIM processes. Instead, the Department decided to concentrate its oversight attention on components' actual investments. The one exception, according to Department officials, is the tracking of the FBI's development of ITIM processes because the FBI accounts for about 50 percent of the Department's IT budget. Oversight of the development of other components' ITIM processes was abandoned in 2002. The Oversight Phase in the ITSM Framework, as discussed below, involves monitoring components' IT projects rather than overseeing or approving components' ITIM processes or the development of the processes. According to the Department CIO, oversight of the components' ITIM processes is currently performed on an ad hoc basis.
According to the ITSM framework, project oversight will occur during the operational years of IT projects and will be divided into three tiers: the Department Dashboard Process (Tier 1), the Project Oversight Process (Tier 2), and the Executive Oversight Process (Tier 3).
The Department Dashboard (Tier 1) is a query tool that provides users with the ability to access a database of Department components' IT systems using a web browser interface. The Dashboard is designed to provide the Department, component CIOs, and project managers with a "quick reference" on the current cost, schedule, performance, and risks for major or highly visible component investment projects that are in the Department's IT portfolio. Projects are identified as being in a state of completion, planning, operation, or on hold to be reviewed by the Department CIO. The Department Dashboard gives component project managers and reviewers access to IT project data. Data in the Dashboard includes project cost, schedule performance, and risks. The Dashboard is accessible through the Department Intranet.
Project managers record the risks, milestones, and costs of projects into the Dashboard. Based on the risks associated with the project, the project manager rates the status of the project as red, yellow, or green. Issues regarding excessive cost or funding shortfalls are rated red. Issues with the potential to have excessive costs or funding shortfalls are rated with a yellow status. If there are no excessive cost issues, projects are rated green. Department officials then review the project information in the Dashboard, paying special attention to projects designated as red or yellow. Project managers are required to update the status of their projects by the 10th business day of each month. However, the project managers can bring a project to the attention of the Department CIO at any time. The Dashboard flags any changes made in baseline data and then displays the project with a red flag. A Department Dashboard official can then follow up with inquiries to the project managers. The Dashboard categorizes projects by component. Once the Dashboard is reviewed by the component CIO and the Dashboard Policy Advisor, the Department CIO reviews it. The CIO then holds meetings to discuss the status of projects. Currently all components, with the exception of the FBI, are connected to the Dashboard, which covers approximately 80 investments. Project managers are involved in the process through training sessions, a user guide, and one-on-one meetings as necessary.
The Project Oversight Process (Tier 2) will consist of approximately 12 to 15 projects that are selected from those in the first tier for review and face-to-face meetings with project managers to make sure the projects are meeting their expected performance. The projects selected for this tier are those that may be high-risk, over budget, politically sensitive, or otherwise demand closer scrutiny. A Department official explained that this is the level at which members of the IT and Policy and Planning Staff in the Department will become directly involved.
In the third tier, the Executive Oversight Process, approximately six projects will be selected from Tier 2, based on Department or congressional priorities, for evaluation from an investment, business, and return-on-investment perspective. This process is carried out by the Department Executive Review Board (DERB) assembled at the level of the CIO, Controller, and Deputy Attorney General. This process began as a pilot program, but its scope is now being expanded to include the Department's entire portfolio.
According to the GAO's ITIM framework, instituting IT investment boards is a key component in the IT investment management process because the boards define the membership, guiding policies, operations, roles, responsibilities, and authorities for each designated board and, if appropriate, each board's support staff. Prior to the establishment of the DERB in 2004, various committees and boards were formed to facilitate the sharing of information between the Department and its components, including the following.
By the end of FY 2003, all but the CIO Council were disbanded and replaced with the DERB, which is now responsible for reviewing the major IT investments of all components.
A DERB official explained that investments are selected for review based on an investment's budget or the mission-critical nature of the investment. In terms of budget, two types of projects will be reviewed: projects that are to run for more than 10 years and funded at more than $20 million, or short-term projects running for about 1 year with a budget of at least $15 million. Projects considered to be mission-critical or strategically important for the Department are reviewed, even though they may not be costly, because of the high risk involved with meeting the Department's mission. The DERB's Department-level oversight occurs in meetings where members discuss the investments as well as the planning, budget, risk, and assessment of current component projects. The first official DERB meeting was in November 2004 and since its inception, the DERB has met approximately five times. We found that while the DERB contributes to the cohesive nature of the ITSM framework, it is neither as comprehensive in its functions nor as capable of devoting sufficient time to individual projects as the disbanded boards that were designed to be tailored to specific IT resources.
The Clinger-Cohen Act and OMB Circular A-130 require agencies to ensure that IT investments are made with an overall focus on the agency's mission and with senior management oversight. When ITIM processes using a select, control, and evaluate methodology are performed properly, the agency should reduce the risk and maximize the benefits from IT investments.
In March 2001, the Department issued Order 2880.1A, which required its components to implement ITIM methodologies and submit these methodologies for review by the Department CIO. The Department also provided guidance for developing ITIM processes. The Department planned to rely on the components' submissions to meet ITIM requirements. While most components submitted ITIM plans, progress was slow to implement ITIM processes. This strategy did not provide the components with a clear vision of how they should create their ITIM processes to meet the overall mission of the Department, as the Department did not have a fully developed IT Strategic Plan or Enterprise Architecture that would outline the overall mission of the Department and identify the IT investments that should be made to achieve that mission.
In 2004 the Department developed the ITSM framework, which was designed to lead to Department-level ITIM processes. In our judgment, the ITSM framework can result in fully mature ITIM processes if carried out properly. The Department's ITSM Framework includes the funding, oversight, and planning requirements outlined in the Clinger-Cohen Act. The Department has made some progress in implementing the processes outlined in its ITSM. Still, not all of the processes have been implemented. For example, the Investment Planning Guide and enterprise portfolio are two key elements of the ITSM that are not yet fully implemented. Without these elements, the Department cannot provide components with a complete picture of what investments should be pursued. Additionally, it is not clear that all of the Department's IT investments will be adequately covered by the ITSM.
We believe that if the Department's ITSM is successfully implemented, mature ITIM processes will result. However, at this early stage it is difficult to assess whether all the components will have developed compatible ITIM processes or be covered adequately by the Department's ITIM processes. Major components, such as the DEA and FBI, are well ahead of the Department and its ITSM development.
The ultimate success of the Department's current efforts to develop mature ITIM processes is difficult to evaluate at this early stage. The effort is likely to take years, and the Department has no firm schedule for developing its ITIM processes or ensuring the development of compatible component-level ITIM processes. In the meantime, the Department risks investing in or maintaining systems that are duplicative or may need to be replaced, altered, or eliminated if they do not align with the mission and the goals of the Department.
We recommend that JMD: