OIG Audit Report 07-40

Sentinel Audit III: Status of the Federal Bureau of Investigation’s
Case Management System (Redacted - Public Version)

The FBI’s IT Systems Life Cycle Management Directive (LCMD) is comprised of interrelated components that include Life Cycle Phases, Control Gate Reviews and Boards, and Project Level Reviews. Because Sentinel has multiple phases, it will pass many of the life cycle phases, control gate reviews, and project level reviews multiple times.

Phases

The LCMD has established nine phases that occur during the development, implementation, and retirement of IT projects. During these phases, specific requirements must be met for the project to obtain the necessary FBI management approvals to proceed to the next phase.

Control Gate Reviews & Boards

The approvals to proceed from one phase to the next occur through seven control gates, where management boards meet to discuss and approve or disapprove a project’s progression to future phases of development and implementation. The seven control-gate reviews provide management control and direction, decision-making, coordination, confirmation of successful performance of activities, and determination of a system’s readiness to proceed to the next life cycle phase.

Project-level Reviews

Project-level Reviews support the IT Systems Life Cycle process. Project Level Reviews determine program or project readiness to proceed to the next activities of the project life cycle. Each Project Level Review feeds information up to the Executive-level Control Gates, as data is developed and milestones are completed.

FBI LCMD PHASES

PHASE NAME	DESCRIPTION
Concept Exploration	Identifies the mission need, develops and evaluates alternate solutions, and develops the business plan.
Requirements Development	Defines the operational, technical and test requirements, and initiates project planning.
Acquisition Planning	Allocates the requirements among the development segments, researches and applies lessons learned from previous projects, identifies potential product and service providers, and identifies funding.
Source Selection	Solicits and evaluates proposals and selects the product and service providers.
Design	Creates detailed designs for system components, products, and interfaces; establishes testing procedures for a system’s individual components and products and for the testing of the entire system once completed.
Development and Test	Produces and tests all system components, assembles and tests all products, and plans for system testing.
Implementation and Integration	Executes functional, interface, system, and integration testing; provides user training; and accepts and transitions the product to operations.
Operations and Maintenance	Maintains and supports the product, and manages and implements necessary modifications.
Disposal	Shuts down the system operations and arranges for the orderly disposition of system assets.

FBI LCMD CONTROL GATE REVIEWS

GATE	DESCRIPTION
Gate 1	System Concept Review approves the recommended system concept of operations and occurs at the end of Phase 1 of LCMD.
Gate 2	Acquisition Plan Review approves the Systems Specification and Interface Control documents as developed in Phase 2 and the approach and resources required to acquire the system as defined in the Acquisition Plan as developed in Phase 3.
Gate 3	Final Design Review approves the build-to and code-to documentation and associated draft verification procedures. It also ensures that the design presented can be produced and will meet its design-to specification at verification. The gate review occurs after the contractor is selected in Phase 4 and system design is completed in Phase 5.
Gate 4	Deployment Readiness Review approves the readiness of the system for deployment in the operational environment. The gate review occurs after the system is developed and tested in Phase 6. Approval through Gate 4 signifies readiness for system implementation.
Gate 5	System Test Readiness Review verifies readiness to perform an official system-wide data gathering verification test for either qualification or acceptance. The gate review occurs mid-way through Phase 7.
Gate 6	Operational Acceptance Review approves overall system and product validation by obtaining customer acceptance and determining whether the operations and maintenance organization agrees to, and has the ability to, support continuous operations of the system. The gate review occurs at the end of Phase 7.
Gate 7	Disposal Review authorizes termination of the Operations and Maintenance life cycle phase and disposes of system resources. The gate review occurs at the end of Phase 8 and results in Phase 9.

EXECUTIVE REVIEW BOARDS RESPONSIBLE
FOR CONTROL GATE REVIEWS

New FBI Process for Overseeing IT Projects

In November 2006, a new FBI IT governance secretariat began operations. The governance secretariat established several working groups to assess an IT project each time it requests approval to pass through an LCMD gate. Based on the need for varying expertise, the role of each working group varies according to the LCMD gate, but the entire process requires input from the following working groups: the Investment Project Review Working Group, Technical Review Working Group, Enterprise Architecture Working Group, and the Configuration Management Quality Assurance Working Group.

Assessments Under New Governance Process

As Sentinel approaches an LCMD gate, the Sentinel PMO works with the working group responsible for doing assessments for that gate. LCMD control gate documentation is normally submitted 3 weeks in advance of the final assessment for review.

The cognizant working group has 3 days to provide a preliminary assessment of the documentation. To save resources and time, the FBI will cancel the formal gate review if the working group discovers significant issues during the preliminary assessment. If a project’s manager disagrees with the working group’s preliminary assessment, the Chief Technology Officer makes a determination.

If a project passes the preliminary assessment, the working groups have 10 days to conduct a full assessment. The executive summaries of the working groups are compiled along with conditions necessary for the project to clear the gate, and a formal gate review meeting is conducted, during which one of the following four FBI IT Decision Board decides whether the project should clear the gate.

The Investment Management Board oversees the System Concept Review (Control Gate 1).
The Project Review Board oversees the Acquisition Plan Review (Control Gate 2) and the Disposal Review (Control Gate 7).
The Technical Development and Deployment Board oversees the Final Design Review (Control Gate 3), the Deployment Readiness Review (Control Gate 4), the System Test Readiness Review (Control Gate 5), and the Operational Acceptance Review (Control Gate 6).

Previous FBI Process for Overseeing IT Projects

The FBI’s previous IT governance system did not require working group assessments of a project’s documentation at each LCMD control gates. However, under the old system, the Technical Review Board was required to review the project at Gate 3, the Final Design Review.

The IMPRB leads the System Concept Review and the Acquisition Plan Review (Control Gates 1 and 2) and ensures that all IT acquisitions are aligned and comply with FBI policies, strategic plans, and investment management requirements.
The Technical Review Board leads the Final Design Review (Control Gate 3) and ensures that IT systems comply with technical requirements and meet FBI needs.
The Change Management Board leads the Deployment Readiness Review, System Test Readiness Review, Operational Acceptance Review and the Disposal Review (Control Gates 4 through 7) and controls and manages developmental and operational efforts that change the FBI's operational IT environment.
The Enterprise Architecture Board ensures that IT systems comply with Enterprise Architecture requirements.
The IT Policy Review Board establishes, coordinates, maintains and oversees implementation of IT policies.

PROJECT LEVEL REVIEWS:
CONCEPT EXPLORATION PHASE THROUGH DESIGN PHASE

REVIEW NAME	DESCRIPTION
Mission Needs Review	Examines the user need or technological opportunity, the deficiencies in the current set of systems, alternative and the proposed solution, and a business case or rationale for further investigating changes to the FBI’s information systems.
System Specification Review	The decision point to proceed with the development of an Acquisition Plan, the allocation of high level system requirements to segment specifications, and the development of Project Plans that will manage the acquisition.
Source Selection Acquisition Review	Approves source selection results and authorizes contract negotiations.
Contract Implementation Review	The first review between the customer and the solution provider following a contract award.
Requirements Clarification Review	Ensures the solution provider has a full understanding of the requirements for the system or segment and can articulate this understanding through proposed implementations of the requirement.
Design Concept Review	Technical review of the decomposition of the system or product (hardware, software, and manual operations).
Preliminary Design Review	Can be a single event or spaced out over time during the Design Phase to cover logical groupings of configuration items. The review proves that the concept and the specification for the concept are feasible and will satisfy higher level requirements allocated to it, and to approve the preliminary design-to specifications and associated verification plans. All hardware, software, support equipment, facilities, personnel, and tooling should be reviewed in descending order of system to assembly.
Critical Design Review	Approves the build-to and code-to documentation and associated draft verification procedures, to ensure that the design presented can be produced, and that when built is expected to meet its design-to specification at verification.
Product Test Readiness Review	Series of technical reviews at which the customer concurs that the solution provider is ready to conduct official "sell-off" tests during which official verification data will be produced.
Site Test Readiness Review	Technical review at which the customer concurs that the supplier is ready to conduct official "sell-off" tests during which official verification data will be produced.
Site Acceptance Review	Technical review where customer organization accepts the system or segment delivered to the site.
Operational Readiness Review	Technical review between the Project Office and the product user to verify readiness for system validation required by the Operational Readiness Plan developed in compliance with the Mission Requirements Concept of Operations Document at the outset of the project.
Operational Acceptance Test	Tests the operational capability of the system from a deployed user perspective. Becomes the basis for government acceptance of the Phase 1 product.
Deployment Acceptance Review	Provides the final approval ("go-ahead") to deploy the Phase 1 system.

FBI'S LCMD IT Systems Life Cycle

[Image Not Available Electronically]