OIG Audit Report 07-40

Sentinel Audit III: Status of the Federal Bureau of Investigation’s
Case Management System (Redacted - Public Version)

Rank	Risk Condition	Risk Consequence	Impact Phase	Mitigation Strategy
1	Losing key contractor workforce members that are overworked.	Add significant delays due to retraining the contractor staff.	1	M1. Bring to Lockheed Martin’s Risk Opportunity Management Board as a risk and ask for their assessment and plans to mitigate (e.g., cross-training, documented procedures, mentoring). M2. Identify critical positions (ask PMO on-site staff and managers to id high performers and critical positions), and trigger level at which risk is raised. M3. Ask Lockheed Martin to provide backup plan for high performers and critical positions (by name) identified in M2. M4. Identify industry norms for turnover and monitor monthly for larger than expected departures. M5. Plan for activities to help morale – high level recognition and awards, plan for completion celebration. M6. Implement policies that min. weekend/late night work: Do not schedule “due dates” for deliverables on Mondays or Fridays; Analyze schedule and demand realistic planning; Call for “stand downs” that concentrate work on key attributes (e.g.,.quality, security) and permit people to take day off or have light work days. M7. Collect information on how a different agency makes use of uncleared and SECRET cleared people on an information system project.
2⁶²
3	Source of authoritative attributes are not known or available by end of Phase 2 planning.	Insufficient information available for LMSI to scope / resource the effort will impact cost and schedule.	2	M1 Recognition and re-characterization of the risk by PMO. M2 Extract and “re-use” relevant information available from latest APG Report. M3 PMO – Identify and validate ROLES; then gain approval by BPR/APG. M4 PMO – Identify and validate RULES; then gain approval by BPR/APG. M5 Identify and validate LABELS; then gain approval by BPR/APG. M6 EDS – ID and validate SOURCES. M7 System Development Unit coordinate with EDS via an electronic communication to confirm responsibility of the design, installation, and population of the Bureau Personnel Management System. M8 Address as part of requirements decomposition process.
4	Cleansing of data from phased-out legacy systems may have been under estimated.	1. Requires government-furnished Data Staging partition by 11/1/06 (In FBI facility with certification and accreditation complete and Oracle 10G with real application clusters installed. 2. Placing cleansed data back into the legacy data base may impact those continuing to use legacy applications. 3. Need to maintain security control of data in staging area (Data will not be protected by ACS or Sentinel access controls. 4. Data cleansing is a Phase 2 risk mitigation activity and should not delay Phase 1 critical path activities.	2	Consequence 1 M1 Use new staging or SIT hardware to perform data cleansing. Delay data cleansing until receipt of hardware. Consequence 2 M1 Data migration alternative trade studies. Consequence 3 M1 Cleansing to be done only in FBI facility. M2 Access limited to select group read into “process”. FBI only? Consequence 4 M1 Remove dependencies in schedule between data cleansing and the design concept review, the preliminary design review, and the critical design review.
5	Data migration from phased-out legacy systems may have been under estimated.	Some data may be lost or compromised, or ACS may not be able to be replaced.	2	M1 Identify all required data elements. M2 Develop mapping of ACS elements to Sentinel data requirements. M3 Develop migration plan to support data conversion to new environment. M4 Develop test plan to validate migration strategy. M5 Ensure management funds adequate to provide analysis if required. M6 Work with ITOD to determine scope of effort. M7 Review results of previous data cleansing efforts for issues provide lessons learned to Lockheed Martin. M8 Ensure system design provides from migration. M9 Integration of data, design, and migration integrated product teams. M10 Establish the Data Quality Board at FBI headquarters.
Rank	Risk Condition	Risk Consequence	Impact Phase	Mitigation Strategy
6	Sentinel interfacing with legacy systems may have been under estimated.	1. Some data may be lost or compromised, or ACS may not be able to replace (Universal Index is the highest risk). 2. Current approach (Dynamic Extract vs. Backup Tapes) requires significant effort in Clarksburg. 3. Placing cleansed data back into the legacy database may impact those continuing to use legacy applications. 4. Need to maintain security control of data in staging area (Data will not be protected by ACS or Sentinel access controls).	2	Consequence 1 - M1 Identify all required data elements. M2 Develop mapping of ACS elements to Sentinel data requirements. M3 Develop migration plan to support data conversion to new environment. M4 Establish working relation with owners of legacy data. M5 Data Migration Team to participate in Phase 2 Design Process. M6 Perform multiple dry runs of migration process. M7 Develop test plan to validate migration strategy. M8 Perform validation testing. M9 Establish FBI Data Migration Governance Board/Working Group. M10 Work with ITOD to determine scope of effort. M11 Review results of previous data cleansing efforts for issues provide lessons learned to Lockheed Martin. M12 Provide support for incremental transition. Consequence 2 - M1 Complete Phase 1 benchmarking to validate approaches. Consequence 3 - M1 Data Migration Trade Study and Data Migration Plan agree data will not be put directly back into ACS. Quality Assessment Reports will allow cleansing in ACS by ITOD and data re-exported to staging environment. Multiple dry runs to enhance data. Consequence 4 - M1 Cleansing to be done only in FBI facility. M2 Access limited to select group read into “process”. FBI only?
7	Sentinel requirements do not include conformance to recently developed Information Sharing standards from major stakeholders/intelligence community partners.	The Sentinel information sharing concepts / designs / implementations may be incompatible with stakeholders and intelligence community partners’ standards. Translation (or harmonization) or compliance with these standards may require significant increases to cost and schedule.	2	M1 Participate in FBI Information Sharing Policy Board (ISPB). M2 Participate in the Information Sharing Council (ISC) sponsored by the Office of the Director of National Intelligence. M3 Participate in the Department of Justice’s Law Enforcement Information Sharing Program. M4 The PMO and LMSI will work together with the Office of the Chief Information Officer to technically evaluate existing standards from DOJ (National Information Exchange Model), the Office of the Director of National Intelligence (Common Information Sharing Standard), and others (as applicable). M5 The PMO and the Office of the Chief Information Officer will develop a resource (cost, schedule, skills, etc.) estimate and a gap analysis between Sentinel requirements and the standards for each technical choice. M6 The PMO will choose a standard and socialize the choice through FBI/DOJ chain of command. M7 The PMO will obtain funding and issue a contract modification to Lockheed Martin for Phases 2-4 to include chosen standard in design, implementation, and operations and maintenance efforts. M8 Participate in the working group on the counterterrorism information sharing standard sponsored by the Program Manager for the Information Sharing Environment Working Group on Director of National Intelligence Common Terrorism Information Sharing Standards.
8	System design may cause network bandwidth to be exceeded at dis-advantaged locations.	Users will experience poor performance e.g., response time, timeouts, etc. This will trigger a need to for performance engineering to investigate and isolate the causes of the performance problem and then actions taken to remedy the causes	2	M1 Reinitiate task to perform end to end analysis of network performance to provide early indication of potential bandwidth problems.
9	Integration and implementation of enterprise COTS components requires significantly more developed software than estimated in LM proposal and planned for in Phase 2.	Development and integration tasks and sufficient contractor resources do not allow for needed software development. Re-planning and re-staffing needed to bring required software development resources to bear.	2	Ensure that an accurate estimate is prepared of the software needed for integration and implementation of COTS components. Track for each major enterprise COTS component and refine estimates as needed. Ensure alignment of schedules with estimate of software development needed. Ensure that project has staff with appropriate software development skills.
10	Contractor Team does not have adequate staff with specialized skills in complex enterprise COTS components assigned to Sentinel. Consultants with Top Secret clearances expected to be in short supply.	Problems such as schedule slippage, poor design, and integration issues occur with COTS design, integration, and implementation.	2	Obtain benchmarks from comparable programs e.g., EPA on number and level of consultants needed for each major enterprise COTS. Ensure that skills for major enterprise COTS for Phase 2 are consistent with these benchmarks. Monitor and track needed staffing from consultants.
Rank	Risk Condition	Risk Consequence	Impact Phase	Mitigation Strategy
The FBI did not rank Risks after # 10	User requirements may change significantly as a result of the BPR initiative and impact Sentinel’s schedule and budget.	Funding and schedule will not support project completion.	1	M1 Place the SRS under configuration control prior to RFP release. M2 Maintain strict requirements and configuration controls throughout the project. M3 Ensure user advocacy group is the focal point for all user changes / needs. M4 Ensure contractors are aware and adhere to change process, including communication with user community. M5 Ensure core FBI capabilities are addressed early in system development. M6 Include user community earlier in requirements clarification; Ensure continuous feedback with user community. M7 Concurrence of SRS contents to be achieved by each division. M8 Review SRS and add “Mandatory or Desired” for each requirement.
The FBI did not rank Risks after # 10	Privacy Impact Assessment (PIA) requirements impact cost and schedule.	Cost and schedule could expand to accommodate new requirements.	2	M1 Work with the Office of General Counsel (OGC) to define the hard system requirements and verify against the SRS, include OGC personnel in high level design meetings, so they can understand what / how various data elements are being used. M2 Work with the OGC and the Office of the Director of National Intelligence to accommodate interim, best guess requirements; comply with RFC process as requirements firm up. M3 Document the OGC and the Office of the Director of National Intelligence guidance through use of electronic communications.
The FBI did not rank Risks after # 10	Sentinel program office and prime contractor may periodically suffer shortfalls of human capital due to availability of properly cleared and skilled staff.	Consequence 1 – Higher costs associated with tight labor market could exceed the current contract cost with Lockheed Martin/PMO. Consequence 2 – Non-availability of staff could negatively impact work to be accomplished as per the schedule timeline.	1	M1 Develop and maintain a well conceived staffing plan. M2 Maintain job descriptions for all PMO positions. M3 Identify critical positions within the PMO and temporary skills needed as surge expertise. M4 Create succession plan for each critical position. M5 Identify and document processes, policies, and procedures for all units for employees to follow. M6 Cross-train employees and identify back-ups for important tasks, skills, functions, and roles. M7 Identify skill requirements for inclusion in the OCIO strategic staffing plan.
	Lockheed Martin may periodically suffer shortfalls of human capital due to availability of properly cleared and skilled staff.	Consequence 1– Higher costs assoc with tight labor market could exceed current contract cost. Consequence 2 – Non-availability of staff could neg. impact work to be accomplished per schedule timeline.	2	M1 Develop a well conceived staffing plan, defining the staff’s skill requirements, associated contractor staffing levels, and actions for filling positions, and aligning it with the personnel needs identified in the staffing plan and the proposal. M2 Thoroughly vet contractor resumes, selecting an integrated team of subject matter experts from systems engineering and technical assistance contractors from the Lockheed Martin team. M3 Use corporate “reach back” for surges by assigning personnel TDY when necessary.
Rank	Risk Condition	Risk Consequence	Impact Phase	Mitigation Strategy
The FBI did not rank Risks after # 10	Activities related to data cleansing of data from phased-out legacy systems may have been under estimated.	1. The target word search will take an inordinate amount of time both computer and admin. There are 23 million docs in ACS; 19 percent WordPerfect, 80 percent unknown (assumed WordPerfect), and about 1 percent miscellaneous, some in obsolete formats. Requires scanning 13,000 docs per day 7 days a week for 6 months. 2. Requires government-furnished Data Staging facility (in FBI facility with certification and accreditation complete and Oracle 10G with real application clusters installed). 3. Requires significant contractor access to data center. 4. Current approach (Dynamic Extract vs. Backup Tapes) requires significant effort in Clarksburg, WV. 5. Placing cleansed data back into the legacy data base may impact those continuing to use legacy applications. 6. Need to maintain security control of data in staging area (data will not be protected by ACS or Sentinel access control).	1,2	Consequence #1 M1 Re-evaluate target word search requirements. M2 Define and incremental migration schedule. M3 Capitalize on document scans currently being done for export to IDW, RDex, OCEDETF, etc. These are the most current and relevant documents. M4 Start document scan effort early. M5 Scan new documents as they enter ACS. M6 Benchmark target word tools and throughput in Phase 1. Consequence #2 M1 Start assembly of development facility in Phase 1. Consequence #3 M1 Re-evaluate target word search requirements. M2 LM apply for SCI clearances now. Consequence #4 M1 Complete Phase 1 benchmarking to validate approaches. Consequence #5 M1 Data Migration Trade Study and Data Migration Plan agree data will not be put directly back in to ACS. Quality Assessment Report will allow cleansing in ACS by ITOD and data re-exported. Multiple dry runs to enhance data. Consequence #6 M1 Cleansing to be done only in FBI facility. M2 Access limited to select group read into “process”. FBI only?
The FBI did not rank Risks after # 10	Heavy reliance on COTS products that don’t provide the seamless, integrated environment promised by the vendor.	Major schedule slips. A system that is not secure. More custom coding than the contractor expects.	2	Explore other vendor products. Find subcontractors that know technical details of the IBM products suite.
	PMO does not have adequate staff with specialized skills in complex enterprise COTS. Consultants with Top Secret clearances expected to be in short supply.	Problems such as schedule slippage, poor design, and integration issues occur with COTS design, integration, and implementation.	2	Obtain benchmarks from comparable programs e.g., EPA on number and level of consultants needed for each major enterprise COTS. Ensure that skills for major enterprise COTS for Phase 2 are consistent with these benchmarks. Monitor and track needed staffing for consultants.

Source: FBI

Footnotes

No risk was ranked #2.