The Federal Bureau of Investigation’s Efforts to Hire, Train, and Retain Intelligence Analysts

Audit Report 05-20
April 2005
Office of the Inspector General


Appendix 1

Objectives, Scope, and Methodology


Objectives

The primary objective of the audit was to determine how effectively the FBI recruits, selects, trains, and staffs the various categories of intelligence analysts and reports officers. As a part of accomplishing this objective we reviewed: 1) analyst hiring requirements; 2) progress made toward meeting analyst hiring goals; 3) progress made toward establishing a comprehensive training program and meeting the training goals; 4) analyst staffing and utilization to support the FBI’s mission; and 5) progress toward retaining analysts.

Scope and Methodology

The audit was performed in accordance with the Government Auditing Standards, and included tests and procedures necessary to accomplish the audit objectives. We conducted work at the FBI Headquarters in Washington, DC, and six FBI field offices: Albany, NY; Baltimore, MD; Newark, NJ; Philadelphia, PA; Pittsburgh, PA; and Washington, DC. In general, our audit data covered October 1, 2001 through July 8, 2004.

To conduct our audit, we interviewed officials from the FBI, including intelligence analysts. In addition, we interviewed officials from other intelligence agencies. The FBI officials interviewed were from the Office of Intelligence, the Counterterrorism, Counterintelligence, Criminal Investigative and Cyber Divisions, as well as the Administrative Services and Finance Divisions. The officials from other agencies included the Central Intelligence Agency, the Defense Intelligence Agency, and the Joint Military Intelligence College. In addition, we reviewed documents related to the budgeting, hiring, training, utilization and retention of intelligence analysts including various Concepts of Operations published by the Office of Intelligence, budget documentation, organizational structures, congressional testimony, and prior GAO and OIG reports.

To determine how the FBI determines its requirements for intelligence analysts and allocates intelligence analysts, we examined the methodologies the FBI employed to determine the number of intelligence analysts needed by the FBI, the number of additional analysts requested in its FYs 2003 and 2004 budgets, and its current and future allocation of intelligence analysts. We accomplished this by reviewing internal documentation maintained by the Finance Division and the Threat Forecasting and Operational Requirements CONOPS and interviewing officials from the Office of Intelligence and the Finance and Administrative Services Divisions.

To determine the progress the FBI has made in hiring intelligence analysts, we examined hiring data for both internal and external hires, background investigation data, and educational data. We also analyzed demographic data gathered from our survey of intelligence analysts. (A detailed discussion of our survey of intelligence analysts is included below.) The FBI does not maintain much of the demographic data we needed for this audit, such as prior military intelligence experience, prior experience in the United States Intelligence Community, and travel outside the United States for a period of six months or longer. As a result, we relied on a survey to gather this information. In addition, we interviewed officials from the Office of Intelligence about the automated application system now being used by the FBI. We also interviewed hiring officials at the FBI about the system the FBI previously used to hire intelligence analysts.

To determine the progress the FBI has made in providing introductory training to intelligence analysts, we examined curricula for the Basic Intelligence Analyst (BIA) course and the Analytical Cadre Educational Strategy 1 course, attendance data for the BIA, and student feedback on the BIA. To obtain the perspective of intelligence analysts who have attended the BIA, we interviewed selected analysts who took the course. In addition, we collected data on the BIA in our survey of intelligence analysts concerning the topics covered by the course, suggestions for improvement, and the BIA’s ability to prepare intelligence analysts to do their job. We also interviewed CIA and DIA officials concerning those agencies’ approaches to training intelligence analysts.

To determine how FBI intelligence analysts are being utilized, we interviewed intelligence analysts at headquarters and six field offices. In addition, our survey of FBI intelligence analysts included a series of questions about their work and perceptions of their work.

To determine the progress the FBI has made in retaining highly qualified and productive intelligence analysts, we examined the Human Talent CONOPS and attrition data. We also interviewed officials from the Office of Intelligence. To determine whether FBI intelligence analysts plan on staying with the FBI as intelligence analysts, we included appropriate questions in our survey of FBI intelligence analysts.

Survey of FBI Intelligence Analysts

As mentioned above, to meet the objectives of our audit we requested all FBI intelligence analysts to complete an on-line survey. The methods we used in the survey are described below.

Our survey was conducted using an anonymous Internet-based survey. Not all FBI intelligence analysts have public FBI e-mail addresses. As a result, we could not directly e-mail our survey notification to the intelligence analysts. Instead, we notified them of the survey through the FBI Inspection Division and the FBI Office of Intelligence. Using the FBI’s internal e-mail system, the Inspection Division sent an e-mail to all intelligence analysts notifying them of the survey and instructing them how to access it. The Office of Intelligence posted the same notification and instructions on its FBI intranet site. In addition, the Office of Intelligence sent e-mail reminders to the analysts asking them to complete the survey. Analysts accessed the survey using a special Internet address dedicated to the survey. The survey itself was password protected. Through this method, we obtained 817 usable responses from 1,247 intelligence analysts employed by the FBI at the time we launched our survey, a response rate of 66 percent.

The practical difficulties of conducting any survey introduce various types of errors related to survey responses. For example, differences in how a particular question is interpreted and differences in the sources of information available to respondents can be sources of error. In addition, respondents might not be uniformly conscientious in expressing their views or they may be influenced by concerns about how their answers might be viewed by the OIG, the FBI, or the public. We included steps intended to minimize such errors. For example, to address differences in how questions were interpreted, we pre-tested our survey with 16 intelligence analysts at FBI headquarters and 4 field offices. We modified our survey questions based on the results of these pre-tests. In addition, we solicited comments from the Office of Intelligence about the content and clarity of our survey. The Office of Intelligence did not provide any suggestions for improving the clarity or content of our survey.

When we analyzed the basic results of our survey, we verified the results we obtained using our survey software by exporting the data to another software program and performing the same analysis. Detailed results of our survey are contained in Appendix 7.



Previous Page Back to Table of Contents Next Page